Methods, systems, equipment, and media for ship outfitting path planning and simulation modeling.

Abstract

This application discloses a method, system, equipment, and medium for ship outfitting path planning and simulation modeling, belonging to the field of ship digital assembly technology. This application solves the problems of low path planning efficiency, insufficient accuracy of virtual-real fusion, and lack of dynamic collision warning in complex compartment environments, significantly reducing worker cognitive load, improving assembly qualification rate and collision warning accuracy. It is applicable to the digital outfitting operations of complex systems and equipment such as ship piping and shaft / rudder systems, providing key technical support for digital shipbuilding.

Description

Technical Field

[0001] This application relates to the field of digital ship assembly, and in particular to a method, system, equipment, and medium for ship outfitting path planning and simulation modeling. Background Technology

[0002] In recent years, with the trend towards larger and lighter ships, hull size and weight have gradually increased, while stiffness has relatively decreased. Furthermore, the increasing demands for refined ship design have led to growing requirements for the outfitting of ship systems and equipment. Outfitting of ship systems and equipment accounts for over 45% of the entire shipbuilding cycle, playing a crucial role in the overall shipbuilding process. Currently, outfitting operations often suffer from inefficient path planning in narrow passages, reliance on manual calibration and process information resulting in insufficient accuracy in virtual-to-real fusion, and a lack of dynamic collision warning and identification mechanisms under multiple constraints. Therefore, improper outfitting operations often lead to irreversible structural deformation and damage, as well as safety accidents, significantly impacting shipbuilding efficiency. Summary of the Invention

[0003] The purpose of this application is to provide a method, system, equipment, and medium for ship outfitting path planning and simulation modeling, which solves the problem of outfitting path planning and dynamic simulation of system equipment in complex ship cabin environments that is difficult to achieve with existing technologies. By guiding assembly through a combination of virtual and real methods, it significantly reduces the cognitive burden of workers on process information, thereby improving the efficiency and quality of system equipment assembly and reducing the error rate.

[0004] The solution adopted in this application is to provide a method for ship outfitting path planning and simulation modeling, including the following steps:

[0005] Collect environmental characteristic parameters of the ship system equipment outfitting site, and construct a consistent three-dimensional virtual environment through surface fitting and data alignment;

[0006] Based on multiple constraints such as obstacles, narrow passages and dynamic environment, the Narrow Passage Optimization Algorithm (NARRT*) is adopted. Based on the convex hull pose distance metric model, a six-degree-of-freedom path is dynamically planned. The optimal outfitting path of the ship system equipment is generated through hierarchical bounding box collision detection and B-spline path smoothing. Based on the optimal outfitting path, the dynamic simulation and collision warning of the outfitting process are realized by combining the three-dimensional visualization engine.

[0007] Industrial cameras, laser trackers, and optical targets are used to collect outfitting process data. A mapping relationship between characteristic deviations and the three-dimensional geometric model of the ship system equipment assembly is established. Long short-term memory neural networks are used to predict deviation trends and generate correction instructions to correct the optimal outfitting path.

[0008] Furthermore, the process of collecting environmental characteristic parameters of the ship system equipment outfitting site and constructing a consistent three-dimensional virtual environment through surface fitting and data alignment includes:

[0009] The site feature data is collected by a 3D laser scanner, industrial camera and depth sensor. The feature data includes at least one of the following: chimney section, side passage, seawater main unit, bulk carrier shaft and rudder system, oil tanker pipe bundle unit system, auxiliary positioning position, work space layout and equipment base.

[0010] Point and surface feature information is extracted based on surface fitting and data alignment methods;

[0011] Based on the collected feature data and extracted point and surface feature information, a product assembly information model, an assembly process information model, and an assembly path simulation information model are constructed through scene 3D rendering. The product assembly information model, assembly process information model, and assembly path simulation information model are then used to form a virtual scene of the outfitting site.

[0012] Multiple coordinate systems are defined, and the position and pose data of the ship system equipment are corrected in real time by identifying markers, thereby correcting the parameters of the virtual scene of the outfitting site and reconstructing a three-dimensional virtual environment consistent with the outfitting site.

[0013] Furthermore, the construction of a product assembly information model, an assembly process information model, and an assembly path simulation information model based on the collected feature data and extracted point and surface feature information through scene 3D rendering includes:

[0014] Extract and integrate geometric information, hierarchical structure information, basic attribute information and pose information of parts from the collected feature data, integrate the information using XML format files, create part node files, and form a product assembly information model in augmented reality scenarios;

[0015] Extract and integrate the task types, assembly processes and steps, operation objects, tools and resources, operation methods, assembly paths and prerequisites from the collected feature data, create XML files for the steps, refine the process steps, and form an assembly process information model.

[0016] The assembly object is obtained by using a planning algorithm. The assembly base environment, assembly motion path, dynamic constraints and interference requirements are used as the motion posture information of the parts. The motion details of the part assembly process are described based on the motion posture information of the parts, and an assembly path XML file is created to form an assembly path simulation information model.

[0017] Furthermore, the method employs the Narrow Channel Optimization (NARRT*) algorithm, which dynamically programs a six-degree-of-freedom path based on a convex hull pose distance metric model. The optimal outfitting path for the ship's system equipment is generated through hierarchical bounding box collision detection and B-spline path smoothing.

[0018] Mapping the ship outfitting workspace as a six-degree-of-freedom space The translational degrees of freedom are The rotational degrees of freedom are , These represent the x-axis direction, y-axis direction, and z-axis direction, respectively. These represent the rotational degrees of freedom in the x-axis direction, the y-axis direction, and the z-axis direction, respectively.

[0019] Define free space Obstacle space and narrow areas ;

[0020] Calculation of local obstacle density based on hypersphere sampling method Based on local obstacle density Dynamically adjust the sampling strategy; when When the value is greater than or equal to the first threshold, it is determined to be a narrow channel, and directional bias sampling is performed along the main axis of the narrow channel; when When the value is less than the first threshold, it is determined to be an open area, and uniform random sampling is performed within the open area;

[0021] The maximum displacement distance between poses is calculated based on the rigid body transformation of the convex hull vertex set, which serves as the distance metric for path planning. A smooth B-spline path is generated by bidirectional tree synchronous expansion and hierarchical bounding box collision detection, and the output includes a sequence of path key points containing pose sequence and time parameters.

[0022] Furthermore, the calculation of the maximum displacement distance between poses using rigid body transformation based on the convex hull vertex set, as a distance metric for path planning, includes:

[0023] By obtaining the three-dimensional geometric model of the ship system equipment assembly, the set of its convex hull vertices is calculated. ;in To perform a rigid body transformation on the nth pose, ;

[0024] definition and Two poses, perform rigid body transformation on the set of convex hull vertices. The rigid body transformation matrix Including translation components and rotational components ; These represent the translation components along the x-axis, y-axis, and z-axis, respectively. These are the rotation components along the x-axis, y-axis, and z-axis, respectively.

[0025] Calculate the maximum and average displacement distances of the corresponding vertices. , the maximum displacement distance As a position and The distance between them.

[0026] Furthermore, the dynamic simulation and collision warning of the outfitting process based on the optimal outfitting path combined with a 3D visualization engine includes:

[0027] Based on the 3D visualization engine, the 3D geometric model and planned path data of the ship system equipment assembly are loaded, and the motion trajectory and key pose points of the assembly are rendered in real time. When the minimum distance is detected to be less than the safety threshold, a collision warning is triggered.

[0028] The 3D visualization engine highlights risk areas and automatically generates path correction suggestions, updating the optimal outfitting path.

[0029] Furthermore, the process of acquiring outfitting process data using industrial cameras, laser trackers, and optical targets, establishing a mapping relationship between characteristic deviations and the three-dimensional geometric model of the ship system equipment assembly, and predicting deviation trends and generating correction instructions to correct the optimal outfitting path through a long short-term memory neural network includes:

[0030] A multi-sensor network is formed by deploying industrial cameras, laser trackers, and optical targets, setting up multi-view observation stations, and planning the spatial distribution of the target array.

[0031] The three-dimensional coordinates of feature points are calculated using bundle adjustment, and the transformation relationship between the global coordinate system and the local coordinate system is established.

[0032] Extract the measured pose data of key control points, calculate the deviation value from the theoretical model, and establish the mapping relationship between the deviation field and the three-dimensional geometric model of the ship system equipment assembly;

[0033] Radial basis function interpolation is used to display the deviation distribution heatmap in real time, and a long short-term memory neural network is used to predict the deviation development trend and generate correction instructions. The optimal outfitting path is corrected according to the correction instructions.

[0034] This application also provides a ship outfitting path planning and simulation modeling system, including:

[0035] The data acquisition and virtual modeling module is used to collect environmental characteristic parameters of the ship system equipment outfitting site, and to construct a consistent three-dimensional virtual environment through surface fitting and data alignment.

[0036] The path planning and simulation module is used to dynamically plan a six-degree-of-freedom path based on multiple constraints such as obstacles, narrow passages and dynamic environment, using the narrow passage optimization algorithm (NARRT*) and the convex hull pose distance metric model. It generates the optimal outfitting path for ship system equipment through hierarchical bounding box collision detection and B-spline path smoothing. Based on the optimal outfitting path, it combines with a 3D visualization engine to realize dynamic simulation and collision warning of the outfitting process.

[0037] The deviation detection and correction module is used to collect outfitting process data using industrial cameras, laser trackers and optical targets, establish the mapping relationship between characteristic deviations and the three-dimensional geometric model of the ship system equipment assembly, predict deviation trends through long short-term memory neural networks and generate correction instructions to correct the optimal outfitting path.

[0038] Furthermore, the path planning and simulation module includes:

[0039] Narrow passage determination unit, used to calculate local obstacle density based on hypersphere sampling method. Based on local obstacle density Dynamically adjust the sampling strategy;

[0040] The convex hull pose distance calculation unit is used to calculate the maximum displacement distance between poses based on the rigid body transformation of the convex hull vertex set, and serves as a distance metric for path planning.

[0041] The path smoothing unit is used to generate a smooth B-spline path using bidirectional tree synchronous expansion and hierarchical bounding box collision detection, and outputs a path key point sequence containing pose sequence and time parameters.

[0042] Furthermore, the deviation detection and correction module includes:

[0043] The multi-sensor calibration unit is used to deploy industrial cameras, laser trackers and optical targets to form a multi-sensor network, set up multi-view observation stations and plan the spatial distribution of the target array.

[0044] The deviation heatmap generation unit is used to display the deviation distribution heatmap in real time through radial basis function interpolation.

[0045] The warning level classification unit is used to predict the development trend of deviations based on long short-term memory and output warning signals including normal, attention, and danger levels.

[0046] This application also provides a computer device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the steps of the method described above.

[0047] This application also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the method described above.

[0048] This application solves the problems of low path planning efficiency, insufficient accuracy of virtual-real fusion, and lack of dynamic collision warning in complex cabin environments. It significantly reduces the cognitive load on workers, improves the assembly qualification rate and collision warning accuracy, and is applicable to the digital outfitting operation of complex system equipment such as ship pipelines and shaft and rudder systems, providing key technical support for digital shipbuilding. Attached Figure Description

[0049] Figure 1 Construct a logic diagram for the virtual scene of outfitting site in this application;

[0050] Figure 2 This is a flowchart of the NARRT* algorithm in this application;

[0051] Figure 3 This is a diagram of the multi-sensor network and deviation tracking in this application;

[0052] Figure 4 This is a schematic diagram of the ship outfitting path planning and simulation modeling method in this application. Detailed Implementation

[0053] The technical solutions of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0054] This embodiment takes the chimney section outfitting operation in a large container ship construction project as an example to illustrate the specific implementation method of this application in detail.

[0055] Example 1

[0056] This embodiment provides a method for ship outfitting path planning and simulation modeling based on multi-constraint virtual-real fusion, which is implemented according to the following steps:

[0057] Step 1: Collect environmental feature parameters of the ship system equipment outfitting site, extract point and surface feature information through surface fitting and data alignment, correct the virtual environment parameter model of the outfitting site, and construct a three-dimensional virtual outfitting environment based on key features that is consistent with the outfitting site.

[0058] Step 2: Combining multiple constraints such as obstacles, narrow passages, and dynamic environment, the optimal outfitting sequence and path of the ship's system equipment are generated based on the Narrow Passage Optimization Algorithm (NARRT*). The outfitting process is verified through three-dimensional visualization simulation, and potential collision problems that may occur during the outfitting process are visualized, analyzed, and warned.

[0059] Among them, based on multiple constraints such as obstacles, narrow passages and dynamic environment, the Narrow Passage Optimization Algorithm (NARRT*) is used to generate the optimal outfitting path sequence of ship system equipment. Through hierarchical bounding box collision detection and B-spline path smoothing, combined with a 3D visualization engine, dynamic simulation and collision warning of the outfitting process are realized.

[0060] Step 3: Collect outfitting process data using industrial cameras, laser trackers, and optical targets to form a set of feature parameters for the outfitting targets of the ship system equipment. Reconstruct a three-dimensional model of the ship system equipment. By establishing a mapping relationship between feature measurement deviations and the three-dimensional system model, and by using a long short-term memory neural network to predict deviation trends, the system equipment outfitting deviation information can be visualized, tracked, and intelligently predicted and corrected.

[0061] like Figure 1 As shown, step 1 specifically involves: First, using a 3D laser scanner, industrial camera, and depth sensor to collect feature data of the outfitting site, and extracting point and surface feature information by combining surface fitting and data alignment methods; Second, based on the collected data, constructing a virtual scene of the outfitting site through scene 3D rendering, including a product assembly information model, an assembly process information model, and an assembly path simulation information model; Finally, employing a hybrid tracking technology based on natural features and multi-sensor fusion, defining multiple coordinate systems and describing the position and pose of ship system equipment, and using marker identification to correct the pose data in real time, improving model accuracy, correcting the virtual environment parameters of the ship outfitting site, and reconstructing a 3D virtual environment consistent with the outfitting site.

[0062] Furthermore, the on-site feature data includes container ship chimney sections, side passages, seawater main pipe units, bulk carrier shaft and rudder systems, oil tanker pipe bundle unit systems, etc., and also includes key features such as auxiliary positioning positions, work space layout, and equipment bases.

[0063] Furthermore, the product assembly information model is further developed by CAD software to extract and integrate information such as the geometric shape and hierarchical structure of parts, and integrates information using XML format files to create part node files, forming a product assembly information model in an augmented reality scenario; the model information includes part geometric information, hierarchical structure information, basic attribute information and pose information.

[0064] Optionally, the component geometry information includes the three-dimensional geometric shape, patch model, and wireframe model of each component of the ship system equipment;

[0065] Optionally, the hierarchical structure information includes the parent-child hierarchical relationship between various components of the ship system equipment and the assembly constraints between components;

[0066] Optionally, the basic attribute information includes the name, number, material properties (density, elastic modulus, coefficient of thermal expansion, etc.), mass and center of gravity of each component of the ship system equipment, outfitting process requirements, etc.

[0067] Optionally, the pose information includes the initial pose, target pose, relative pose, absolute pose, and bounding box center pose of each component of the ship system equipment.

[0068] Furthermore, the assembly process information model is developed by secondarily using CAPP software to extract and integrate key information such as task types and procedures in the assembly process, create XML files for each step, refine the specific process, and form an AR-guided assembly process information model. The model information includes assembly procedures and steps, operation objects, tools and resources, operation methods, assembly paths, and prerequisites.

[0069] Optionally, the process and steps include the breakdown of assembly tasks such as the installation base of the ship system equipment and the positioning bolt holes;

[0070] Optionally, the objects of operation include part numbers, accessories, etc.

[0071] Optionally, tools and resources include wrench type, crane load capacity, manpower allocation, etc.

[0072] Optionally, the operation method includes a description of the assembly action, precautions such as collision avoidance sensors;

[0073] Optionally, the assembly path and prerequisites include motion trajectory planning documents for various components and preliminary steps.

[0074] Furthermore, the assembly path simulation information model is formed by secondary development of CAD software, using planning algorithms to automatically plan and obtain the motion posture information of parts, describe the motion details of the parts assembly process, create part assembly path XML files, and form an assembly path simulation information model; the model information includes assembly objects, assembly base environment, assembly motion path, dynamic constraints and interference requirements.

[0075] Optionally, the assembly base environment includes a list of assembled parts, obstacles, workspace constraints, etc.

[0076] Optionally, the assembly motion path includes initial pose, target pose, motion path key point sequence, motion duration, motion speed, etc.

[0077] Optionally, dynamic constraints include degrees of freedom restrictions, equipment capabilities, etc.

[0078] Optionally, the intervention requirements may include collision detection rules, safety warning thresholds, etc.

[0079] like Figure 2 As shown, the NARRT* algorithm in step 2 includes:

[0080] Mapping the ship outfitting workspace as a six-degree-of-freedom configuration space Define free space Obstacle space and narrow areas ;

[0081] Calculation of local obstacle density based on hypersphere sampling method ,when If the channel is identified as narrow, directional offset sampling is performed along the main axis of the channel.

[0082] Based on convex hull Rigid body transformation calculation of maximum displacement distance between poses , as a distance metric for path planning;

[0083] A smooth B-spline path is generated using bidirectional tree synchronous expansion and hierarchical bounding box collision detection, and the output is an XML file containing pose sequence and time parameters.

[0084] Further, step 2 specifically involves: First, constructing a three-dimensional configuration space model for ship outfitting and defining narrow passage areas within its workspace; second, calculating the pose distance of the assembly in the configuration space based on a convex hull pose distance metric model; then, using the NARRT* algorithm to perform path search in the configuration space, dynamically adjusting the sampling strategy according to the regional environment type, verifying path feasibility through a hierarchical bounding box collision algorithm, thereby generating the optimal outfitting path sequence and performing path smoothing; finally, implementing dynamic simulation of the outfitting process based on a three-dimensional visualization engine, monitoring potential collision risks in real time, and generating early warning information.

[0085] Furthermore, the configuration space model maps the ship outfitting workspace into a six-degree-of-freedom space, including translational degrees of freedom. and rotational degrees of freedom Meanwhile, the free space is defined as Obstacle space and narrow areas Obstacle density in narrow passage areas is calculated using the hypersphere sampling method. Make a judgment.

[0086] Furthermore, the convex hull-based pose distance metric model includes, by acquiring the three-dimensional geometric model of the ship system equipment assembly under test, calculating the set of convex hull vertices. ;definition and Two poses, perform rigid body transformation on the set of convex hull vertices. The rigid body transformation matrix Including translation components and rotational components ; Calculate the maximum and average displacement distances of the corresponding vertices. , the maximum displacement distance As a position and The distance between them.

[0087] Furthermore, the NARRT* algorithm, an improvement on the classic RRT algorithm, employs a sensitive adaptive sampling strategy in the ship outfitting workspace, particularly in open areas. Uniform random sampling is used in narrow regions. Oriented offset sampling is performed along the main axis of the channel. Based on the convex hull model, the pose distance is accurately measured. At the same time, combined with the dynamic environment classification mechanism, the initial tree and the target tree are expanded in both directions synchronously. The hierarchical bounding box is used for real-time collision detection to complete the dynamic path optimization, generate a smooth B-spline path curve, and output an XML file containing pose sequence and time parameters.

[0088] Furthermore, the outfitting process dynamic simulation loads the three-dimensional geometric model of the ship system equipment assembly and the planned path data based on the three-dimensional visualization engine, renders the motion trajectory and key pose points of the assembly in real time, and triggers a collision warning and highlights the risk area when the minimum distance is detected to be less than the safety threshold, automatically generates path correction suggestions and updates the path planning scheme.

[0089] That is, the dynamic simulation includes:

[0090] Real-time rendering of assembly motion trajectory and key pose points; triggering collision warning when the minimum spacing is detected to be less than the safety threshold.

[0091] Risk areas are highlighted using a 3D visualization engine, and path correction suggestions are automatically generated.

[0092] like Figure 3As shown, step 3 specifically involves: deploying industrial cameras, laser trackers, and optical targets to form a multi-sensor network; designing multi-view observation stations; planning the spatial distribution of the target array; calculating the three-dimensional coordinates of feature points using bundle adjustment; establishing the transformation relationship between the global coordinate system and the local coordinate system; extracting the measured pose data of key control points; calculating the deviation value from the theoretical model; correcting the integrated information model based on the measured data; establishing the mapping relationship between the deviation field and the three-dimensional model; and displaying the deviation heatmap in real time using visualization methods and predicting the deviation development trend based on machine learning algorithms.

[0093] Furthermore, the industrial camera, optical target, and laser tracker are arranged with a spacing of no more than 5 meters, the optical target is made of a high reflectivity material, and the laser tracker has a measurement accuracy of 0.01 mm / m.

[0094] Furthermore, the mapping between the deviation field and the three-dimensional model is achieved by using radial basis function interpolation, resulting in a matching error of less than 0.5 mm between the reconstructed model and the theoretical model; that is, the deviation field mapping is constructed using radial basis function interpolation, with a matching error of less than 0.5 mm.

[0095] Furthermore, the deviation prediction uses a long short-term memory neural network for deviation prediction. The prediction time span can be set from 1 to 24 hours. That is, the deviation trend is predicted from 1 to 24 hours based on the long short-term memory neural network, and the warning signal including normal, attention and danger levels is output.

[0096] The beneficial effects of this application are as follows: This application achieves efficient path planning in narrow passage environments through the NARRT* algorithm, improves planning accuracy by using convex hull pose distance measurement, and constructs a deviation prediction model by combining multi-sensor data, significantly improving assembly efficiency and quality. This method is compatible with mainstream CAD / CAPP systems, achieves dynamic simulation of the outfitting process through a 3D visualization engine, supports six-degree-of-freedom motion simulation and AR / VR interaction, realizing "what you see is what you get" assembly guidance, establishing a complete "planning-simulation-detection-prediction" technical route, and creating a fully digital closed loop from virtual planning to actual assembly. This significantly improves the success rate of path planning, the accuracy of collision detection and early warning, and the precision of deviation detection, reduces assembly error rate, and reduces the cognitive burden on workers. It is applicable to the outfitting operations of complex systems and equipment such as pipelines and chimneys in shipbuilding, providing key technical support for digital shipbuilding.

[0097] Example 2

[0098] Embodiment 2 includes all the technical features of Embodiment 1. In Embodiment 2, a three-dimensional virtual environment for ship outfitting is constructed, and the specific implementation includes the following steps:

[0099] (1) Data Acquisition: Using a 3D laser scanner (scanning accuracy ±2mm), an industrial camera (resolution 2592×1944 pixels), and a depth sensor, comprehensive data was collected from the outfitting site of the container ship's chimney sections. The collected data included the chimney section's own structure, surrounding side passages, installed seawater main pipe units and other equipment, as well as key feature data such as auxiliary positioning positions, work space layout, and equipment bases.

[0100] (2) Feature extraction and model construction: The collected data is imported into MATLAB software, and the complex surface is processed by the surface fitting algorithm. Combined with the data alignment method based on ICP (Iterative Closest Point), the point and surface feature information is extracted. Based on the extracted features, the AutoCAD software is further developed to extract the geometric shape (3D geometric shape, patch model, wireframe model), hierarchical structure and other information of the chimney segment components. The XML format files are integrated to create component node files and form a product assembly information model. The CAPP software is further developed to extract and integrate key information such as task types and processes (such as the assembly task decomposition of mounting base and positioning bolt holes) in the assembly process. XML files are created for the process steps to refine the process and build an assembly process information model. The AutoCAD secondary development is used again to automatically plan and obtain the motion posture information of the components using the Dijkstra planning algorithm to describe the motion details of the assembly process, create component assembly path XML files, and form an assembly path simulation information model.

[0101] (3) Model Correction: A hybrid tracking technology based on natural features and multi-sensor fusion is adopted to define multiple coordinate systems, including the world coordinate system, equipment coordinate system, and sensor coordinate system. AR markers are placed in the chimney sections and surrounding areas. The markers are identified by industrial cameras, and the pose data is corrected in real time. The collected real-time data is used to correct the constructed virtual scene to ensure that the virtual environment parameters are consistent with the actual outfitting site, thus completing the construction of the three-dimensional virtual environment.

[0102] The process of generating the optimal outfitting sequence and path includes the following steps:

[0103] (1) Configuration space model construction: The chimney segment outfitting workspace is mapped into a six-degree-of-freedom space, where the translational degree of freedom is 1. The rotational degrees of freedom are Define free space Obstacle space and narrow areas The hypersphere sampling method was used, with the chimney segments as the center, to sample within a 5-meter radius and calculate the obstacle density in the local area. Calculations show that the area surrounding the segmented chimney... It was determined to be a narrow passage area.

[0104] (2) Pose distance calculation: Obtain the three-dimensional geometric model of the chimney segment assembly, and use the convex hull generation function of SolidWorks software to calculate the set of convex hull vertices. .definition and Two poses, perform rigid body transformation on the set of convex hull vertices. The rigid body transformation matrix Including translation components and rotational components Calculate the maximum displacement distance of the corresponding vertex. and average displacement distance ,by As a position and The distance between them.

[0105] (3) Path search and optimization: The NARRT* algorithm is used for path search. In open areas... Using a uniform random sampling strategy, in narrow areas Oriented offset sampling is performed along the main axis of the channel. The initial tree and target tree are expanded synchronously in both directions, and collision detection is performed in real time using hierarchical bounding boxes (OBB bounding boxes). After multiple iterations of optimization, a smooth B-spline path curve is generated, and an XML file containing the pose sequence and time parameters is output.

[0106] (4) Dynamic simulation and early warning: Based on the Unity 3D visualization engine, the 3D model of the chimney segment assembly and the planned path data are loaded, and the motion trajectory and key pose points of the assembly are rendered in real time. The safety threshold is set to 200mm. When the minimum distance is detected to be less than the safety threshold, a collision warning is triggered, the risk area is highlighted in red, and path correction suggestions are automatically generated and the path planning scheme is updated.

[0107] The deviation tracking and prediction correction process includes the following steps:

[0108] (1) Sensor network layout: An industrial camera is placed every 5 meters around the chimney segment outfitting area, for a total of 8 cameras; a laser tracker (measurement accuracy 0.01 mm / m) is installed; 20 optical targets made of high reflectivity material are attached to key parts of the chimney segment to form a multi-sensor network. Four multi-view observation stations are designed to ensure comprehensive observation of all parts of the chimney segment.

[0109] (2) Data Processing and Deviation Calculation: The three-dimensional coordinates of feature points are calculated using bundle adjustment, establishing the transformation relationship between the global and local coordinate systems. Measured pose data of key control points are collected using a laser tracker and industrial camera, compared with the theoretical model, and the deviation value is calculated. For example, at a certain key control point, the measured… The coordinates are 12.345m, theoretically... If the coordinates are 12.350m, then The directional deviation value is -0.005m.

[0110] (3) Model Correction and Prediction: The integrated information model is corrected based on measured data. Radial basis function interpolation is used to establish the mapping relationship between the deviation field and the three-dimensional model, ensuring that the matching error between the reconstructed model and the theoretical model is less than 0.5 mm. A long short-term memory neural network model is built using Python. Deviation data from the past 6 hours are input, the prediction time span is set to 12 hours, and the deviation warning level (normal, caution, danger) is output. When the deviation of a certain part is predicted to exceed the allowable range, a warning is issued to the construction personnel in a timely manner to guide them to make adjustments and ensure the high-precision completion of the chimney segment outfitting operation.

[0111] Example 3

[0112] Embodiment 3 includes all the technical features of Embodiment 2. For example... Figure 4 As shown in Embodiment 3, a method for ship outfitting path planning and simulation modeling is provided, including the following steps:

[0113] Collect environmental characteristic parameters of the ship system equipment outfitting site, and construct a consistent three-dimensional virtual environment through surface fitting and data alignment;

[0114] Based on multiple constraints such as obstacles, narrow passages and dynamic environment, the Narrow Passage Optimization Algorithm (NARRT*) is adopted. Based on the convex hull pose distance metric model, a six-degree-of-freedom path is dynamically planned. The optimal outfitting path of the ship system equipment is generated through hierarchical bounding box collision detection and B-spline path smoothing. Based on the optimal outfitting path, the dynamic simulation and collision warning of the outfitting process are realized by combining the three-dimensional visualization engine.

[0115] Industrial cameras, laser trackers, and optical targets are used to collect outfitting process data. A mapping relationship between characteristic deviations and the three-dimensional geometric model of the ship system equipment assembly is established. Long short-term memory neural networks are used to predict deviation trends and generate correction instructions to correct the optimal outfitting path.

[0116] Furthermore, the process of collecting environmental characteristic parameters of the ship system equipment outfitting site and constructing a consistent three-dimensional virtual environment through surface fitting and data alignment includes:

[0117] The site feature data is collected by a 3D laser scanner, industrial camera and depth sensor. The feature data includes at least one of the following: chimney section, side passage, seawater main unit, bulk carrier shaft and rudder system, oil tanker pipe bundle unit system, auxiliary positioning position, work space layout and equipment base.

[0118] Point and surface feature information is extracted based on surface fitting and data alignment methods;

[0119] Based on the collected feature data and extracted point and surface feature information, a product assembly information model, an assembly process information model, and an assembly path simulation information model are constructed through scene 3D rendering. The product assembly information model, assembly process information model, and assembly path simulation information model are then used to form a virtual scene of the outfitting site.

[0120] Multiple coordinate systems are defined, and the position and pose data of the ship system equipment are corrected in real time by identifying markers, thereby correcting the parameters of the virtual scene of the outfitting site and reconstructing a three-dimensional virtual environment consistent with the outfitting site.

[0121] Furthermore, the construction of a product assembly information model, an assembly process information model, and an assembly path simulation information model based on the collected feature data and extracted point and surface feature information through scene 3D rendering includes:

[0122] Extract and integrate geometric information, hierarchical structure information, basic attribute information and pose information of parts from the collected feature data, integrate the information using XML format files, create part node files, and form a product assembly information model in augmented reality scenarios;

[0123] Extract and integrate the task types, assembly processes and steps, operation objects, tools and resources, operation methods, assembly paths and prerequisites from the collected feature data, create XML files for the steps, refine the process steps, and form an assembly process information model.

[0124] The assembly object is obtained by using a planning algorithm. The assembly base environment, assembly motion path, dynamic constraints and interference requirements are used as the motion posture information of the parts. The motion details of the part assembly process are described based on the motion posture information of the parts, and an assembly path XML file is created to form an assembly path simulation information model.

[0125] Furthermore, the method employs the Narrow Channel Optimization (NARRT*) algorithm, which dynamically programs a six-degree-of-freedom path based on a convex hull pose distance metric model. The optimal outfitting path for the ship's system equipment is generated through hierarchical bounding box collision detection and B-spline path smoothing.

[0126] Mapping the ship outfitting workspace as a six-degree-of-freedom space The translational degrees of freedom are The rotational degrees of freedom are , These represent the x-axis direction, y-axis direction, and z-axis direction, respectively. These represent the rotational degrees of freedom in the x-axis direction, the y-axis direction, and the z-axis direction, respectively.

[0127] Define free space Obstacle space and narrow areas ;

[0128] Calculation of local obstacle density based on hypersphere sampling method Based on local obstacle density Dynamically adjust the sampling strategy; when When the value is greater than or equal to the first threshold, it is determined to be a narrow channel, and directional bias sampling is performed along the main axis of the narrow channel; when When the value is less than the first threshold, it is determined to be an open area, and uniform random sampling is performed within the open area;

[0129] The maximum displacement distance between poses is calculated based on the rigid body transformation of the convex hull vertex set, which serves as the distance metric for path planning. A smooth B-spline path is generated by bidirectional tree synchronous expansion and hierarchical bounding box collision detection, and the output includes a sequence of path key points containing pose sequence and time parameters.

[0130] Furthermore, the calculation of the maximum displacement distance between poses using rigid body transformation based on the convex hull vertex set, as a distance metric for path planning, includes:

[0131] By obtaining the three-dimensional geometric model of the ship system equipment assembly, the set of its convex hull vertices is calculated. ;in To perform a rigid body transformation on the nth pose, ;

[0132] definition and Two poses, perform rigid body transformation on the set of convex hull vertices. The rigid body transformation matrix Including translation components and rotational components ; These represent the translation components along the x-axis, y-axis, and z-axis, respectively. These are the rotation components along the x-axis, y-axis, and z-axis, respectively.

[0133] Calculate the maximum and average displacement distances of the corresponding vertices. , the maximum displacement distance As a position and The distance between them.

[0134] Furthermore, the dynamic simulation and collision warning of the outfitting process based on the optimal outfitting path combined with a 3D visualization engine includes:

[0135] Based on the 3D visualization engine, the 3D geometric model and planned path data of the ship system equipment assembly are loaded, and the motion trajectory and key pose points of the assembly are rendered in real time. When the minimum distance is detected to be less than the safety threshold, a collision warning is triggered.

[0136] The 3D visualization engine highlights risk areas and automatically generates path correction suggestions, updating the optimal outfitting path.

[0137] Furthermore, the process of acquiring outfitting process data using industrial cameras, laser trackers, and optical targets, establishing a mapping relationship between characteristic deviations and the three-dimensional geometric model of the ship system equipment assembly, and predicting deviation trends and generating correction instructions to correct the optimal outfitting path through a long short-term memory neural network includes:

[0138] A multi-sensor network is formed by deploying industrial cameras, laser trackers, and optical targets, setting up multi-view observation stations, and planning the spatial distribution of the target array.

[0139] The three-dimensional coordinates of feature points are calculated using bundle adjustment, and the transformation relationship between the global coordinate system and the local coordinate system is established.

[0140] Extract the measured pose data of key control points, calculate the deviation value from the theoretical model, and establish the mapping relationship between the deviation field and the three-dimensional geometric model of the ship system equipment assembly;

[0141] Radial basis function interpolation is used to display the deviation distribution heatmap in real time, and a long short-term memory (LSTM) neural network is used to predict the deviation development trend and generate correction instructions. The optimal outfitting path is corrected according to the correction instructions.

[0142] Example 4

[0143] Embodiment 4 of this application provides a ship outfitting path planning and simulation modeling system, including:

[0144] The data acquisition and virtual modeling module is used to collect environmental characteristic parameters of the ship system equipment outfitting site, and to construct a consistent three-dimensional virtual environment through surface fitting and data alignment.

[0145] The path planning and simulation module is used to dynamically plan a six-degree-of-freedom path based on multiple constraints such as obstacles, narrow passages and dynamic environment, using the narrow passage optimization algorithm (NARRT*) and the convex hull pose distance metric model. It generates the optimal outfitting path for ship system equipment through hierarchical bounding box collision detection and B-spline path smoothing. Based on the optimal outfitting path, it combines with a 3D visualization engine to realize dynamic simulation and collision warning of the outfitting process.

[0146] The deviation detection and correction module is used to collect outfitting process data using industrial cameras, laser trackers and optical targets, establish the mapping relationship between characteristic deviations and the three-dimensional geometric model of the ship system equipment assembly, predict deviation trends through long short-term memory neural networks and generate correction instructions to correct the optimal outfitting path.

[0147] The product assembly information model includes:

[0148] Component geometric information (3D geometric shape, patch model, wireframe model), hierarchical structure information (parent-child relationship, assembly constraints);

[0149] Basic attribute information (name, material properties, quality, process requirements) and pose information (initial pose, target pose, bounding box center pose).

[0150] Furthermore, the path planning and simulation module includes:

[0151] Narrow passage determination unit, used to calculate local obstacle density based on hypersphere sampling method. Based on local obstacle density Dynamically adjust the sampling strategy;

[0152] The convex hull pose distance calculation unit is used to calculate the maximum displacement distance between poses based on the rigid body transformation of the convex hull vertex set, and serves as a distance metric for path planning.

[0153] The path smoothing unit is used to generate a smooth B-spline path using bidirectional tree synchronous expansion and hierarchical bounding box collision detection, and outputs a path key point sequence containing pose sequence and time parameters.

[0154] Furthermore, the deviation detection and correction module includes:

[0155] The multi-sensor calibration unit is used to deploy industrial cameras, laser trackers and optical targets to form a multi-sensor network, set up multi-view observation stations and plan the spatial distribution of the target array.

[0156] The deviation heatmap generation unit is used to display the deviation distribution heatmap in real time through radial basis function interpolation.

[0157] The warning level classification unit is used to predict the development trend of deviations based on long short-term memory and output warning signals including normal, attention, and danger levels.

[0158] The system supports AR / VR interaction, uses a virtual-real fusion engine to provide visual guidance for the assembly process, and is compatible with mainstream CAD / CAPP systems, forming a fully digital closed loop.

[0159] This application also provides a computer device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the steps of the method described above.

[0160] This application also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the method described above.

[0161] This application solves the problems of low path planning efficiency, insufficient accuracy of virtual-real fusion, and lack of dynamic collision warning in complex cabin environments. It significantly reduces the cognitive load on workers, improves the assembly qualification rate and collision warning accuracy, and is applicable to the digital outfitting operation of complex system equipment such as ship pipelines and shaft and rudder systems, providing key technical support for digital shipbuilding.

[0162] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some or all of the technical features therein. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of this application.

Claims

1. A method for ship outfitting path planning and simulation modeling, characterized in that, Including the following steps: Collect environmental characteristic parameters of the ship system equipment outfitting site, and construct a consistent three-dimensional virtual environment through surface fitting and data alignment; Based on multiple constraints such as obstacles, narrow passages and dynamic environment, a narrow passage optimization algorithm is adopted. A six-degree-of-freedom path is dynamically planned based on the convex hull pose distance metric model. The optimal outfitting path of the ship system equipment is generated through hierarchical bounding box collision detection and B-spline path smoothing. Based on the optimal outfitting path, the dynamic simulation and collision warning of the outfitting process are realized by combining the three-dimensional visualization engine. Industrial cameras, laser trackers, and optical targets are used to collect outfitting process data. A mapping relationship between characteristic deviations and the three-dimensional geometric model of the ship system equipment assembly is established. Long short-term memory neural networks are used to predict deviation trends and generate correction instructions to correct the optimal outfitting path. The method employing a narrow channel optimization algorithm, dynamically planning a six-degree-of-freedom path based on a convex hull pose distance metric model, and generating the optimal outfitting path for the ship's system equipment through hierarchical bounding box collision detection and B-spline path smoothing includes: The ship outfitting workspace is mapped as a six-degree-of-freedom space (x, y, z, θ). x ,θ y ,θ z ), where the translational degrees of freedom are (x, y, z) and the rotational degrees of freedom are (θ). x ,θ y ,θ z ), x, y, z are the x-axis, y-axis, and z-axis directions respectively, θ x ,θ y ,θ z These represent the rotational degrees of freedom in the x-axis direction, the y-axis direction, and the z-axis direction, respectively. Define free space C free Obstacle Space C obs and narrow area C narrow ; The local obstacle density ρ0 is calculated based on the hypersphere sampling method, and the sampling strategy is dynamically adjusted based on the local obstacle density ρ0. When ρ0 is greater than or equal to a first threshold, it is determined to be a narrow channel, and directional bias sampling is performed along the main axis of the narrow channel. When ρ0 is less than the first threshold, it is determined to be an open area, and uniform random sampling is used in the open area. The maximum displacement distance between poses is calculated based on the rigid body transformation of the convex hull vertex set, which serves as the distance metric for path planning. A smooth B-spline path is generated by bidirectional tree synchronous expansion and hierarchical bounding box collision detection, and the output is a sequence of path key points containing pose sequence and time parameters. The process of collecting outfitting process data using industrial cameras, laser trackers, and optical targets, establishing a mapping relationship between characteristic deviations and the three-dimensional geometric model of the ship system equipment assembly, and predicting deviation trends and generating correction instructions to correct the optimal outfitting path through a long short-term memory neural network includes: A multi-sensor network is formed by deploying industrial cameras, laser trackers, and optical targets, setting up multi-view observation stations, and planning the spatial distribution of the target array. The three-dimensional coordinates of feature points are calculated using bundle adjustment, and the transformation relationship between the global coordinate system and the local coordinate system is established. Extract the measured pose data of key control points, calculate the deviation value from the theoretical model, and establish the mapping relationship between the deviation field and the three-dimensional geometric model of the ship system equipment assembly; Radial basis function interpolation is used to display the deviation distribution heatmap in real time, and a long short-term memory neural network is used to predict the deviation development trend and generate correction instructions. The optimal outfitting path is corrected according to the correction instructions.

2. The ship outfitting path planning and simulation modeling method according to claim 1, characterized in that, The process of collecting environmental characteristic parameters of the ship system equipment outfitting site and constructing a consistent three-dimensional virtual environment through surface fitting and data alignment includes: The site feature data is collected by a 3D laser scanner, industrial camera and depth sensor. The feature data includes at least one of the following: chimney section, side passage, seawater main unit, bulk carrier shaft and rudder system, oil tanker pipe bundle unit system, auxiliary positioning position, work space layout and equipment base. Point and surface feature information is extracted based on surface fitting and data alignment methods; Based on the collected feature data and extracted point and surface feature information, a product assembly information model, an assembly process information model, and an assembly path simulation information model are constructed through scene 3D rendering. The product assembly information model, assembly process information model, and assembly path simulation information model are then used to form a virtual scene of the outfitting site. Multiple coordinate systems are defined, and the position and pose data of the ship system equipment are corrected in real time by identifying markers, thereby correcting the parameters of the virtual scene of the outfitting site and reconstructing a three-dimensional virtual environment consistent with the outfitting site.

3. The ship outfitting path planning and simulation modeling method according to claim 2, characterized in that, The process of constructing a product assembly information model, an assembly process information model, and an assembly path simulation information model based on the collected feature data and extracted point and surface feature information through scene 3D rendering includes: Extract and integrate geometric information, hierarchical structure information, basic attribute information and pose information of parts from the collected feature data, integrate the information using XML format files, create part node files, and form a product assembly information model in augmented reality scenarios; Extract and integrate the task types, assembly processes and steps, operation objects, tools and resources, operation methods, assembly paths and prerequisites from the collected feature data, create XML files for the steps, refine the process steps, and form an assembly process information model. The assembly object is obtained by using a planning algorithm. The assembly base environment, assembly motion path, dynamic constraints and interference requirements are used as the motion posture information of the parts. The motion details of the part assembly process are described based on the motion posture information of the parts, and an assembly path XML file is created to form an assembly path simulation information model.

4. The ship outfitting path planning and simulation modeling method according to claim 1, characterized in that, The rigid body transformation based on the convex hull vertex set calculates the maximum displacement distance between poses, which serves as a distance metric for path planning. By obtaining the three-dimensional geometric model of the ship system equipment assembly, the set of convex hull vertices V = {v1, v2, ..., v} is calculated. n }; where v n To perform a rigid body transformation on the nth pose, i = 1, 2, ..., n; Define two poses, q1 and q2, and perform rigid body transformations on the convex hull vertex set: v1 = T(q1)·V, v2 = T(q2)·V, where the rigid body transformation matrix T(q) includes translation components t = [t x ,t y ,t z ] and rotational component R = R z (θ z )·R y (θ y )·R x (θ x );t x ,t y ,t z These represent the translation components along the x-axis, y-axis, and z-axis, respectively. R z (θ z ),R y (θ y ),R x (θ x These are the rotation components along the x-axis, y-axis, and z-axis, respectively. Calculate the maximum and average displacement distances of the corresponding vertices. The maximum displacement distance d max It serves as the distance metric between poses q1 and q2.

5. The ship outfitting path planning and simulation modeling method according to claim 1, characterized in that, The dynamic simulation and collision warning of the outfitting process based on the optimal outfitting path combined with a 3D visualization engine includes: Based on the 3D visualization engine, the 3D geometric model and planned path data of the ship system equipment assembly are loaded, and the motion trajectory and key pose points of the assembly are rendered in real time. When the minimum distance is detected to be less than the safety threshold, a collision warning is triggered. The 3D visualization engine highlights risk areas and automatically generates path correction suggestions, updating the optimal outfitting path.

6. A ship outfitting path planning and simulation modeling system, characterized in that, The system is used to implement the ship outfitting path planning and simulation modeling method according to any one of claims 1 to 5, the system comprising: The data acquisition and virtual modeling module is used to collect environmental characteristic parameters of the ship system equipment outfitting site, and to construct a consistent three-dimensional virtual environment through surface fitting and data alignment. The path planning and simulation module is used to dynamically plan a six-degree-of-freedom path based on multiple constraints such as obstacles, narrow passages and dynamic environment, using a narrow passage optimization algorithm and a convex hull pose distance metric model. It generates the optimal outfitting path for ship system equipment through hierarchical bounding box collision detection and B-spline path smoothing. Based on the optimal outfitting path, it combines a 3D visualization engine to realize dynamic simulation and collision warning of the outfitting process. The deviation detection and correction module is used to collect outfitting process data using industrial cameras, laser trackers and optical targets, establish the mapping relationship between characteristic deviations and the three-dimensional geometric model of the ship system equipment assembly, predict deviation trends through long short-term memory neural networks and generate correction instructions to correct the optimal outfitting path.

7. The ship outfitting path planning and simulation modeling system according to claim 6, characterized in that, The path planning and simulation module includes: The narrow passage determination unit is used to calculate the local obstacle density ρ0 based on the hypersphere sampling method and dynamically adjust the sampling strategy based on the local obstacle density ρ0. The convex hull pose distance calculation unit is used to calculate the maximum displacement distance between poses based on the rigid body transformation of the convex hull vertex set, and serves as a distance metric for path planning. The path smoothing unit is used to generate a smooth B-spline path using bidirectional tree synchronous expansion and hierarchical bounding box collision detection, and outputs a path key point sequence containing pose sequence and time parameters.

8. The ship outfitting path planning and simulation modeling system according to claim 6, characterized in that, The deviation detection and correction module includes: The multi-sensor calibration unit is used to deploy industrial cameras, laser trackers and optical targets to form a multi-sensor network, set up multi-view observation stations and plan the spatial distribution of the target array. The deviation heatmap generation unit is used to display the deviation distribution heatmap in real time through radial basis function interpolation. The warning level classification unit is used to predict the development trend of deviations based on long short-term memory and output warning signals including normal, attention, and danger levels.

9. A computer device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the steps of the method according to any one of claims 1 to 5.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 5.

Images