Ship pipe identification and assembly method and system based on binocular camera
By using a binocular camera-based method for identifying and managing ship pipes, combined with deep learning and reinforcement learning, high-precision pipe identification and efficient management of pipe distribution have been achieved. This solves the problems of low accuracy and low efficiency in existing technologies, and significantly improves production efficiency and economic benefits.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- JIANGSU MODERN SHIPBUILDING TECH
- Filing Date
- 2025-12-25
- Publication Date
- 2026-05-01
AI Technical Summary
Existing methods for identifying and assembling ship pipes lack the ability to accurately measure three-dimensional geometric parameters, making it impossible to achieve accurate identification at the model level. Furthermore, assembly and distribution management still relies on manual decision-making, resulting in low identification accuracy, low efficiency, and high costs.
A binocular camera-based identification and assembly method is adopted, which combines deep learning and reinforcement learning. The binocular camera acquires stereo images, performs 3D reconstruction and pipe identification, and uses an adaptive window semi-global matching algorithm and multi-dimensional feature fusion to achieve accurate identification of pipe types and local components. The assembly process is optimized through a reinforcement learning model.
It significantly improves the accuracy and efficiency of ship pipe identification, with an identification accuracy rate of 96.5% and a collection and distribution accuracy rate of 98.7%. It reduces the labor intensity and cost of workers, meets the accuracy requirements of ship installation, has a short equipment investment payback period, and has significant economic benefits.
Smart Images
Figure CN121963177A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of intelligent ship manufacturing technology, and in particular to a method and system for identifying and assembling ship pipes based on a binocular camera. Background Technology
[0002] Pipe identification and assembly management technology is a key component in intelligent shipbuilding, encompassing multiple production stages including pipe system manufacturing, pre-assembly, and on-site installation. As the "vascular system" of a ship, the piping system comprises hundreds of types, including power pipelines, pressure transmission pipelines, and fire-fighting pipelines; a single large ship may contain tens of thousands of pipes. The unique characteristics of pipe identification and assembly in ships are primarily reflected in: the high variety and similarity of pipe components, the complex and variable environment, and the extremely high precision requirements for assembly.
[0003] Existing methods for identifying and assembling ship pipes include:
[0004] 1. RFID / Barcode-based Management System: Some shipyards use RFID tags or QR codes to identify pipes and read the information through scanning equipment. However, the tags are easily damaged, and the readability rate is only about 40% after processes such as high-temperature welding and sandblasting. In addition, this method relies on pre-labeling and cannot identify pipes that are not labeled or whose tags are missing.
[0005] 2. Identification method based on manual measurement: This method uses tools such as tape measures and squares to measure the pipe dimensions and angles, and compares them with drawings to confirm the model. This method is too time-consuming (10-15 minutes to measure a single complex pipe), has low accuracy (measurement stroke ±5mm), and cannot accurately measure curved pipes.
[0006] 3. Recognition based on monocular vision images: A few studies have attempted to use industrial cameras to capture images of pipes and extract features using traditional image processing algorithms (such as Canny edge detection and Hough transform). Monocular vision cannot obtain visual depth information, making it difficult to accurately measure three-dimensional parameters such as pipe diameter and length, with a recognition accuracy of only about 60%.
[0007] 4. Laser scanning-based 3D reconstruction: This method uses a laser scanner to acquire point clouds of the pipes and then matches them with the CAD model via ICP registration. However, this method is costly (over 1 million RMB for a single set of equipment), has a long scanning time (5-10 minutes per pipe), and performs poorly on reflective metal surfaces, making it impractical.
[0008] 5. General workpiece recognition based on deep learning: Some attempts were made to directly apply general object detection models such as YOLO and Mask R-CNN. These models can only identify the presence and approximate location of pipes, but cannot extract key parameters such as pipe diameter, bending angle, and flange type, thus failing to meet production requirements. Summary of the Invention
[0009] To address the shortcomings of existing technologies, this invention provides a method and system for identifying and assembling ship pipes based on a binocular camera, thereby solving the technical problems of lacking accurate measurement capabilities of three-dimensional geometric parameters, being unable to achieve accurate identification at the model level, and still relying on manual decision-making for assembly and distribution management.
[0010] This invention provides a method for identifying and assembling ship pipes based on a binocular camera, comprising: a pipe image acquisition process, a 3D reconstruction process, a pipe identification process, a pipe matching process, and an assembly process, wherein...
[0011] The tube image acquisition process includes: acquiring a pair of stereo images of the tube by selecting the optimal acquisition angle based on the tube's geometry using a binocular camera;
[0012] The 3D reconstruction process includes: performing stereo correction on the acquired stereo image pairs, generating a depth map, reconstructing the 3D point cloud of the pipe, and extracting the geometric parameters of the pipe from the 3D point cloud;
[0013] The tube identification process includes: using stereo image pairs, depth maps, and geometric parameters as input parameters for a deep neural network model to identify the type of tube and its local components;
[0014] The pipe matching process includes: performing multi-level matching of the identification results with the design database to obtain complete part information;
[0015] The collection and distribution process includes: obtaining the optimal collection and distribution plan through a reinforcement learning model based on the pipe type identification results, pipe matching results, and inventory status.
[0016] Furthermore, the specific method for selecting the optimal acquisition angle based on the pipe geometry during the pipe image acquisition process is as follows:
[0017] When the tube is straight, the optimal sampling angle is 0°.
[0018] When the pipe is bent, the optimal sampling angle is 45°.
[0019] When the pipe is a tee, the optimal sampling angle is 90°.
[0020] Furthermore, in the three-dimensional reconstruction process, a semi-global matching algorithm with an adaptive window is used for stereo matching, wherein the size of the matching window is adjusted in the pipe edge region according to the curvature of the pipe edge region.
[0021] Furthermore, the matching cost function of the semi-global matching algorithm for the adaptive window is:
[0022] ;
[0023] in, ;
[0024] ;
[0025] ;
[0026] ;
[0027] In the formula, p is the current pixel position; d is the disparity value; and r is the aggregation path direction. and An adaptive smoothing penalty term; and These are the weighting coefficients for the edge preservation term and the curvature constraint term, respectively; E edge For edge-preserving terms; E curv Curvature constraint; and Based on the penalty parameter; These are the gradient normalization coefficients; The gradient of the disparity map; The second derivative of the disparity map; For indicator functions; The reference curvature is estimated based on the cylindrical model of the tube; This is the edge detection threshold.
[0028] Furthermore, the deep neural network model is a deep neural network model with a three-branch structure. The features of the three branches are fused into multi-dimensional features through a topological mechanism. The three branches include an RGB branch, a depth branch, and a geometry branch. The RGB branch is used to extract texture features; the depth branch is used to process depth images; and the geometry branch is used to encode geometric parameters.
[0029] Furthermore, in the deep neural network model, the multi-dimensional feature fusion formula for the features of the three branches is as follows:
[0030] ;
[0031] In the formula, This is the fused composite feature vector; Texture feature vectors extracted for the RGB branch; The shape feature vector extracted for the depth branch; The feature vector of parameters for encoding geometric branches; For feature cross-attention operations; For attention mechanism module; It is a multilayer perceptron.
[0032] Furthermore, the pipe identification process also includes constructing a graph structure of each component of the pipe and using a message-passing graph neural network for pipe structure identification.
[0033] Furthermore, in the collection and distribution process, the collection and distribution process is modeled as a Markov decision process, and the Markov decision process is solved by a reinforcement learning model;
[0034] In the Markov decision process, the state space includes: current inventory, production plan, and already allocated pipes; the action space is the set of pipes to be allocated; and the reward function is:
[0035] ;
[0036] In the formula, Total reward value; Incentives for efficient distribution; Rewards are given based on accuracy. As a workload balancing bonus; To incur penalties; , , , These are the weighting coefficients for distribution efficiency, accuracy, workload balance, and penalty costs, respectively.
[0037] The present invention also provides a ship pipe identification and collection system based on a binocular camera, which is applicable to the ship pipe identification and collection method based on a binocular camera. The system includes: a binocular acquisition device, an edge computing module, and a cloud server.
[0038] The binocular acquisition device includes: a binocular camera and a support frame; an edge computing module connected to the binocular camera for real-time stereo matching and preliminary recognition; and a cloud server connected to the edge computing module for deploying a complete deep neural network model, designing a database, and integrating optimization algorithms.
[0039] Furthermore, the support frame is an angle-adjustable tripod, and the adjustable angles include at least three fixed shooting angles: 0°, 45°, and 90°.
[0040] The beneficial effects of this invention are:
[0041] This invention significantly improves recognition accuracy by fusing binocular vision with deep learning, achieving a 96.5% accuracy rate for identifying ship pipes, a 36% improvement compared to traditional monocular vision methods. Specifically, the accuracy rates are 98.2% for straight pipes, 95.8% for bends, 94.6% for tees, and 93.1% for complex pipe combinations. The geometric parameter measurement accuracy is: diameter < ±2mm, length < ±5mm, and angle < ±1°, fully meeting the requirements for ship installation accuracy.
[0042] The present invention significantly improves the processing speed per second, with a complete identification process for a single tube (Lock 3.5, image acquisition 1s, stereo matching 0.5s, feature extraction 0.8s, identification matching 1.2s), which is more than 100 times faster than manual measurement and identification (5-15 minutes); it supports batch processing, with a throughput of 13,000 tubes / day when using a standard 8-channel terminal, meeting the production needs of large shipyards.
[0043] This invention significantly improves the efficiency of batch picking. The intelligent batch picking algorithm increases the accuracy of pallet matching to 98.7% and reduces the mismatch rate to 1.3% (compared to approximately 5-8% for manual picking). The average batch picking time is reduced from 25 minutes per batch manually to 3 minutes per batch. Through path optimization, the picking distance is reduced by 45%, significantly reducing the labor intensity of workers.
[0044] This invention features robustness and enhanced recognition. The system maintains stable performance under different lighting conditions (50-5000 lux), different surface conditions (original / coated / corroded), and partial deviations (<30%). Through an incremental learning mechanism, the recognition rate of 100 marked samples of the new ship type pipe can reach more than 90%, and 500 samples can reach 95%.
[0045] This invention offers significant economic benefits. Based on a medium-sized shipyard producing 20 ships annually, the system can reduce manpower for pipe identification by 10 people, reduce mismatch rework losses by approximately 5 million RMB per year, and shorten the entire production cycle to 3-5 days per ship. The equipment investment payback period is only 18 months, with an annual return on investment (ROI) exceeding 165%. Attached Figure Description
[0046] The features and advantages of the invention will be more clearly understood by referring to the accompanying drawings, which are schematic and should not be construed as limiting the invention in any way. In the drawings:
[0047] Figure 1 This is a flowchart of a specific embodiment of the present invention;
[0048] Figure 2 This is a schematic diagram of the multi-dimensional feature fusion process in a specific embodiment of the present invention;
[0049] Figure 3 This is a schematic diagram of a Markov decision framework according to a specific embodiment of the present invention. Detailed Implementation
[0050] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.
[0051] The present invention will be further illustrated below with reference to specific embodiments. Those skilled in the art should understand that these embodiments are for illustrative purposes only and are not intended to limit the scope of the invention. Modifications to the present invention in various equivalent forms all fall within the scope defined by the appended claims.
[0052] like Figure 1-3 As shown, this invention provides a method for identifying and assembling ship pipes based on a binocular camera, including: a pipe image acquisition process, a 3D reconstruction process, a pipe identification process, a pipe matching process, and an assembly process, wherein...
[0053] The tube image acquisition process includes: acquiring a pair of stereo images of the tube by selecting the optimal acquisition angle based on the tube's geometry using a binocular camera;
[0054] Before the method is executed, camera calibration is performed automatically, using Zhang Zhengyou's calibration method to obtain the extrinsic parameter matrix. The calibration board uses a 12×9 checkerboard grid, automatically acquiring 20 images from different angles, and completing the calibration when the reprojection is less than 0.5 pixels.
[0055] The tube is placed on the acquisition platform, and the system automatically detects the tube using OpenCV and triggers image capture. The binocular camera simultaneously acquires images from left and right (exposure synchronization accuracy <1μs), while also recording the ambient noise intensity and adaptively adjusting the exposure parameters.
[0056] The specific method for selecting the optimal acquisition angle based on the tube geometry is as follows:
[0057] When the tube is straight, the optimal sampling angle is 0°.
[0058] When the pipe is bent, the optimal sampling angle is 45°.
[0059] When the pipe is a tee, the optimal sampling angle is 90°.
[0060] The sparsity strategy is automatically selected based on the pipe's surface condition (original steel pipe, after coating, rust, etc.). HDR multi-parameter fusion is used for reflective surfaces, morphological cues are used for denoising rusted surfaces, and adaptive histogram equalization is applied to oily areas. The sparsity parameters are automatically determined using a lightweight classification network (MobileNet) to adapt to different operating conditions.
[0061] The 3D reconstruction process includes: performing stereo correction on the acquired stereo image pairs, eliminating the alignment of the correction array using the Bouguet algorithm, calculating the disparity map using the improved SGM algorithm, reconstructing the 3D point cloud using the principle of triangulation, achieving a point cloud density of 5000 points / dm², generating a depth map, and extracting the geometric parameters of the pipe from the 3D point cloud;
[0062] During the 3D reconstruction process, a semi-global matching algorithm with an adaptive window is used for stereo matching. The matching window size is adjusted in the pipe edge region according to the curvature of the pipe edge region.
[0063] First, we define the matching cost calculation for the adaptive window:
[0064] ;
[0065] in, This is the current pixel position. For disparity values, and The images are left and right, respectively. For a two-sided weighting function, The robust cost function is used (either Census transform or AD-Census hybrid cost).
[0066] Adaptive window Automatically adjust size based on image gradient:
[0067] ;
[0068] in, For pixels Image gradient at that location, This is the edge detection threshold.
[0069] Automatically narrow the matching window to the edge region of the tube (where the gradient is greater than the threshold). To maintain edge sharpness, expand the window in smooth areas. To improve matching stability.
[0070] In the cost aggregation stage, edge preservation terms and curvature constraints are introduced, and the improved aggregation cost function is as follows:
[0071] ;
[0072] in, For the aggregation path direction, and For adaptive smoothing penalty term, and These are the weighting coefficients for the edge preservation term and the curvature constraint term, respectively.
[0073] The adaptive penalty term is dynamically adjusted based on the image gradient:
[0074] ;
[0075] in, and Based on the penalty parameter, is the gradient normalization coefficient. This design allows for reduced smoothing constraints in edge regions to maintain disparity discontinuities, and enhanced smoothing constraints in smooth regions to suppress noise.
[0076] The edge preservation term is defined as:
[0077] ;
[0078] in, For the disparity map gradient, This is an indicator function. This item ensures that parallax edges align with image edges.
[0079] Curvature constraints are designed for the cylindrical geometry of the tube:
[0080] ;
[0081] in, The second derivative (curvature) of the disparity map. This is the reference curvature estimated based on the cylindrical model of the tube. This constraint makes the reconstructed 3D point cloud more consistent with the cylindrical surface characteristics of the tube.
[0082] Simultaneously, a multi-scale pyramid matching strategy is adopted, which first quickly matches at 1 / 4 resolution to obtain the initial disparity, and then refines it step by step to the full resolution, thereby increasing the matching speed by 3 times (single frame processing time <500ms).
[0083] Eight types of geometric parameters were extracted from the point cloud: total pipe length, pipe diameter, wall thickness, bending angle, bending radius, flange specifications, number of branches, and end face shape. The pipe's bending direction was determined using RANSAC cylindrical and planar surfaces through principal component analysis (PCA).
[0084] The tube identification process includes: using stereo image pairs, depth maps, and geometric parameters as input parameters for a deep neural network model to identify the type of tube and its local components;
[0085] The deep neural network model is a three-branch structure. The features of the three branches are fused into multi-dimensional features through a topological mechanism. The three branches include: RGB branch, depth branch, and geometry branch. The RGB branch is used to extract texture features; the depth branch is used to process depth images; and the geometry branch is used to encode geometric parameters.
[0086] RGB data is used to extract 512-dimensional texture features, depth data to extract 256-dimensional shape features, and geometric branching to encode 128-dimensional parametric features. These three types of features are merged into a comprehensive 896-dimensional feature extraction through a focus module. The comprehensive feature is then subjected to dimensionality reduction through two fully connected layers (896→256→64), and finally, a Softmax classifier is used to output the probability distribution of 42 pipe categories (including 15 categories of straight pipes, 12 categories of bends, 8 categories of tees, 4 categories of four-way connectors, and 3 categories of reducers). A Top-3 candidate result is also output for manual confirmation.
[0087] The multi-dimensional feature fusion formula for the three branches is as follows:
[0088] ;
[0089] In the formula, This is the fused composite feature vector; Texture feature vectors extracted for the RGB branch; The shape feature vector extracted for the depth branch; The feature vector of parameters for encoding geometric branches; This is a feature cross-attention operation used to model the relationship between texture features and depth features; For feature splicing operations; This is an attention mechanism module used to adaptively learn the importance weights of different feature channels; It is a multilayer perceptron used for nonlinear mapping of geometric parameter features.
[0090] The network uses knowledge-enhanced training, the teacher network uses complete point cloud training, and the student network uses only binocular images, which improves the recognition accuracy by 15% while maintaining reasoning efficiency.
[0091] For local components of the pipe, each component (straight pipe section, elbow, flange, tee joint, etc.) is modeled as a graph structure, where nodes represent components and edges represent connections. A message-passing graph neural network (MPGNN) is used to learn the spatial and assembly constraints between components, enabling structural identification of complex pipe fittings. The adjacency matrix of the graph is automatically constructed through spatial location measurements using binoculars, eliminating the need for manual labeling.
[0092] An improved Mask R-CNN is used to detect local components (flanges, welds, elbows, joints, etc.) of pipes. Each component outputs a bounding box, segmentation mask, and 7-dimensional attribute protections (type, size, angle, position, etc.). Component information is used for subsequent precise matching.
[0093] The pipe matching process includes: multi-level matching of the identification results with the design database: first, initial dimensions are determined by pipe type and size range (reducing to <100 candidates); then, geometric similarity is calculated (based on chamfer distance); finally, confirmation is achieved through local component matching (IoU > 0.85 is considered a successful match). Complete part information is obtained after a successful match.
[0094] The collection and distribution process includes: based on the pipe type identification results, pipe matching results, and inventory status, obtaining the optimal collection and distribution plan through a reinforcement learning model, specifically including:
[0095] Demand analysis for collection and distribution: The system receives the production plan, parses out the list, quantity, priority, and operation constraints of the pipes to be collected and distributed; groups the demand according to the installation area and workpiece, with each group containing no more than 50 pipes, and handles them on-site;
[0096] Inventory status assessment: Real-time query of pipe inventory database, statistics on available quantity, shelf location, and quality inspection status of each type of pipe; for out-of-stock pipes, automatically generate purchase requirements or transfer requests.
[0097] Intelligent pallet allocation optimization: The reinforcement learning agent generates the optimal pallet allocation scheme based on the current state; the algorithm considers six optimization objectives: minimizing the sorting selection path, balancing the workload of each area, prioritizing the critical path, reducing inventory occupation, avoiding misallocation and rework, and improving turnover efficiency; the decision time for a single pallet allocation is less than 2 seconds.
[0098] The collection and allocation process is modeled as a Markov decision process, and the Markov decision process is solved by a reinforcement learning model.
[0099] In the Markov decision process, the state space includes: current inventory, production plan, and already allocated pipes; the action space is the set of pipes to be allocated; and the reward function is:
[0100] ;
[0101] In the formula, Total reward value; The reward for efficient distribution represents the number of distribution tasks completed per unit of time. The accuracy reward represents the correctness of tube identification and matching; The workload balancing reward represents the degree of balance in workload across different work areas. Penalty costs include rework costs for mismatches, inventory backlog costs, and delivery delay costs; , , , These are weighting coefficients for distribution efficiency, accuracy, workload balance, and penalty costs, which are adjusted according to actual production needs.
[0102] Distribution execution and feedback: A distribution instruction sheet is generated and sent to the on-site workers via PAD. The workers pick the pipes according to the instructions, scan and confirm, and update the inventory. The system records the actual distribution results as a reward signal for reinforcement learning to continuously optimize the decision-making model.
[0103] The system employs an incremental learning strategy based on memory replay, maintaining a sample buffer pool with a capacity of 5000 samples to store the feature displays of various typical tubes. When a new ship-shaped tube is identified, the system retrieves the most similar historical samples based on similarity, performing the retraining only on the last two layers of the network (freezing the remaining layers). The model completes automatically within 3 minutes, avoiding complete retraining.
[0104] Visualized monitoring and early warning: Provides a 3D visualized distribution dashboard that displays distribution progress, inventory allocation, and anomaly alarms in real time; when the distribution confidence level is less than 0.85 or the inventory is insufficient, an automatic early warning is issued, and managers can remotely intervene and make adjustments.
[0105] A specific embodiment of the present invention also provides a ship pipe identification and distribution system based on a binocular camera, which is applicable to the ship pipe identification and distribution method based on a binocular camera. The system includes: a binocular acquisition device, an edge computing module, and a cloud server.
[0106] The binocular acquisition device includes: a binocular camera and a support frame. The binocular unit integrates two 12-megapixel industrial cameras, a structured light supplement module, and an automatic matching mechanism. The support frame is an angle-adjustable tripod with at least three fixed shooting angles: 0°, 45°, and 90°. The edge computing module is connected to the binocular camera for real-time stereo matching and preliminary recognition. It is responsible for connecting to the NVIDIA Jetson AGX Orin for real-time stereo matching and preliminary recognition. The cloud server is connected to the edge computing module for deploying a complete deep neural network model, design database, and integrated optimization algorithm.
[0107] The following are the verification results of this invention:
[0108] Binocular measurement accuracy experiment: Tests were conducted using standard pipe fittings (with known precise dimensions), including 15 different specifications. Experimental results show that at a working distance of 1.5 meters, the average absolute workpiece distance for pipe measurements was 1.8 mm, with a maximum trajectory of 3.2 mm; the average workpiece distance for length measurements was 4.1 mm; the average workpiece distance for angle measurements was 0.7°; and the standard deviation of the repeatability test (100 measurements of the same pipe fitting) was <0.5 mm, demonstrating the system's reliability.
[0109] Recognition model generalization experiment: A test set was constructed by collecting 50,000 pipe images from 5 shipyards and 3 ship types. Experimental results show that the SCPNet model achieves a data accuracy of 92.3% on unseen shipyards, demonstrating good generalization ability; through t-SNE visualization of the feature space, different types of pipes form obvious dialog boxes, and the inter-class distance is 2.5 times greater than the intra-class distance.
[0110] Comparative experiment on optimized distribution: Several typical production batches (200-500 pipes per batch) were compared. The experimental results show that: Compared with the rule-based algorithm, the reinforcement learning algorithm reduces distribution completion time by 38%, path length by 41%, and inventory turnover rate by 52%; compared with manual decision-making, the mismatch rate is reduced by 75%, and the number of emergency distribution adjustments is reduced by 60%.
[0111] Field Application: The system was tested at a shipyard for 6 months, identifying over 80,000 pipes and completing over 1,500 batches. Experimental results showed a 96.2% accuracy rate in identification, 98.4% in batching accuracy, and 99.2% system availability (downtime <2 hours per month for large-scale projects). A survey of on-site workers revealed that 87% of them felt the system significantly reduced their workload.
[0112] Although embodiments of the invention have been described in conjunction with the accompanying drawings, those skilled in the art can make various modifications and variations without departing from the spirit and scope of the invention, and such modifications and variations all fall within the scope defined by the appended claims.
Claims
1. A method for identifying and assembling pipes on ships based on a binocular camera, characterized in that, include: The process includes pipe image acquisition, 3D reconstruction, pipe identification, pipe matching, and assembly. The tube image acquisition process includes: acquiring a pair of stereo images of the tube by selecting the optimal acquisition angle based on the tube's geometry using a binocular camera; The 3D reconstruction process includes: performing stereo correction on the acquired stereo image pairs, generating a depth map, reconstructing the 3D point cloud of the pipe, and extracting the geometric parameters of the pipe from the 3D point cloud; The tube identification process includes: using stereo image pairs, depth maps, and geometric parameters as input parameters for a deep neural network model to identify the type of tube and its local components; The pipe matching process includes: performing multi-level matching of the identification results with the design database to obtain complete part information; The collection and distribution process includes: obtaining the optimal collection and distribution plan through a reinforcement learning model based on the pipe type identification results, pipe matching results, and inventory status.
2. The method for ship pipe identification and assembly based on a binocular camera as described in claim 1, characterized in that, The specific method for selecting the optimal acquisition angle based on the tube geometry during the tube image acquisition process is as follows: When the tube is straight, the optimal sampling angle is 0°. When the pipe is bent, the optimal sampling angle is 45°. When the pipe is a tee, the optimal sampling angle is 90°.
3. The method for identifying and assembling ship pipes based on a binocular camera as described in claim 1, characterized in that, In the 3D reconstruction process, a semi-global matching algorithm with an adaptive window is used for stereo matching, wherein the size of the matching window is adjusted in the pipe edge region according to the curvature of the pipe edge region.
4. The method for identifying and assembling ship pipes based on a binocular camera as described in claim 3, characterized in that, The matching cost function of the semi-global matching algorithm for the adaptive window is: ； in, ; ； ； ； In the formula, p is the current pixel position; d is the disparity value; and r is the aggregation path direction. and An adaptive smoothing penalty term; and These are the weighting coefficients for the edge preservation term and the curvature constraint term, respectively; E edge For edge-preserving terms; E curv Curvature constraint; and Based on the penalty parameter; These are the gradient normalization coefficients; The gradient of the disparity map; The second derivative of the disparity map; For indicator functions; The reference curvature is estimated based on the cylindrical model of the tube; This is the edge detection threshold.
5. The method for identifying and assembling ship pipes based on a binocular camera as described in claim 1, characterized in that, The deep neural network model is a three-branch deep neural network model. The features of the three branches are fused into multi-dimensional features through a topological mechanism. The three branches include an RGB branch, a depth branch, and a geometry branch. The RGB branch is used to extract texture features; the depth branch is used to process depth images; and the geometry branch is used to encode geometric parameters.
6. The method for ship pipe identification and assembly based on a binocular camera as described in claim 5, characterized in that, In the deep neural network model, the multi-dimensional feature fusion formula for the three branches is as follows: ； In the formula, This is the fused composite feature vector; Texture feature vectors extracted for the RGB branch; The shape feature vector extracted for the depth branch; The feature vector of parameters for encoding geometric branches; For feature cross-attention operations; For attention mechanism module; It is a multilayer perceptron.
7. The method for identifying and assembling ship pipes based on a binocular camera as described in claim 1, characterized in that, The pipe identification process also includes constructing a graph structure of each component of the pipe and using a message-passing graph neural network for pipe structure identification.
8. The method for identifying and assembling ship pipes based on a binocular camera as described in claim 1, characterized in that, In the aforementioned collection and distribution process, the collection and distribution process is modeled as a Markov decision process, and the Markov decision process is solved through a reinforcement learning model; In the Markov decision process, the state space includes: current inventory, production plan, and already allocated pipes; the action space is the set of pipes to be allocated; and the reward function is: ； In the formula, Total reward value; Incentives for efficient distribution; Rewards are given based on accuracy. As a workload balancing bonus; To incur penalties; , , , These are the weighting coefficients for distribution efficiency, accuracy, workload balance, and penalty costs, respectively.
9. A binocular camera-based system for identifying and assembling ship pipes, applicable to the binocular camera-based method for identifying and assembling ship pipes as described in any one of claims 1-8, characterized in that, The system includes: a binocular acquisition device, an edge computing module, and a cloud server; The binocular acquisition device includes: a binocular camera and a support frame; an edge computing module connected to the binocular camera for real-time stereo matching and preliminary recognition; and a cloud server connected to the edge computing module for deploying a complete deep neural network model, designing a database, and integrating optimization algorithms.
10. The ship pipe identification and distribution system based on a binocular camera as described in claim 9, characterized in that, The support frame is an angle-adjustable tripod, with adjustable angles including at least three fixed shooting angles: 0°, 45°, and 90°.