Method and system for constructing 3D model of recreation facility based on image recognition

Abstract

The invention relates to the technical field of computer vision, and discloses a recreation facility 3D model construction method and system based on image recognition, and the method comprises the steps: obtaining multi-view image data of a recreation facility, extracting invariant feature points, carrying out the deep learning matching fusion, and combining with triangulation to obtain an initial three-dimensional point cloud; complementing the point cloud of the shielded region to form a complete structure, segmenting the key part, extracting a preliminary skeleton line path, and performing interpolation connection on the interrupt paragraph to obtain a continuous skeleton line; and generating a local surface grid and optimizing smoothness, fusing geometric constraint to correct the grid, filling holes, and determining a final three-dimensional reconstruction model through texture mapping matching. According to the method, the continuous skeleton line of the recreation facility can be accurately extracted, and the high-quality 3D model which is high in precision, free of holes and smooth in surface is generated.

Description

Technical Field

[0001] This invention relates to the field of computer vision technology, and in particular to a method and system for constructing 3D models of amusement facilities based on image recognition. Background Technology

[0002] With the popularization of augmented reality and virtual reality technologies and the continuous advancement of image recognition technology, the demand for digital upgrades in theme parks is constantly increasing. The 3D digital reconstruction of amusement rides has become a key industry requirement, as the completeness, smoothness, and accuracy of the models directly affect safety inspection efficiency, virtual experience effects, and the value of digital assets. Amusement rides are mostly composed of numerous slender metal rods, curved tracks, and supporting structures, resulting in complex structures with significant mutual occlusion, placing extremely high demands on the environmental adaptability and detail capture capabilities of 3D reconstruction technology.

[0003] Currently, existing technologies in the field of 3D reconstruction are mainly divided into two categories. One category relies on laser scanning or multi-view structured light equipment, which offers high accuracy but requires professional operation and is difficult to deploy in large outdoor amusement park settings, making it difficult to achieve convenient and efficient reconstruction. The other category is based on image recognition technology, which achieves reconstruction through multi-view image acquisition and feature matching. While it offers flexible deployment and lower cost, its design approach is mostly applicable to conventional structures and lacks targeted optimization for the special structures of amusement park facilities, failing to fully consider the challenges posed by severe occlusion between components.

[0004] In practical applications, the slender rods and tracks of amusement facilities are prone to image data loss due to mutual occlusion. Existing image recognition-based methods are difficult to effectively supplement the data of occluded areas and cannot accurately extract continuous spatial skeleton lines. As the core of the topological relationship of the supporting facility, if the skeleton lines are interrupted or twisted due to occlusion or position estimation deviation, subsequent surface reconstruction will inevitably produce problems such as holes, deformities or unevenness at turns.

[0005] This technological limitation results in a core drawback of existing technologies: they cannot accurately extract continuous and reliable spatial skeleton lines through image recognition technology in reconstruction scenarios of amusement facilities with severe occlusion and complex structures. Consequently, the accuracy of 3D models is insufficient, failing to meet the industry's high-quality requirements for the 3D digital reconstruction of amusement facilities, and thus suffers from a lack of accuracy. Summary of the Invention

[0006] This invention provides a method and system for constructing 3D models of amusement facilities based on image recognition. It effectively solves the problem of interrupted skeleton lines caused by the complex structure and severe occlusion of amusement facilities, ensures the integrity of the model's topological relationship, significantly improves the geometric accuracy of the 3D model, and meets the high-quality requirements of safety inspection, virtual experience development, and other applications.

[0007] In a first aspect, to solve the above-mentioned technical problems, the present invention provides a method for constructing 3D models of amusement facilities based on image recognition, comprising: Acquire multi-view image data of amusement park rides; Extract an invariant set of feature points from the image data, match and fuse the feature point set, and combine it with triangulation to obtain initial three-dimensional point cloud data; A point cloud density distribution map is generated based on the initial 3D point cloud data. Occlusion areas in the point cloud density distribution map whose density does not meet the preset standard are detected. Data is supplemented for the occlusion areas through multi-view fusion to form a complete 3D point cloud structure. Key structural components are segmented from the complete 3D point cloud structure, and centerline tracing is performed on the key structural components. After topology construction and curve fitting, a preliminary skeleton line path is obtained. Filter out the interrupted segments in the preliminary skeleton line path, and perform path continuity deviation checks and corrections on the interrupted segments and adjacent paths to obtain a continuous skeleton line sequence; A local surface mesh is generated based on the continuous skeleton line sequence, and a smooth transition adjustment is performed on the local surface mesh to obtain a smooth surface mesh. The geometric feature constraints of the smooth surface mesh are adjusted, and then the hole structure in the mesh is identified and filled to obtain a hole-free surface model. The porous surface model is matched with the image data by texture mapping, the best texture source is selected and the seam transition is optimized to determine the final three-dimensional digital reconstruction model.

[0008] Secondly, the present invention provides a 3D model construction system for amusement facilities based on image recognition, comprising: The image acquisition module is used to acquire multi-view image data of amusement facilities; The feature point cloud module is used to extract an invariant set of feature points from the image data, perform matching and fusion on the feature point set, and combine triangulation calculations to obtain initial three-dimensional point cloud data. The occlusion completion module is used to generate a point cloud density distribution map based on the initial 3D point cloud data, detect occlusion areas in the point cloud density distribution map whose density does not meet the preset standard, and supplement the data of the occlusion areas through multi-view fusion to form a complete 3D point cloud structure. The skeleton segmentation module is used to segment key structural components from the complete 3D point cloud structure, perform centerline tracing on the key structural components, and obtain preliminary skeleton line paths after topology construction and curve fitting. The breakpoint connection module is used to filter out interrupted segments in the preliminary skeleton line path, and to perform path continuity deviation checks and corrections on the interrupted segments and adjacent paths to obtain a continuous skeleton line sequence. The mesh optimization module is used to generate a local surface mesh based on the continuous skeleton line sequence, and to perform smooth transition adjustment on the local surface mesh to obtain a smooth surface mesh. The correction and hole filling module is used to adjust the geometric feature constraints of the smooth surface mesh, and then identify and fill the hole structure in the mesh to obtain a hole-free surface model. The texture mapping module is used to match the non-porous surface model with the image data through texture mapping, select the best texture source and optimize the seam transition, and determine the final three-dimensional digital reconstruction model.

[0009] Thirdly, the present invention also provides an electronic device, including a processor, a memory, and a computer program stored in the memory and configured to be executed by the processor, wherein the processor executes the computer program to implement the image recognition-based 3D model construction method for amusement facilities as described in any one of the above.

[0010] Fourthly, the present invention also provides a computer-readable storage medium comprising a stored computer program, wherein, when the computer program is executed, it controls the device where the computer-readable storage medium is located to execute the image recognition-based 3D model construction method for amusement facilities as described above.

[0011] Compared with the prior art, the present invention has the following beneficial effects: (1) This invention retrieves the two-dimensional image blocks corresponding to the occluded area through a multi-view fusion algorithm, generates incremental point clouds through parallax calculation, and then combines surface normal vector continuity detection with the initial point cloud fusion. This can accurately fill the data gaps caused by the occlusion of amusement facilities by components, provide a complete and continuous point cloud basis for subsequent skeleton line extraction, and avoid reconstruction deviations caused by data loss.

[0012] (2) This invention determines continuity by calculating the angle between the tangent vectors of the neighborhood path of the breakpoint and the Euclidean distance, generates adaptive interpolation nodes for the interrupted segments with excessive deviation and establishes new topological connections, which can effectively solve the problem of skeleton line breakage and distortion, ensure the integrity of facility topological relationships, provide reliable centerline support for surface reconstruction, and improve the geometric accuracy of the model.

[0013] (3) This invention identifies the bending and turning areas of the mesh through curvature analysis, adjusts the position of the boundary nodes to optimize the smoothness of the mesh, and then combines the wavefront method to fill holes and geometric feature constraint deformation correction to effectively eliminate the deformities, sharp corners and hole defects on the model surface, so that the model not only conforms to the actual geometric features, but also has good smooth transition, meeting the high-quality requirements of scenarios such as security detection and virtual experience development. Attached Figure Description

[0014] Figure 1 This is a schematic diagram of a method for constructing a 3D model of an amusement facility based on image recognition, provided in the first embodiment of the present invention. Figure 2 This is a schematic diagram of a 3D model construction system for amusement facilities based on image recognition, provided in the second embodiment of the present invention. Detailed Implementation

[0015] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0016] Reference Figure 1 The first embodiment of the present invention provides a method for constructing 3D models of amusement facilities based on image recognition, including the following steps: S11, acquire multi-view image data of amusement facilities; S12, extract an invariant set of feature points from the image data, match and fuse the set of feature points, and combine it with triangulation to obtain initial three-dimensional point cloud data; S13, Generate a point cloud density distribution map based on the initial three-dimensional point cloud data, detect occlusion areas in the point cloud density distribution map whose density does not meet the preset standard, and supplement the data of the occlusion areas through multi-view fusion to form a complete three-dimensional point cloud structure. S14, key structural components are segmented from the complete three-dimensional point cloud structure, the centerline of the key structural components is traced, and the preliminary skeleton line path is obtained after topology construction and curve fitting. S15, filter out the interrupted segments in the preliminary skeleton line path, and perform path continuity deviation checks and corrections on the interrupted segments and adjacent paths to obtain a continuous skeleton line sequence; S16, Generate a local surface mesh based on the continuous skeleton line sequence, and perform smooth transition adjustment on the local surface mesh to obtain a smooth surface mesh; S17, perform geometric feature constraint adjustment on the smooth surface mesh, then identify and fill the hole structure in the mesh to obtain a hole-free surface model; S18, perform texture mapping matching between the non-porous surface model and the image data, select the best texture source and optimize the seam transition, and determine the final three-dimensional digital reconstruction model.

[0017] In step S11, it is necessary to acquire multi-view image data of the amusement park ride, including: A multi-view image acquisition system deployed in amusement park settings captures raw image data of amusement facilities from multiple perspectives. The acquisition system includes multiple high-definition cameras that simultaneously acquire images from different angles. The original image data is preprocessed to obtain multi-view image data of the amusement facility, wherein the preprocessing includes noise reduction, color correction and distortion correction.

[0018] It should be noted that, firstly, when capturing raw image data of amusement facilities from multiple perspectives using a multi-view image acquisition system deployed in amusement park settings, the acquisition system includes eight high-definition cameras that simultaneously capture images from the front, side, top, bottom, and four oblique angles. The resolution of each camera is 1920×1080 pixels, and the acquisition frequency is set to 10 frames per second, ensuring comprehensive coverage of the key structures of the amusement facilities without any blind spots.

[0019] Next, the original image data is preprocessed to obtain multi-view image data of the amusement park rides. Preprocessing includes denoising, color correction, and distortion correction. Denoising employs a Gaussian filtering algorithm with a 3×3 kernel size, effectively filtering out high-frequency noise caused by changes in ambient light. Color correction adjusts the RGB channel gain of the image using a grayscale world algorithm, ensuring color consistency deviation ≤5% across different viewpoints. Distortion correction performs radial and tangential distortion correction based on the camera's intrinsic parameter matrix, resulting in a pixel offset error ≤1 pixel after correction, ensuring the accuracy of subsequent feature extraction.

[0020] In step S12, an invariant set of feature points is extracted from the image data, the feature point set is matched and fused, and initial 3D point cloud data is obtained by combining triangulation calculations, including: Extract a set of feature points that are rotation-invariant and scale-invariant from the image data; The set of feature points is input into a pre-trained deep learning feature matching network, and a list of high-confidence feature point matching pairs is generated by learning the semantic relationships between feature points. Based on the feature point matching pair list, triangulation calculations are performed to solve the three-dimensional coordinates of the key structures of the amusement facility, and all three-dimensional coordinates are integrated to form initial three-dimensional point cloud data.

[0021] It should be noted that, firstly, when extracting a set of feature points with rotation invariance and scale invariance from multi-view image data, the SIFT algorithm is used for feature point extraction. Each feature point contains a 128-dimensional local descriptor, which can maintain feature stability under scene changes in viewpoint and scale scaling. An average of 5,000 feature points are extracted from a single image, mainly concentrated in key areas such as the track edges of amusement facilities, rod connection points, and corners of supporting structures.

[0022] Subsequently, the set of feature points is input into a pre-trained deep learning feature matching network, which is trained based on a CNN architecture. The input layer receives a 256-dimensional vector formed by concatenating 128-dimensional descriptors of each pair of SIFT feature points. This vector is then connected sequentially to a 1D convolutional layer 1, a ReLU activation function, and a 1D max pooling layer 1, followed by a 1D convolutional layer 2, a ReLU activation function, and a 1D max pooling layer 2. After that, a fully connected layer 1 and a ReLU activation function are connected. The fully connected layer 1 has 256 neurons. Finally, a fully connected layer 2 and a Sigmoid activation function are connected to output the matching confidence. Features are transferred between layers through linear connections that adapt the tensor dimensions.

[0023] The training data comes from 500 sets of multi-view images of different types of amusement facilities. After extracting SIFT feature pairs, "matched" and "mismatched" labels were manually added to construct 1 million sets of samples. These samples were divided into training and validation sets in an 8:2 ratio. This ratio was determined based on the sample size and model training requirements. The 1 million sets of samples constitute a large-scale dataset. 80% of the training set provides sufficient data to support the model in learning the semantic association rules between feature points, ensuring that the network fully fits the effective features. The 20% validation set can cover a sufficiently diverse distribution of feature pairs, stably evaluate the model's generalization ability under different hyperparameter configurations, and avoid evaluation bias caused by insufficient validation samples. This division method is a common optimization strategy for feature matching tasks in the field of deep learning, and its reliability has been verified in similar image feature association learning scenarios.

[0024] Before training, L2 normalization is performed on each set of feature point descriptors to eliminate scale differences between different feature point descriptors. During training, the binary cross-entropy loss function is used to calculate the cross-entropy value between the predicted confidence and the true label. The Adam optimizer is used to adaptively adjust the learning rate to improve training stability. The 1D convolution kernel size is set to 3 to cover the local correlation information of the feature descriptors and avoid capturing irrelevant noise. The number of output channels is increased from 64 to 128 to adapt to the abstraction requirements of features from local to global. The initial learning rate is set to 0.001 and decays by 0.1 every 10 training epochs to avoid gradient oscillation in the later stages of training. The training batch size is set to 512 to balance training efficiency and the memory load of a conventional industrial-grade GPU.

[0025] The network learns the semantic relationships between feature points, and determines feature pairs with a confidence score ≥ 0.9 as high-confidence matching pairs, while eliminating mismatched points with a confidence score < 0.9. For example, it generates 20,000 matching pairs from 40,000 feature points in an 8-view image with a matching accuracy of ≥ 95%, forming a feature point matching pair list.

[0026] Finally, when performing triangulation calculations on the list based on feature point matching, the 3D coordinates of the key structures of the amusement facility are solved step by step based on the principle of multi-view geometry. First, the pixel coordinates of feature points under each viewpoint are converted into homogeneous coordinates. Then, the effects of optical distortion and pixel scale are eliminated by combining the calibrated camera intrinsic parameter matrix, resulting in the ray direction vector in the camera coordinate system. This intrinsic parameter matrix includes optical characteristic parameters such as focal length and optical center position. Next, the rotation matrix and translation vector in the camera extrinsic parameter matrix are used to establish epipolar constraint relationships between different viewpoints, clarifying the projection association range of feature points in the multi-view and reducing unnecessary computation. Subsequently, a direct linear transformation algorithm is used to construct an overdetermined system of equations. The initial 3D coordinates are solved through singular value decomposition. This system of equations is derived from the projection relationships of each viewpoint, ensuring the uniqueness and stability of the solution. To address the errors in the initial solution, the least squares method is introduced to minimize the reprojection error, iteratively optimizing the 3D coordinate values ​​so that the pixel offset of the feature points after reprojection onto the images of each viewpoint is ≤1 pixel, ultimately obtaining high-precision spatial coordinates.

[0027] Taking a roller coaster track as an example, the pixel coordinates of the same feature point from eight different viewpoints are used to perform coordinate transformation and optimization, combined with the extrinsic parameter matrices of each camera, to accurately calculate the three-dimensional spatial coordinates of key positions on the track. The three-dimensional coordinates of all feature points are then integrated to form initial three-dimensional point cloud data.

[0028] In step S13, a point cloud density distribution map is generated based on the initial 3D point cloud data. Occlusion areas in the point cloud density distribution map whose density does not meet a preset standard are detected. Data is then supplemented to the occlusion areas through multi-view fusion to form a complete 3D point cloud structure, including: Based on the initial three-dimensional point cloud data, the point cloud density in each cell is calculated according to the preset spatial grid cells to generate a point cloud density distribution map. The regions corresponding to grid cells with point cloud density lower than a preset density threshold in the point cloud density distribution map are identified as occluded regions, and the target two-dimensional image blocks in the multi-view image data corresponding to the occluded regions are retrieved. The disparity of the target two-dimensional image block is calculated to infer depth information and generate incremental three-dimensional point cloud data. The incremental 3D point cloud data is fused with the initial 3D point cloud data, and the continuity is determined by detecting the change in the surface normal vector direction of the fused point cloud. If the change in the direction of the surface normal vector is less than a preset angle threshold, then the surface normal vector is determined to meet the continuity requirement, and thus it is determined to be a complete three-dimensional point cloud structure. If the change in the direction of the surface normal vector is greater than or equal to a preset angle threshold, it is determined that the continuity requirement is not met, and the marked area is an incomplete occluded area. The multi-view image data corresponding to the incomplete occluded area is retrieved again, and secondary incremental 3D point cloud data is generated and fused with the current fused point cloud again. The continuity detection steps are repeated until the change in the direction of the surface normal vector is less than the preset angle threshold, and it is determined to be a complete 3D point cloud structure.

[0029] It should be noted that the preset spatial grid unit size is 5cm×5cm×5cm. This size is determined in conjunction with the detailed scale of the amusement facility structure, which can accurately reflect the local point cloud density differences without causing excessive computation due to overly fine grids.

[0030] Next, when regions corresponding to grid cells with point cloud density below a preset density threshold in the point cloud density distribution map are identified as occluded regions, the preset density threshold is set to 10 points per cubic centimeter. This threshold was determined through testing with 100 sets of amusement park facility point cloud data. When the density is below this value, the probability of missing structural details is ≥85%, which cannot meet the data integrity requirements for subsequent skeleton extraction and surface reconstruction. Simultaneously, this avoids misclassifying normal low-density areas as occlusions, ensuring the accuracy of occluded region identification. The target two-dimensional image patch in the multi-view image data corresponding to the occluded region is retrieved, prioritizing viewpoint images within a 20° radius around the occluded region to ensure that the image patch contains indirect visual information about the occluded region.

[0031] Then, a semi-global matching algorithm is used to calculate disparity for the target 2D image patch, infer depth information, and generate incremental 3D point cloud data. The disparity search range is set to 0-64 pixels, and the matching cost function adopts census transform, which can effectively improve the matching accuracy in weak texture areas. The incremental point cloud density is consistent with the initial point cloud, and an average of 2000 points are added to a single occluded area.

[0032] Finally, the incremental 3D point cloud data is fused with the initial 3D point cloud data, and the continuity is determined by detecting the change in the surface normal direction of the fused point cloud. The preset angle threshold is 10°, which is set based on the technical goal of surface continuity of amusement facilities. Comparative tests show that when the change in the surface normal direction is less than 10°, there are no visually visible cracks or step-like deviations in the point cloud fusion area, and holes are less likely to be generated during subsequent surface mesh generation; if it is greater than or equal to 10°, 30% of the test samples show unevenness in the fusion area.

[0033] If the change in the surface normal vector direction is less than 10°, the surface normal vector is determined to meet the continuity requirement, thus establishing a complete 3D point cloud structure. If the change in the surface normal vector direction is greater than or equal to 10°, the marked area is an incomplete occluded area. Multi-view image data corresponding to the incomplete occluded area is retrieved again, and secondary incremental 3D point cloud data is generated and fused with the currently fused point cloud. This continuity detection step is repeated until the change in the surface normal vector direction is less than 10°. The final complete 3D point cloud structure contains 4.5 million points, with an occluded area completion rate ≥98% and no significant data loss.

[0034] In step S14, key structural components are segmented from the complete 3D point cloud structure, and centerline tracing is performed on the key structural components. After topology construction and curve fitting, a preliminary skeleton line path is obtained, including: Calculate the principal curvature values ​​of each point in the complete three-dimensional point cloud structure, and segment the complete three-dimensional point cloud structure according to the differences in the principal curvature values ​​to obtain point cloud subsets corresponding to key structural components; The point cloud subset is first clustered, and then the point cloud position is iteratively updated to shrink the point cloud subset toward the geometric center, generating a skeleton node set. A topology graph is constructed based on the skeleton node set, and the main path sequence is extracted by traversing the topology graph using a depth-first search. Curve fitting is performed on the main path sequence to generate a smooth centerline trajectory, which is then used as the initial skeleton line path for key structural components.

[0035] It should be noted that, firstly, a local neighborhood is constructed for each point cloud data point, with a radius of 8mm. This radius is determined based on the initial point cloud density and the minimum dimensions of the amusement facility structure, ensuring sufficient coverage of neighboring points to reflect local geometric features without introducing interference points from irrelevant areas due to excessive size. Next, a quadratic surface is fitted to the point cloud data within each local neighborhood using the least squares method. By solving for the eigenvalues ​​and eigenvectors of the quadratic surface, the two principal curvature values ​​and their corresponding principal curvature directions are obtained for that point. The principal curvature values ​​reflect the degree of curvature in the local area of ​​the point cloud, while the principal curvature directions represent the main orientation of the curvature. The principal curvature value is set to ≤0.02mm. -1 It is a slender metal rod, ≥0.1mm -1 For track components, the threshold is determined based on the structural characteristics of the two types of components: slender rods have a gentle curvature change, while the curved areas of the track have a significant curvature. This difference allows for precise segmentation. After segmentation, the track point cloud subset contains 1.35 million points, the rod point cloud subset contains 1.8 million points, and the support structure point cloud subset contains 1.35 million points.

[0036] Next, when clustering the point cloud subsets, the K-Means algorithm is used to cluster them according to their spatial location, with the number of clusters matching the actual number of structural components. Then, the Laplace shrinkage algorithm is used to iteratively update the point cloud positions, shrinking the point cloud subsets towards the geometric center. The number of iterations is set to 50. Testing shows that this number of iterations can shrink the point cloud into a sparse set of skeleton nodes while preserving the core structural shape. For example, the track point cloud subset shrinks to 2000 skeleton nodes, and each member of the member point cloud subset shrinks to 50-100 skeleton nodes.

[0037] Then, when constructing the topology graph, each skeleton node is used as a vertex of the graph, and the spatial connections between adjacent nodes are used as edges. The topology graph is traversed using depth-first search to extract the main path sequence. Taking a roller coaster track as an example, starting from the starting node, all connected nodes are traversed to obtain a main path sequence containing 1800 nodes, completely covering the ascending, curving, and descending sections of the track.

[0038] Finally, a B-spline curve fitting algorithm was used to fit the main path sequence. The fitting order was set to 3, which is the optimal choice to balance the reconstruction needs of the amusement facility skeleton line, engineering practicality, and algorithm characteristics. The curve parameters were then optimized using the least squares method to ensure that the average distance deviation between the fitted curve and the skeleton nodes was ≤2mm. This deviation standard meets the accuracy requirements of the amusement facility skeleton line and can generate a smooth centerline trajectory as the initial skeleton line path.

[0039] In step S15, interrupted segments in the preliminary skeleton line path are filtered out, and path continuity deviation checks and corrections are performed on the interrupted segments and adjacent paths to obtain a continuous skeleton line sequence, including: Traverse the initial skeleton line path, detect interrupted segments formed by abrupt changes in distance between adjacent nodes or interrupted connections, and mark the breakpoints of the interrupted segments; A local search range is constructed with each breakpoint as the center, and a set of neighboring path points falling into the range is obtained. Based on the spatial location correlation, the breakpoint pairs to be connected are selected, the tangent vector of the neighboring path point set is calculated, and the Euclidean distance between the breakpoint pairs to be connected is calculated. The tangent vector and the Euclidean distance together form the attitude correlation vector. If the spatial angle of the tangent vector is greater than or equal to a preset angle threshold, or the Euclidean spacing is greater than or equal to a preset distance threshold, then the deviation between the spatial angle of the tangent vector and the preset angle threshold, and the deviation between the Euclidean spacing and the preset distance threshold are calculated first, and then integrated to obtain the geometric discontinuity deviation value. Based on the deviation value, generate an appropriate number of interpolation nodes. Based on the geometric distribution trend of the neighborhood path point set and the spatial positional relationship of the breakpoints to be connected, determine the estimated connection direction between the breakpoints. Distribute the interpolation nodes evenly along the direction. Merge the interpolation nodes with the neighborhood path point set and establish a new topological connection. Replace the original interrupted segment and integrate it to obtain a preliminary continuous skeleton line sequence. If the spatial angle between the tangent vectors is less than a preset angle threshold and the Euclidean spacing is less than a preset distance threshold, then the continuity requirement is met, no interpolation filling is required, the corresponding interrupted segment in the preliminary skeleton line path is retained, and it is filled to the corresponding position of the preliminary continuous skeleton line sequence. By integrating all interpolated and filled interrupted segments and the retained interrupted segments in spatial order, a continuous skeleton line sequence is finally obtained.

[0040] It should be noted that, firstly, the initial skeleton line path is traversed to detect interrupted segments caused by abrupt changes in distance or broken connections between adjacent nodes. An interruption is defined as a distance exceeding 5cm between adjacent nodes. This distance threshold is set with reference to the average distribution characteristics of skeleton nodes. The average spacing between skeleton nodes for track-type components is 2.3cm, and the average spacing between skeleton nodes for slender rod-type components is 1.8cm. 5cm is approximately 2-3 times the average spacing for both types of components, accurately distinguishing normal node spacing from abnormal spacing caused by occlusion or missing point clouds. Furthermore, considering the manufacturing tolerance and installation error of ≤3cm for critical components of amusement facilities, this threshold eliminates interference from spacing fluctuations caused by tolerances, avoiding misjudging normal segments as interruptions or missing true breakpoints. Based on this, interrupted segments caused by abrupt changes in distance or broken connections are detected and marked. For example, in the area of ​​a roller coaster track obscured by the supporting structure, there are 3 breakpoints, forming 3 interrupted segments with spacings of 8cm, 12cm, and 6cm respectively.

[0041] Subsequently, when constructing a local search range centered on each breakpoint, the search range is set as a spherical area with a radius of 10cm. This radius is determined based on the average distribution spacing of the skeleton nodes and the maximum interruption spacing of a single component of the amusement facility. This ensures that the search range can fully cover the set of neighboring path points around the breakpoint, ensuring that potential connection points of the same structure are captured, while avoiding the introduction of irrelevant nodes of other components due to the excessively large range, thus balancing the comprehensiveness and efficiency of the search.

[0042] After obtaining the set of neighboring path points falling within the range, the pairs of breakpoints to be connected are selected based on spatial correlation. First, the Euclidean distance between any two breakpoints is calculated, and non-associated breakpoints with a distance exceeding 30cm are eliminated to avoid matching across structural components. Then, the local trajectory of the neighborhood point set of each breakpoint is fitted using the least squares method to determine whether the fitted trajectories of the two breakpoints show a continuous curvature trend. If the trajectory deviation is ≤5mm, they are determined to belong to the two ends of the same structural interruption. At the same time, considering the consistency of the principal curvature direction of the component where the breakpoint is located, cross-matching of breakpoints from different structural components is eliminated. Finally, the pairs of breakpoints to be connected are selected only for the same structural interruption segment. Then, the tangent vector of the neighborhood path point set is calculated, and the Euclidean distance between the pairs of breakpoints to be connected is also calculated.

[0043] It is worth noting that the preset angle threshold is 15° and the preset distance threshold is 10cm. Both thresholds are determined based on the technical objective of ensuring the continuity of the skeleton lines. After 100 sets of tests, when the included angle of the tangent vector space is less than 15° and the Euclidean spacing is less than 10cm, the paths at both ends of the breakpoint have good consistency, and direct connection can ensure continuity. If the distance exceeds this range, interpolation filling is required.

[0044] If the spatial angle of the tangent vectors is greater than or equal to 15°, or the Euclidean spacing is greater than or equal to 10cm, the geometric discontinuity deviation is calculated using an equal-weight normalization method. The deviations for both indices are calculated separately: the spatial angle deviation is the difference between the actual angle and 15°, and the Euclidean spacing deviation is the difference between the actual spacing and 10cm. The maximum reasonable deviation for the angle is taken as 45°, and the maximum reasonable deviation for the spacing as 50cm. Each deviation is divided by its corresponding maximum reasonable deviation to obtain a result normalized to the [0,1] interval. Finally, the arithmetic mean of the two normalized deviations is taken as the geometric discontinuity deviation value. When the deviation value is ≤0.5, 3 interpolation nodes are generated; when it is 0.5-1.0, 5 interpolation nodes are generated; and when it is >1.0, 8 interpolation nodes are generated. This number adapts to the discontinuity connection requirements of different deviation levels.

[0045] The predicted connection direction is determined based on the geometric distribution trend of the neighborhood path point set. Linear interpolation is used to uniformly distribute interpolation nodes along this direction. First, spatial line segments are constructed using the two endpoints of the pair of breakpoints to be connected as the start and end points. Then, based on the determined number of interpolation nodes, the length and unit vector of the line segment are calculated. Finally, the line segment is divided equally according to the number of nodes, and each division point is the spatial coordinate of the interpolation node. This ensures that the interpolation nodes are uniformly distributed along the predicted connection direction and consistent with the geometric trend of the neighborhood path point sets at both ends. These interpolation nodes are then merged with the neighborhood path point sets to establish a new topological connection, replacing the original broken segment.

[0046] If the spatial angle between the tangent vectors is less than 15° and the Euclidean spacing is less than 10cm, the continuity requirement is met, the corresponding segment is retained, and integrated into the continuous skeleton line sequence. The final continuous skeleton line sequence has a continuity deviation of ≤3° and no obvious breaks or distortions.

[0047] In step S16, a local surface mesh is generated based on the continuous skeleton line sequence, and a smooth transition adjustment is performed on the local surface mesh to obtain a smooth surface mesh, including: Centered on each node of the continuous skeleton line, a cross section with a preset radius is extended along the direction of the local normal vector obtained based on the geometric property analysis of the nodes of the continuous skeleton line to generate an initial mesh ring around the node. The mesh rings of adjacent nodes are connected to form a local surface mesh. A curvature tensor field is constructed for the local surface mesh to identify bending and turning regions where the principal curvature value exceeds a preset curvature threshold. Calculate the curvature change gradient field of the curved turning region. If the gradient magnitude exceeds a preset gradient threshold, extract the set of mesh boundary nodes at the edge of the corresponding region. Based on the curvature distribution and neighborhood mesh characteristics of the curved transition region, the local ideal curvature center is calculated, the radial offset of each boundary node in the mesh boundary node set to the local ideal curvature center is calculated, a radial displacement vector is generated based on the offset, the position of the nodes in the mesh boundary node set is adjusted by the displacement vector, deformed patches are deleted and retriangulated, and the topological connection is reconstructed to obtain a smooth surface mesh.

[0048] It is worth noting that, firstly, the local normal vector is determined by analyzing the geometric properties of the nodes of the continuous skeleton line. The tangent vector of each node is calculated first, and a local coordinate system is constructed by combining the changing trend of the tangent vectors of adjacent nodes. Then, the local normal vector that is perpendicular to the tangent vector and fits the cross-sectional shape of the structure is derived by referring to the principal curvature direction of the component where the node is located, ensuring that the expansion direction is consistent with the radial direction of the actual component of the amusement facility.

[0049] The preset radius is determined based on the dimensions of the structural components. The cross-sectional radius of the track components is set to 30cm, and the cross-sectional radius of the slender rods is set to 5cm. This size is consistent with the actual dimensions of the amusement facility components, ensuring that the grid can cover the actual structural area. Each grid ring contains 32 vertices, and a local surface grid is formed by connecting grid rings of adjacent nodes. The grid unit is a quadrilateral facet, and the initial number of grids is approximately 500,000.

[0050] Next, a curvature tensor field is constructed for the local surface mesh. First, six neighboring vertices are selected for each mesh vertex to form a local neighborhood set. The local quadratic surface of the neighborhood set is fitted using the least squares method. By solving the second-order fundamental form matrix of the quadratic surface, the curvature tensor of that vertex is obtained. Then, the curvature tensor is decomposed to obtain the principal curvature values ​​and directions, thus constructing a curvature tensor field covering the entire local surface mesh. Based on this tensor field, bending and turning regions where the principal curvature values ​​exceed a preset curvature threshold are identified, where the preset curvature threshold is set to 0.05 mm. -1 This threshold was determined through structural tests on different types of amusement facilities. It can accurately identify curved areas such as track turns and rod corners, while excluding slight curvature fluctuations in straight sections.

[0051] Then, the curvature gradient field of the bending transition region is calculated. For each vertex within the bending transition region, based on the principal curvature values ​​of its three adjacent vertices, the differences in the principal curvature values ​​in the three-dimensional spatial directions are calculated. These differences are then divided by the Euclidean distance between the vertex and its adjacent vertices to obtain the curvature rate of change components in the three directions. These components are combined to form a gradient vector. The magnitude of the gradient vector is then calculated. If the gradient magnitude exceeds a preset gradient threshold of 0.02 mm... -2 Extract the set of mesh boundary nodes at the edge of the corresponding region. This gradient threshold is set based on the technical requirements of mesh smoothing. If the gradient magnitude exceeds this value, it indicates a sharp change in curvature, and the initial mesh is prone to distortion or stretching deformation, requiring adjustment.

[0052] Subsequently, when calculating the local ideal curvature center based on the curvature distribution of the bending transition region, for each vertex in the bending transition region, along the normal extension line of its principal curvature direction, the point at a distance from the vertex equal to the corresponding principal curvature radius is taken as the initial curvature center of that vertex. The neighborhood weighted average method is used to fuse the initial curvature centers of all vertices in the region, with the weight taken as the similarity between the principal curvature value of the vertex and the principal curvature values ​​of the neighboring vertices, and finally the local ideal curvature center is obtained, ensuring that the center is consistent with the overall bending trend of the region.

[0053] Finally, the radial offset from the boundary nodes to the center is calculated, generating a radial displacement vector of 0.5-2 mm. This vector is used to adjust the node positions by translating each boundary node along the radial displacement vector by the corresponding offset. During translation, the spatial positions of neighboring nodes are used as constraints to ensure that the Euclidean spacing between the adjusted nodes and adjacent nodes remains within 0.8-1.2 times the original spacing, avoiding mesh overlap or breakage. Subsequently, distorted patches resulting from node adjustments are deleted, and the mesh is retriangulated. The topology is reconstructed to obtain a smooth surface mesh with uniform transitions at mesh bends and transitions, and a curvature change rate ≤ 0.01 mm. -1 It has no sharp corners or wrinkles.

[0054] In step S17, geometric feature constraints are adjusted for the smooth surface mesh, and then the pore structures in the mesh are identified and filled to obtain a pore-free surface model, including: Traverse the smooth surface mesh, calculate the dihedral angle between adjacent triangular facets, and mark the common edges with dihedral angles less than a preset angle threshold as geometric feature lines to form a set of geometric feature lines; Construct a Laplace deformation constraint equation that includes the set of geometric feature lines, and solve the equation to obtain the deformation-corrected intermediate mesh model; Traverse the topology of the intermediate mesh model, find unclosed boundary edges and organize them into a sequence of hole boundary loops; The wavefront method is used to extend inward from the starting vertex of the boundary ring sequence to generate new triangular facets that fit the local curvature trend. The new triangular facets are then topologically stitched with the intermediate mesh model to obtain a hole-free surface model.

[0055] It should be noted that, firstly, when traversing the smooth surface mesh and calculating the dihedral angle between adjacent triangular facets, common edges with dihedral angles less than a preset angle threshold are marked as geometric feature lines, forming a set of geometric feature lines. The preset angle threshold is set to 30°, which is determined based on the amusement facility structural design specifications. Areas with dihedral angles less than 30° are structural edges or key turning points and must be retained as feature constraints to ensure that the structural form after deformation correction conforms to the actual design.

[0056] Then, a Laplace deformation constraint equation containing geometric feature line set constraints is constructed. First, the nodes in the geometric feature line set are preprocessed to determine the spatial coordinates of all feature line nodes and mark them as fixed constraint vertices. These vertices correspond to the structural edges and key turning points of the amusement facility, and their original positions must remain unchanged during deformation to avoid deviation of the core structural form from the actual design. Next, the Cotangent weighting method is used to calculate the Laplace vector of each mesh vertex. For each unconstrained vertex, its adjacent vertices are traversed, and the cotangent values ​​of the two triangular faces corresponding to the adjacent edges are calculated. The sum of the cotangent values ​​is used as the weight of the adjacent vertex. Then, the coordinates of all adjacent vertices are weighted and averaged to obtain the Laplace vector of that vertex. This vector reflects the vertex's offset trend relative to its neighborhood, ensuring that the mesh retains its local geometric features after deformation. Finally, the Laplace energy function is constructed, with the function expression as follows: ,in The original coordinates of the vertex. The coordinates of the deformed target are... Let be the Laplacian vector of the vertex. The core objective of the energy function is to minimize the deviation between the vertex offset before and after deformation and the Laplacian vector, so that the mesh maintains overall smoothness during deformation. Then, the fixed constraints are transformed into... The equality constraints, combined with the energy functions of unconstrained vertices, are expanded into a unified sparse linear equation system Ax=b, where A is a coefficient matrix, with each row corresponding to the constraint relationship of a vertex, x is the coordinate vector of all deformed vertices, and b is a constant term vector containing the Laplacian vector and the coordinate information of the fixed-constraint vertices. Finally, the conjugate gradient method is used to solve this sparse linear equation system, utilizing the sparsity of the mesh topology to optimize the solution efficiency. The iteration termination condition is set to a residual less than 1e-6 to ensure solution accuracy. Using the geometric feature line node positions as fixed constraints, the Laplacian energy function of the mesh vertices is minimized, and the equations are solved to obtain the deformed intermediate mesh model. This process allows the mesh to correct local distortion deformation while maintaining the feature lines unchanged. The deviation between the deformed mesh and the actual structure is ≤1mm, meeting the accuracy requirements.

[0057] Next, the topology of the intermediate mesh model is traversed to find unclosed boundary edges and organize them into a sequence of hole boundary loops. The criterion for determining a hole boundary loop is that the number of consecutive unclosed boundary edges is ≥5 and they form a closed loop. This criterion can accurately identify holes caused by insufficient completion of structural component junctions or occluded areas. A single hole boundary loop contains 20-50 vertices.

[0058] Finally, the wavefront method is used to expand inward from the starting vertex of the boundary loop sequence, generating new triangular facets that conform to the local curvature trend. During the expansion process, the dihedral angle between the new facet and the adjacent original facet is kept ≤15°. This angle constraint ensures surface continuity. On average, 30 new triangular facets are generated for each hole. The new triangular facets are then topologically stitched with the intermediate mesh model. After stitching, the mesh vertex overlap is ≥99%, resulting in a hole-free surface model.

[0059] In step S18, the non-porous surface model is matched with the image data through texture mapping, the best texture source is selected and the seam transition is optimized to determine the final 3D digital reconstruction model, including: Calculate the angle between the normal of each triangular facet in the hole-free surface model and the optical axis of each camera in the image data, and filter viewpoints with angles smaller than a preset angle to form a candidate viewpoint set; The candidate viewpoint set is processed by depth buffering to exclude occluded triangular facets and determine the visible texture area and area coverage corresponding to each candidate viewpoint. Using the area coverage rate as the core optimization index, an energy optimization function is constructed to select the best texture source image from the images corresponding to the candidate viewpoint set; An initial texture map is generated based on the optimal texture source image. The seam boundaries in the initial texture map are identified. A Poisson equation is constructed to smoothly interpolate the seam boundaries to obtain a fused texture map. The fused texture map is precisely registered to the hole-free surface model to ensure that the texture is aligned with the geometry, thus determining the final 3D digital reconstruction model.

[0060] It should be noted that, firstly, the angle between the normal of each triangular facet in the hole-free surface model and the optical axis of each camera in the image data is calculated. Viewpoints with angles less than a preset threshold of 45° are then selected to form a candidate viewpoint set. This angle setting is based on the clarity requirements of texture acquisition; an angle less than 45° avoids texture stretching distortion caused by large-angle tilting, ensuring clear reproduction of texture details.

[0061] Next, when processing the candidate viewpoint set using a depth buffer, a depth buffer is first constructed based on the camera intrinsic and extrinsic parameters of each candidate viewpoint. The triangular facets of the hole-free surface model are projected onto the image plane of the corresponding viewpoint. The depth value of each vertex of the triangular facet is calculated and stored in the depth buffer. Then, the facets are sorted in ascending order of depth value, retaining the visible triangular facets with the smallest depth value and removing background triangular facets occluded by foreground facets. This determines the visible texture region corresponding to each candidate viewpoint. By statistically analyzing the ratio of the number of triangular facets in the visible region to the total number of triangular facets in the model, the region coverage is obtained. Taking the side of a roller coaster track as an example, the visible texture region coverage of a certain candidate viewpoint is 85%, covering a track length of 30 meters.

[0062] Then, using regional coverage as the core optimization metric, an energy optimization function is constructed by combining image resolution and illumination uniformity. ,in The weights for each indicator are determined based on the core requirements and practical application scenarios of texture mapping for 3D models of amusement facilities, and verified through multiple sets of tests. C represents the area coverage, ranging from [0,1], which is the ratio of the visible texture area to the total area of ​​the corresponding model area. R is the image resolution normalization value, obtained by dividing the image pixel density by the maximum pixel density. L is the illumination uniformity normalization value, ranging from [0,1], obtained by dividing the image grayscale variance by the maximum grayscale variance. During the screening process, the energy value E of each candidate viewpoint is calculated. The smaller the energy value, the better the overall quality of the texture source. The image corresponding to the viewpoint with the smallest energy value is selected as the best texture source. This weight allocation, after testing, prioritizes images with large coverage, clear details, and stable illumination, ensuring the quality of texture mapping.

[0063] Subsequently, an initial texture map is generated based on the optimal texture source image. The optimal texture source images from multiple perspectives are preprocessed, with invalid background areas cropped and removed. The textures are then scaled according to the geometric proportions of each part of the model to ensure the texture resolution matches the surface details of the model. Next, a texture atlas packing algorithm is used. A greedy strategy distributes the scaled texture blocks across a single 2048×2048 pixel texture canvas, rationally planning texture coordinates to avoid overlap while maintaining a 2-3 pixel gap between texture blocks to reduce seam interference. The packed texture map is then processed using the ETC2 compression format. Block compression and quantization encoding reduce storage capacity while preserving key texture details during compression, ensuring a texture quality loss of ≤3% after compression. This approach maintains texture quality while reducing storage and transmission pressure.

[0064] After identifying the seam boundaries in the initial texture map, a Poisson equation is constructed to smoothly interpolate the seam boundaries. Texture regions 3-5 pixels wide on both sides of the seam are extracted as transition zones. The grayscale values ​​of pixels within the transition zone are used as variables to construct the Poisson equation. The core constraint of the equation is the continuity of the grayscale gradient of pixels on both sides of the seam, and the boundary condition is that the original grayscale values ​​of pixels at the edge of the transition zone remain unchanged. The equation is solved using the conjugate gradient method to obtain the optimal grayscale value of each pixel within the transition zone, ensuring that the color transition gradient at the seam is ≤0.02 / pixel, thus obtaining the blended texture map.

[0065] Finally, the blended texture map is precisely registered to the hole-free surface model. A parametric texture coordinate mapping algorithm is used. First, two-dimensional texture coordinates (u, v) are assigned to each vertex of the hole-free surface model. The coordinate assignment is based on the geometric position of the vertex on the model surface and the layout of the texture map. A spherical parametric method is used to unfold the 3D model surface into a 2D plane, ensuring that the texture mapping has no significant stretching or deformation. Then, a one-to-one correspondence between vertex texture coordinates and blended texture map pixels is established through perspective projection transformation. For texture pixels at non-vertex positions on the model surface, bilinear interpolation is used to calculate grayscale values. Ultimately, the texture offset error after registration is ≤1 pixel, ensuring precise alignment between the texture and the geometric structure.

[0066] The final 3D digital reconstruction model has a geometric accuracy of ≤2mm and a texture resolution of ≥10 pixels / cm, which can clearly present the structural details and surface texture features of amusement facilities, meeting the application needs of safety inspection, virtual experience development and other applications.

[0067] In summary, this invention discloses a method for constructing 3D models of amusement rides based on image recognition. The method includes acquiring multi-view image data of the amusement ride, generating an initial 3D point cloud through feature point matching and fusion, completing the point cloud of occluded areas through multi-view fusion, segmenting key structural components and extracting preliminary skeleton line paths, interpolating and filling interrupted segments to obtain a continuous skeleton line sequence, generating and optimizing a surface mesh based on the continuous skeleton lines, fusing geometric constraints to correct the mesh and fill holes, and finally obtaining a 3D digital reconstruction model through texture mapping. This invention effectively solves the problems of low model reconstruction accuracy, interrupted skeleton lines, and surface holes caused by the complex structure and severe occlusion of amusement rides through an integrated technical solution including multi-view image recognition and deep learning, multi-view fusion hole filling, intelligent connection of skeleton line breaks, and mesh smoothing optimization. It achieves rapid construction of high-precision, hole-free, and smoothly transitioning 3D models of amusement rides, providing reliable technical support for theme park design, safety inspection, virtual experience development, and digital asset management.

[0068] Reference Figure 2 The second embodiment of the present invention provides a 3D model construction system for amusement facilities based on image recognition, comprising: The image acquisition module is used to acquire multi-view image data of amusement facilities; The feature point cloud module is used to extract an invariant set of feature points from the image data, perform matching and fusion on the feature point set, and combine triangulation calculations to obtain initial three-dimensional point cloud data. The occlusion completion module is used to generate a point cloud density distribution map based on the initial 3D point cloud data, detect occlusion areas in the point cloud density distribution map whose density does not meet the preset standard, and supplement the data of the occlusion areas through multi-view fusion to form a complete 3D point cloud structure. The skeleton segmentation module is used to segment key structural components from the complete 3D point cloud structure, perform centerline tracing on the key structural components, and obtain preliminary skeleton line paths after topology construction and curve fitting. The breakpoint connection module is used to filter out interrupted segments in the preliminary skeleton line path, and to perform path continuity deviation checks and corrections on the interrupted segments and adjacent paths to obtain a continuous skeleton line sequence. The mesh optimization module is used to generate a local surface mesh based on the continuous skeleton line sequence, and to perform smooth transition adjustment on the local surface mesh to obtain a smooth surface mesh. The correction and hole filling module is used to adjust the geometric feature constraints of the smooth surface mesh, and then identify and fill the hole structure in the mesh to obtain a hole-free surface model. The texture mapping module is used to match the non-porous surface model with the image data through texture mapping, select the best texture source and optimize the seam transition, and determine the final three-dimensional digital reconstruction model.

[0069] It should be noted that the embodiments of the present invention provide an image recognition-based amusement facility 3D model construction system for executing all process steps of the image recognition-based amusement facility 3D model construction method of the above embodiments. The working principles and beneficial effects of the two are one-to-one, so they will not be described again.

[0070] This invention also provides an electronic device. The electronic device includes a processor, a memory, and a computer program stored in the memory and executable on the processor, such as a 3D reconstruction program. When the processor executes the computer program, it implements the steps in the above embodiments of the image recognition-based amusement park 3D model construction method, for example... Figure 1 The step S11 shown. Alternatively, when the processor executes the computer program, it implements the functions of each module / unit in the above system embodiments, such as the texture mapping module.

[0071] For example, the computer program may be divided into one or more modules / units, which are stored in the memory and executed by the processor to complete the present invention. The one or more modules / units may be a series of computer program instruction segments capable of performing specific functions, and the instruction segments are used to describe the execution process of the computer program in the electronic device.

[0072] The electronic device may be a desktop computer, laptop, handheld computer, or smart tablet, etc. The electronic device may include, but is not limited to, a processor and memory. Those skilled in the art will understand that the above components are merely examples of electronic devices and do not constitute a limitation on the electronic device. It may include more or fewer components than described above, or combine certain components, or different components. For example, the electronic device may also include input / output devices, network access devices, buses, etc.

[0073] The processor can be a Central Processing Unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor can be a microprocessor, a processor, or any conventional processor. The processor is the control center of the electronic device, connecting all parts of the electronic device via various interfaces and lines.

[0074] The memory can be used to store the computer programs and / or modules. The processor implements various functions of the electronic device by running or executing the computer programs and / or modules stored in the memory and by calling data stored in the memory. The memory may mainly include a program storage area and a data storage area. The program storage area may store the operating system, at least one application program required for a function (such as sound playback function, image playback function, etc.), etc.; the data storage area may store data created according to the use of the mobile phone (such as audio data, phonebook, etc.). In addition, the memory may include high-speed random access memory, and may also include non-volatile memory, such as hard disk, memory, plug-in hard disk, smart media card (SMC), secure digital (SD) card, flash card, at least one disk storage device, flash memory device, or other volatile solid-state storage device.

[0075] Wherein, if the modules / units integrated in the electronic device are implemented as software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, all or part of the processes in the methods of the above embodiments of the present invention can also be implemented by a computer program instructing related hardware. The computer program can be stored in a computer-readable storage medium, and when executed by a processor, it can implement the steps of the various method embodiments described above. The computer program includes computer program code, which can be in the form of source code, object code, executable files, or certain intermediate forms. The computer-readable medium can include: any entity or system capable of carrying the computer program code, recording media, USB flash drives, portable hard drives, magnetic disks, optical disks, computer memory, read-only memory (ROM), random access memory (RAM), electrical carrier signals, telecommunication signals, and software distribution media, etc. It should be noted that the content included in the computer-readable medium can be appropriately added or removed according to the requirements of legislation and patent practice in the jurisdiction. For example, in some jurisdictions, according to legislation and patent practice, computer-readable media do not include electrical carrier signals and telecommunication signals.

[0076] It should be noted that the system embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Furthermore, in the accompanying drawings of the system embodiments provided by this invention, the connection relationships between modules indicate that they have communication connections, which can be specifically implemented as one or more communication buses or signal lines. Those skilled in the art can understand and implement this without any creative effort.

[0077] The specific embodiments described above further illustrate the purpose, technical solution, and beneficial effects of the present invention. It should be understood that the above descriptions are merely specific embodiments of the present invention and are not intended to limit the scope of protection of the present invention. In particular, it should be noted that any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention for those skilled in the art.

Claims

1. A method for constructing 3D models of amusement rides based on image recognition, characterized in that, include: Acquire multi-view image data of amusement park rides; Extract an invariant set of feature points from the image data, match and fuse the feature point set, and combine it with triangulation to obtain initial three-dimensional point cloud data; A point cloud density distribution map is generated based on the initial 3D point cloud data. Occlusion areas in the point cloud density distribution map whose density does not meet the preset standard are detected. Data is supplemented for the occlusion areas through multi-view fusion to form a complete 3D point cloud structure. Key structural components are segmented from the complete 3D point cloud structure, and centerline tracing is performed on the key structural components. After topology construction and curve fitting, a preliminary skeleton line path is obtained. Filter out the interrupted segments in the preliminary skeleton line path, and perform path continuity deviation checks and corrections on the interrupted segments and adjacent paths to obtain a continuous skeleton line sequence; A local surface mesh is generated based on the continuous skeleton line sequence, and a smooth transition adjustment is performed on the local surface mesh to obtain a smooth surface mesh. The geometric feature constraints of the smooth surface mesh are adjusted, and then the hole structure in the mesh is identified and filled to obtain a hole-free surface model. The porous surface model is matched with the image data by texture mapping, the best texture source is selected and the seam transition is optimized to determine the final three-dimensional digital reconstruction model.

2. The method for constructing 3D models of amusement facilities based on image recognition according to claim 1, characterized in that, The acquisition of multi-view image data of amusement facilities includes: A multi-view image acquisition system deployed in amusement park settings captures raw image data of amusement facilities from multiple perspectives. The acquisition system includes multiple high-definition cameras that simultaneously acquire images from different angles. The original image data is preprocessed to obtain multi-view image data of the amusement facility, wherein the preprocessing includes noise reduction, color correction and distortion correction.

3. The method for constructing 3D models of amusement facilities based on image recognition according to claim 1, characterized in that, The step of extracting an invariant set of feature points from the image data, matching and fusing the feature point set, and combining it with triangulation calculations to obtain initial 3D point cloud data includes: Extract a set of feature points that are rotation-invariant and scale-invariant from the image data; The set of feature points is input into a pre-trained deep learning feature matching network, and a list of high-confidence feature point matching pairs is generated by learning the semantic relationships between feature points. Based on the feature point matching pair list, triangulation calculations are performed to solve the three-dimensional coordinates of the key structures of the amusement facility, and all three-dimensional coordinates are integrated to form initial three-dimensional point cloud data.

4. The method for constructing 3D models of amusement facilities based on image recognition according to claim 1, characterized in that, The process of generating a point cloud density distribution map based on the initial 3D point cloud data, detecting occluded areas in the point cloud density distribution map whose density does not meet a preset standard, and supplementing the occluded areas with data through multi-view fusion to form a complete 3D point cloud structure includes: Based on the initial three-dimensional point cloud data, the point cloud density in each cell is calculated according to the preset spatial grid cells to generate a point cloud density distribution map. The regions corresponding to grid cells with point cloud density lower than a preset density threshold in the point cloud density distribution map are identified as occlusion regions, and the target two-dimensional image blocks in the multi-view image data corresponding to the occlusion regions are retrieved. The disparity of the target two-dimensional image block is calculated to infer depth information and generate incremental three-dimensional point cloud data. The incremental 3D point cloud data is fused with the initial 3D point cloud data, and the continuity is determined by detecting the change in the surface normal vector direction of the fused point cloud. If the change in the direction of the surface normal vector is less than a preset angle threshold, then the surface normal vector is determined to meet the continuity requirement, and thus it is determined to be a complete three-dimensional point cloud structure. If the change in the direction of the surface normal vector is greater than or equal to a preset angle threshold, it is determined that the continuity requirement is not met, and the marked area is an incomplete occluded area. The multi-view image data corresponding to the incomplete occluded area is retrieved again, and secondary incremental 3D point cloud data is generated and fused with the current fused point cloud again. The continuity detection steps are repeated until the change in the direction of the surface normal vector is less than the preset angle threshold, and it is determined to be a complete 3D point cloud structure.

5. The method for constructing 3D models of amusement facilities based on image recognition according to claim 1, characterized in that, The process of segmenting key structural components from the complete 3D point cloud structure, tracing the centerlines of the key structural components, and obtaining preliminary skeleton line paths after topology construction and curve fitting includes: Calculate the principal curvature values ​​of each point in the complete three-dimensional point cloud structure, and segment the complete three-dimensional point cloud structure according to the differences in the principal curvature values ​​to obtain point cloud subsets corresponding to key structural components; The point cloud subset is first clustered, and then the point cloud position is iteratively updated to shrink the point cloud subset toward the geometric center, generating a skeleton node set. A topology graph is constructed based on the skeleton node set, and the main path sequence is extracted by traversing the topology graph using a depth-first search. Curve fitting is performed on the main path sequence to generate a smooth centerline trajectory, which is then used as the initial skeleton line path for key structural components.

6. The method for constructing 3D models of amusement facilities based on image recognition according to claim 1, characterized in that, The process of filtering interrupted segments in the preliminary skeleton line path, performing path continuity deviation checks and corrections on the interrupted segments and adjacent paths, and obtaining a continuous skeleton line sequence includes: Traverse the initial skeleton line path, detect interrupted segments formed by abrupt changes in distance between adjacent nodes or interrupted connections, and mark the breakpoints of the interrupted segments; A local search range is constructed with each breakpoint as the center, and a set of neighboring path points falling into the range is obtained. Based on the spatial location correlation, the breakpoint pairs to be connected are selected, the tangent vector of the neighboring path point set is calculated, and the Euclidean distance between the breakpoint pairs to be connected is calculated. The tangent vector and the Euclidean distance together form the attitude correlation vector. If the spatial angle of the tangent vector is greater than or equal to a preset angle threshold, or the Euclidean spacing is greater than or equal to a preset distance threshold, then the deviation between the spatial angle of the tangent vector and the preset angle threshold, and the deviation between the Euclidean spacing and the preset distance threshold are calculated first, and then integrated to obtain the geometric discontinuity deviation value. Based on the deviation value, generate an appropriate number of interpolation nodes. Based on the geometric distribution trend of the neighborhood path point set and the spatial positional relationship of the breakpoints to be connected, determine the estimated connection direction between the breakpoints. Distribute the interpolation nodes evenly along the direction. Merge the interpolation nodes with the neighborhood path point set and establish a new topological connection. Replace the original interrupted segment and integrate it to obtain a preliminary continuous skeleton line sequence. If the spatial angle between the tangent vectors is less than a preset angle threshold and the Euclidean spacing is less than a preset distance threshold, then the continuity requirement is met, no interpolation filling is required, the corresponding interrupted segment in the preliminary skeleton line path is retained, and it is filled to the corresponding position of the preliminary continuous skeleton line sequence. By integrating all interpolated and filled interrupted segments and the retained interrupted segments in spatial order, a continuous skeleton line sequence is finally obtained.

7. The method for constructing 3D models of amusement facilities based on image recognition according to claim 1, characterized in that, The step of generating a local surface mesh based on the continuous skeleton line sequence, and performing smooth transition adjustments on the local surface mesh to obtain a smooth surface mesh includes: Centered on each node of the continuous skeleton line, a cross section with a preset radius is extended along the direction of the local normal vector obtained based on the geometric property analysis of the nodes of the continuous skeleton line to generate an initial mesh ring around the node. The mesh rings of adjacent nodes are connected to form a local surface mesh. A curvature tensor field is constructed for the local surface mesh to identify bending and turning regions where the principal curvature value exceeds a preset curvature threshold. Calculate the curvature change gradient field of the curved turning region. If the gradient magnitude exceeds a preset gradient threshold, extract the set of mesh boundary nodes at the edge of the corresponding region. Based on the curvature distribution and neighborhood mesh characteristics of the curved transition region, the local ideal curvature center is calculated, the radial offset of each boundary node in the mesh boundary node set to the local ideal curvature center is calculated, a radial displacement vector is generated based on the offset, the position of the nodes in the mesh boundary node set is adjusted by the displacement vector, deformed patches are deleted and retriangulated, and the topological connection is reconstructed to obtain a smooth surface mesh.

8. The method for constructing 3D models of amusement facilities based on image recognition according to claim 1, characterized in that, The process of adjusting the geometric features of the smooth surface mesh, then identifying and filling the holes in the mesh to obtain a hole-free surface model includes: Traverse the smooth surface mesh, calculate the dihedral angle between adjacent triangular facets, and mark the common edges with dihedral angles less than a preset angle threshold as geometric feature lines to form a set of geometric feature lines; Construct a Laplace deformation constraint equation that includes the set of geometric feature lines, and solve the equation to obtain the deformation-corrected intermediate mesh model; Traverse the topology of the intermediate mesh model, find unclosed boundary edges and organize them into a sequence of hole boundary loops; The wavefront method is used to extend inward from the starting vertex of the boundary ring sequence to generate new triangular facets that fit the local curvature trend. The new triangular facets are then topologically stitched with the intermediate mesh model to obtain a hole-free surface model.

9. The method for constructing 3D models of amusement facilities based on image recognition according to claim 1, characterized in that, The step of matching the texture of the non-porous surface model with the image data, selecting the best texture source and optimizing the seam transition, and determining the final 3D digital reconstruction model includes: Calculate the angle between the normal of each triangular facet in the hole-free surface model and the optical axis of each camera in the image data, and filter viewpoints with angles smaller than a preset angle to form a candidate viewpoint set; The candidate viewpoint set is processed by depth buffering to exclude occluded triangular facets and determine the visible texture area and area coverage corresponding to each candidate viewpoint. Using the area coverage rate as the core optimization index, an energy optimization function is constructed to select the best texture source image from the images corresponding to the candidate viewpoint set; An initial texture map is generated based on the optimal texture source image. The seam boundaries in the initial texture map are identified. A Poisson equation is constructed to smoothly interpolate the seam boundaries to obtain a fused texture map. The fused texture map is precisely registered to the hole-free surface model to ensure that the texture is aligned with the geometry, thus determining the final 3D digital reconstruction model.

10. A 3D model construction system for amusement facilities based on image recognition, characterized in that, include: The image acquisition module is used to acquire multi-view image data of amusement facilities; The feature point cloud module is used to extract an invariant set of feature points from the image data, perform matching and fusion on the feature point set, and combine triangulation calculations to obtain initial three-dimensional point cloud data. The occlusion completion module is used to generate a point cloud density distribution map based on the initial 3D point cloud data, detect occlusion areas in the point cloud density distribution map whose density does not meet the preset standard, and supplement the data of the occlusion areas through multi-view fusion to form a complete 3D point cloud structure. The skeleton segmentation module is used to segment key structural components from the complete 3D point cloud structure, perform centerline tracing on the key structural components, and obtain preliminary skeleton line paths after topology construction and curve fitting. The breakpoint connection module is used to filter out interrupted segments in the preliminary skeleton line path, and to perform path continuity deviation checks and corrections on the interrupted segments and adjacent paths to obtain a continuous skeleton line sequence. The mesh optimization module is used to generate a local surface mesh based on the continuous skeleton line sequence, and to perform smooth transition adjustment on the local surface mesh to obtain a smooth surface mesh. The correction and hole filling module is used to adjust the geometric feature constraints of the smooth surface mesh, and then identify and fill the hole structure in the mesh to obtain a hole-free surface model. The texture mapping module is used to match the non-porous surface model with the image data through texture mapping, select the best texture source and optimize the seam transition, and determine the final three-dimensional digital reconstruction model.

Images