Machine Vision-Based Inspection System and Method for High-Strength Bolts in Large-Scale Amusement Facilities

Abstract

This invention relates to the field of large-scale amusement facility inspection technology, and in particular to a machine vision-based high-strength bolt inspection system and method for large-scale amusement facilities, comprising: an image information acquisition unit, a data processing and storage module, a real-time data display module, a defect alarm module, a defect identification module, a defect marking module, a data communication module, and a remote expert system; each module is linked in sequence to realize the acquisition, processing, identification, alarm, and subsequent maintenance guidance of high-strength bolt defects.

Description

Technical Field

[0001] This invention relates to the field of large-scale amusement park ride inspection technology, and in particular to a machine vision-based inspection system and method for high-strength bolts in large-scale amusement park rides. Background Technology

[0002] In recent years, with the large-scale construction of theme parks in my country, large-scale amusement rides have become a must-have classic attraction in major theme parks and small and medium-sized amusement parks. At the same time, the ways to play on large-scale amusement rides are constantly being innovated, the equipment structures are becoming more and more complex, and the operating parameters are constantly challenging the physiological limits of the human body, which also brings huge challenges to the safety inspection and testing of large-scale amusement rides.

[0003] High-strength bolt connections are one of the most common connection methods in large amusement facilities, widely used in the important mechanical structural connections of large amusement equipment, such as the connecting bolts of the truss structure of large aerial sightseeing vehicles, the connecting bolts of high-speed roller coaster tracks and columns, the connecting bolts of ultra-high observation towers, and the connecting bolts of high-altitude bungee jumping towers. However, high-strength bolts are prone to loosening, breakage, or detachment under temperature loads, wind loads, and dynamic loads on the equipment. This not only affects the structural strength and service safety performance of the equipment but may even cause equipment accidents and personal injury.

[0004] Currently, routine inspections of high-strength bolt connections in large amusement facilities mainly rely on maintenance personnel visually inspecting whether the anti-loosening lines on the high-strength bolts have shifted, thus indirectly determining whether the high-strength bolts have become loose. If loose high-strength bolts are found, they need to be tightened with a torque wrench. In addition to visually inspecting whether the anti-loosening lines on the bolts have become loose, the annual inspection of high-strength bolts also requires random torque checks on some high-strength bolts using a torque wrench according to the requirements of the user manual to further determine whether the high-strength bolts have become loose.

[0005] The main problems with manual visual inspection are as follows: For large aerial sightseeing vehicles, there are thousands to tens of thousands of truss connection bolts. Relying on manual inspection is extremely inefficient, and it is difficult to guarantee the accuracy of the inspection, and it is also easy to miss some. For the track and column connection bolts of high-speed roller coasters, the highest point is about 60m above the ground. Currently, telescopes are mainly used for observation, which has low inspection accuracy. For the high-strength bolts connecting ultra-tall observation towers and high-altitude bungee jumping towers, the highest point is as high as 300m above the ground. The current inspection method requires maintenance personnel to climb to the tower connection points regularly to inspect the high-strength bolts one by one. Some equipment towers are installed on the edge of cliffs. High-altitude operations not only pose great difficulties and safety risks, but are also prone to missing some. For some equipment, due to structural design, maintenance personnel need to use large machinery such as cranes and baskets to reach the inspection position, which makes the inspection difficult.

[0006] To address the existing problems, there is an urgent need for a detection system and method that combines artificial intelligence and deep learning, integrates multiple machine vision technologies, is easy to operate and carry, and has automatic detection and identification functions for loose bolts.

[0007] The information disclosed in this background section is intended only to enhance the understanding of the general background of the invention and should not be construed as an admission or in any way implying that the information constitutes prior art known to those skilled in the art. Summary of the Invention

[0008] The purpose of this invention is to provide a machine vision-based high-strength bolt inspection system and method for large amusement facilities, in order to solve the technical problems of low efficiency, easy to miss, large safety hazards, and high difficulty and safety risks of high-altitude operation in the existing technology.

[0009] To achieve the above objectives, the present invention adopts the following technical solution:

[0010] In a first aspect, the present invention provides a high-strength bolt detection system for large amusement facilities based on machine vision, comprising: an image information acquisition unit, a data processing and storage module, a real-time data display module, a defect alarm module, a defect identification module, a defect marking module, a data communication module, and a remote expert system; each module is linked in sequence to realize the acquisition, processing, identification, alarm, and subsequent maintenance guidance of high-strength bolt defects.

[0011] Preferably, the image information acquisition unit is connected to the data processing and storage module, the data processing and storage module is connected to the real-time data display module, the real-time data display module is connected to the defect alarm module, the defect alarm module is connected to the defect identification module, the defect identification module is connected to the defect information marking module and the defect information clearing and resetting module, the defect information clearing and marking module is connected to the data communication module and transmits the defect information to the remote expert system, and the remote expert system provides maintenance solutions and suggestions.

[0012] Preferably, the image information acquisition unit includes a handheld image acquisition component and a drone image acquisition component; the handheld image acquisition component includes a handheld camera bracket, an industrial camera, an electronic display screen, and a light source, and is mainly used to acquire images of high-strength bolts that are relatively low in position and easy to shoot at close range; the drone image acquisition component includes a drone and an onboard industrial camera and a light source, and is mainly used to acquire images of high-strength bolts that are high in position and cannot be easily shot by the handheld component, so as to achieve full-scene coverage acquisition.

[0013] Preferably, the defect alarm module automatically alarms when the data processing and storage module initially determines that the "relative movement of the anti-loosening line position" is abnormal, triggering the defect identification module to further confirm the abnormal event.

[0014] Preferably, the defect identification module accurately identifies the alarm information pushed by the defect alarm module, and through image feature analysis, eliminates false alarms caused by non-bolt failure factors such as unclear anti-loosening line marks, oil stains obscuring the anti-loosening line, and light reflection, and confirms and marks the real defects.

[0015] Preferably, the remote expert system receives the confirmed defect information transmitted by the data communication module, further analyzes the defect type and severity, judges the defect development trend, and generates detailed maintenance plans and operation suggestions; at the same time, it transmits the defect information to the control room to remind maintenance personnel to pay attention to the status changes of the corresponding high-strength bolts in real time.

[0016] Secondly, the present invention provides a detection method for a machine vision-based high-strength bolt detection system for large amusement facilities, comprising the following steps:

[0017] S1. Preprocessing of database of typical high-strength bolt failure models for large amusement facilities;

[0018] S2. Construct a deep learning model for high-strength bolt instance segmentation in large amusement facilities, and train the high-strength bolt instance segmentation model on a database of typical high-strength bolt failure models in large amusement facilities.

[0019] S3, Module for analyzing the status and determining the failure mode of high-strength bolted connection structure markings.

[0020] Preferably, step S1 includes:

[0021] S11. Generate a database of typical high-strength bolt failure models for large amusement facilities;

[0022] S12, Label the failure model data of typical high-strength bolts in large amusement facilities;

[0023] S13. Dataset partitioning and preprocessing augmentation to generate a training dataset for typical high-strength bolt failure models of large amusement facilities.

[0024] Preferably, step S2 includes:

[0025] S21. Deep learning model structure for high-strength bolt instance segmentation;

[0026] S22. Training the deep learning model structure for high-strength bolt instance segmentation.

[0027] Preferably, step S3 includes:

[0028] S31. Segmentation and identification of high-strength bolt components;

[0029] S32. Continuity analysis of high-strength bolt markings, specifically including: S321. Using the pixel mask output in step S31 as input, determine that each detailed component of the high-strength bolt has a pixel mask; S322. Using the "connected component analysis" algorithm to detect the continuity of the high-strength bolt markings; S323. Structured output of the high-strength bolt marking continuity analysis results;

[0030] S33, Spatial relationship mapping of high-strength bolt detailed components, specifically including: S331, calculating the overlap between the marking mask and the component mask; S332, establishing the structural mapping relationship between the marking segment and the bolt component to obtain the spatial relationship mapping table; S333, after the spatial relationship mapping table is constructed, the model uses it as input and combines it with the predefined structural rationality model to automatically analyze the mapping results;

[0031] S34, rule engine decision-making;

[0032] S35 output results.

[0033] By adopting the above technical solution, the present invention has the following beneficial effects:

[0034] This invention provides a machine vision-based high-strength bolt inspection system and method for large amusement rides. This system can detect the loosening of high-strength bolts in complex structural components of various currently used large amusement rides, and has a wide range of applications. The machine vision-based high-strength bolt inspection system and method can conveniently inspect truss connection bolts in large aerial tramway equipment, with high inspection efficiency. This system and method can also inspect bolts connecting ultra-high-speed roller coaster tracks and columns, track connection bolts, and bolts connecting ultra-high observation towers and high-altitude bungee jumping towers at higher locations, with simple and convenient operation and accurate defect location. The remote expert system of this machine vision-based high-strength bolt inspection system and method can accurately assess the degree of loosening of high-strength bolts and provide detailed repair plans and suggestions. The inspection method and system provided by this invention have a simple structure, are easy to operate, and have high inspection efficiency, making them valuable for application. Attached Figure Description

[0035] To more clearly illustrate the specific embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the specific embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0036] Figure 1 This is a schematic diagram of the high-strength bolt anti-loosening line drawing method provided in an embodiment of the present invention;

[0037] Figure 2 This invention provides a high-strength bolt loosening defect model for embodiments of the invention.

[0038] Figure 3 This is a schematic diagram of two detection states for high-strength bolts provided in an embodiment of the present invention;

[0039] Figure 4 A flowchart for high-strength bolt testing provided in this embodiment of the invention;

[0040] Figure 5 This is a diagram of the deep learning model architecture for high-strength bolt instance segmentation provided in an embodiment of the present invention.

[0041] Figure 6 A schematic diagram of the C3k2 module provided in an embodiment of the present invention;

[0042] Figure 7 A flowchart for the detection of high-strength bolts in large amusement facilities based on machine vision, provided for this invention;

[0043] Figure 8 The flowchart of the high-strength bolt connection structure marking status analysis and failure mode judgment module provided by the present invention. Detailed Implementation

[0044] The technical solution of the present invention will now be clearly and completely described with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0045] Example 1

[0046] See attached document Figure 1 , 2 Sections 3 and 4 provide a further detailed description of the technical solution of this invention. Obviously, the described technical solution is only a part of this utility model, and not all of it. Based on the technical solution of this invention, applications such as using the detection system and method of this invention to detect and monitor high-strength bolts by fixing the detection device around the periphery of the high-strength bolt, as well as to detect ordinary bolts, also fall within the protection scope of this invention.

[0047] A machine vision-based high-strength bolt inspection system for large amusement facilities includes an image information acquisition unit, a data processing and storage module, a real-time data display module, a defect alarm module, a defect identification module, a defect marking module, a data communication module, and a remote expert system.

[0048] like Figure 1 The diagram shown illustrates the method for drawing a high-strength bolt anti-loosening line. Specifically, it includes anti-loosening line 1, bolt 2, nut 3, washer 4, and equipment base 5. Anti-loosening line 1 is a straight line drawn from bolt 2, through nut 3 and washer 4, to equipment base 5 after the high-strength bolt has been fully tightened and painted. The pigment used for this anti-loosening line must be fade-resistant and have a clear color contrast with the anti-corrosion paint to avoid affecting the inspection results. Anti-loosening line 1 should be drawn on the side that is easily inspected from the ground.

[0049] The image information acquisition unit is connected to the data processing and storage module, the data processing and storage module is connected to the real-time data display module, the real-time data display module is connected to the defect alarm module, the defect alarm module is connected to the defect identification module, the defect identification module is connected to the defect marking module and the defect information clearing and resetting module, the defect information marking module is connected to the data communication module, the remote expert system and the maintenance plan and suggestion module.

[0050] Mount the handheld industrial camera on the camera bracket, initialize the detection software, run the detection program, and select the preset alarm threshold according to the specifications and dimensions of the high-strength bolt being inspected. Then, select the high-strength bolt to be inspected, align the handheld industrial camera with the bolt, and adjust the camera parameters until the acquired anti-loosening line of the high-strength bolt is clearly displayed on the screen.

[0051] The inspection begins by sending images of high-strength bolts captured by an industrial camera to the data processing and storage module for processing. Pre-processing operations include image segmentation, marking extraction, and spatial relationship mapping. Based on a preset rule engine, a preliminary assessment is made to determine if the anti-loosening markings are broken, misaligned, or detached. For high-strength bolts where the anti-loosening markings continuously cover all components without relative movement, they are marked as normal and displayed via the real-time data display module. For bolts showing signs of broken or misaligned markings, a defect alarm mechanism is triggered, pushing relevant images and preliminary assessment information to the defect identification module for further analysis. The defect identification module identifies the alarmed defects, eliminating false alarms caused by unclear anti-loosening markings or oil obscuring the markings. Defects identified by the module are marked and transmitted to the equipment maintenance workshop via the data communication module. A remote expert system further analyzes the defect information, assesses its development trend, and provides detailed repair plans and suggestions. Simultaneously, the defect information is transmitted to the control room to alert operators to any changes in the high-strength bolts.

[0052] The working principle of the detection system in this embodiment is as follows:

[0053] (1) By examining connection failure cases related to the truss structure connecting bolts of large aerial sightseeing vehicles, the track and column connecting bolts of high-speed roller coasters, the tower connecting bolts of ultra-high sightseeing towers, and the connecting bolts of high-altitude bungee jumping towers in the large amusement facility inspection case database, a database of typical high-strength bolt failure models for large amusement facilities was established. Through summarization, common failure models such as Figure 2 As shown, there are six failure modes: bolt loosening, one of the two nuts being loose, both of the two nuts being loose, nuts not loosening, bolt elongation, one nut falling off, and both nuts falling off simultaneously.

[0054] (2) For high-strength bolts installed at lower positions, a high-resolution industrial camera is used for shooting. For high-strength bolts installed at higher positions, a drone is used for shooting, and a suitable light source system is provided to eliminate shadows and reflections and improve image quality.

[0055] (3) Image information acquisition settings, including software initialization, setting camera parameters, adjusting the image information acquisition device to align with the high-strength bolt being inspected, and keeping the camera perpendicular to the bolt plane as much as possible during shooting to avoid perspective distortion;

[0056] (4) Establish a database of typical high-strength bolt failure models for large amusement facilities and perform data preprocessing.

[0057] (5) High-strength bolt instance segmentation and component recognition: Train a deep learning instance segmentation model, input the preprocessed high-strength bolt image into the deep learning instance segmentation model, and complete the detailed component segmentation of the high-strength bolt;

[0058] (6) Unified extraction and connectivity analysis of marking structure: Mask data of category "mark_line" are automatically filtered and merged to generate a unified marking mask map. Then, a connected component analysis algorithm is executed to detect whether the marking is a complete and continuous structure and to determine whether it has broken. If there are two or more connected regions, it is considered that a structural break has occurred, and the spatial relationship analysis stage is entered.

[0059] (7) Spatial contact relationship between the markings and components: The spatial overlap of each broken marking segment with all component masks is calculated, and whether it is "attached" to a certain structural component is determined based on the pixel overlap area. The relationship between each marking segment and component is constructed into a structured mapping table;

[0060] (8) The spatial relationship mapping table in step (7) is input into the rule engine and pattern matching is performed in conjunction with the failure mode knowledge base. Based on the number of line segments, the attached component status and the missing status, it is inferred whether the current bolt has structural abnormalities and the corresponding failure label is output.

[0061] (9) Visualization and structured result output;

[0062] (10) If the system is determined to be in an abnormal state, an alarm will be triggered and the defect information will be marked.

[0063] (11) After the defect identification module has determined that there are no errors, the defect information needs to be finally calibrated;

[0064] (12) The data communication module transmits the calibrated high-strength bolt defect information to the remote expert system;

[0065] (13) The remote expert system further analyzes and diagnoses the defect information, judges the development trend of the defect, and provides status trend prediction, maintenance plan and suggestions and early warning.

[0066] Compared with existing manual inspection methods, the beneficial effects achieved by this invention are as follows:

[0067] 1. The machine vision-based high-strength bolt detection system and method for large amusement facilities provided by this invention can detect the loosening of high-strength bolts in complex structural components of various currently used large amusement facilities, and has a wide range of applications.

[0068] 2. The machine vision-based high-strength bolt detection system and method for large amusement facilities provided by this invention can conveniently detect the truss connection bolts of large aerial sightseeing vehicles, with high detection efficiency.

[0069] 3. The machine vision-based high-strength bolt inspection system and method for large amusement facilities provided by this invention can inspect high-positioned bolts such as those connecting the track and column of ultra-high-speed roller coasters, track connecting bolts, and connecting bolts of ultra-high sightseeing towers and high-altitude bungee towers. The operation is simple and convenient, and the defect location is accurate.

[0070] 4. The remote expert system of the machine vision-based high-strength bolt detection system and method for large amusement facilities provided by this invention can accurately judge the degree of loosening of high-strength bolts and provide detailed maintenance plans and suggestions.

[0071] 5. The detection method and system provided by this invention have a simple structure, are easy to operate, have high detection efficiency, and have good application value.

[0072] Example 2

[0073] like Figures 5 to 8 As shown, this second embodiment provides a machine vision-based method for detecting high-strength bolts in large amusement facilities, based on the first embodiment. The specific steps include:

[0074] S1. Preprocessing of database of typical high-strength bolt failure models for large amusement facilities;

[0075] In this embodiment, the database of typical high-strength bolt failure models for large amusement facilities is divided into the following proportions: training set: test set: validation set = 7:2:1. The training and test sets are used to train the deep learning model for high-strength bolt instance segmentation, and the validation set is used to predict the high-strength bolt instance segmentation deep learning model. The specific steps for constructing the database of typical high-strength bolt failure models for large amusement facilities in this embodiment are as follows:

[0076] S11. Generate a database of typical high-strength bolt failure models for large amusement facilities;

[0077] The database of typical high-strength bolt failure models for large amusement rides refers to the dataset used to train and predict the deep learning model for high-strength bolt detection in this embodiment, achieving instance segmentation and anomaly detection of high-strength bolts. The training dataset in this embodiment uses handheld and drone photography to capture failure cases related to the connection bolts of the truss structure of large aerial sightseeing vehicles, the connection bolts of high-speed roller coaster tracks and columns, the connection bolts of ultra-high observation towers, and the connection bolts of high-altitude bungee jumping towers. This establishes a database of typical high-strength bolt failure models for large amusement rides. The database covers six failure modes: bolt loosening, one nut loosening in a pair of nuts, both nuts loosening in a pair of nuts, nuts not loosening, bolt elongation, one nut falling off, and both nuts falling off simultaneously, as well as one normal mode. Figure 1-2 As shown.

[0078] To further enhance the coverage and diversity of the database and improve the robustness and generalization ability of deep learning models in complex environments, the database of typical high-strength bolt failure models for large amusement facilities also includes image data under different lighting, angles, and background conditions, constructing a sample set that is closer to real-world application scenarios.

[0079] S12, Label the failure model data of typical high-strength bolts in large amusement facilities;

[0080] To ensure the subsequent deep learning model can accurately perform high-strength bolt instance segmentation and anomaly detection tasks, this embodiment uses each image from the Labelme professional image annotation software's database of typical high-strength bolt failure models for large amusement facilities for instance segmentation and annotation. To better detect anomalies in high-strength bolts, this annotation requires component-level refinement of the high-strength bolt assembly. The annotation uses a polygon instance segmentation mode, which not only identifies the target's position in the image but also preserves its precise contour information. Annotation categories include: bolt_head (bolt top); top_nut (upper nut); bottom_nut (lower nut); washer (washer); base_plate (equipment base); and marking_line (anti-loosening marking line).

[0081] After image annotation is completed, all annotation information is saved in JSON format and mapped one-to-one with the original images, forming a structured annotation dataset. This annotation data will serve as an important input for subsequent training of deep neural network models, significantly improving the accuracy and reliability of the models in complex component identification and bolt condition determination.

[0082] S13. Dataset partitioning and preprocessing augmentation to generate a training dataset for typical high-strength bolt failure models of large amusement facilities.

[0083] After completing the segmentation and annotation of all high-strength bolt image instances and generating a JSON structured annotation dataset in step S12, the original annotation dataset is cleaned, filtered, formatted, sample-divided, and preprocessed and expanded to obtain a training dataset, a test dataset, and a validation dataset for a typical high-strength bolt failure model of a large amusement facility that meets the input requirements of a deep learning model.

[0084] S2. Construct a deep learning model for segmenting high-strength bolt instances in large amusement facilities, and train the high-strength bolt instance segmentation model on a database of typical high-strength bolt failure models in large amusement facilities.

[0085] High-strength bolt structures comprise multiple detailed components such as bolt heads, nuts, washers, and anti-loosening markings. The inspection task requires not only identifying the bolt's location but, more importantly, accurately distinguishing the shape, boundaries, and interrelationships of each component. However, target detection models can only provide bounding box localization, which is insufficient to meet the pixel-level accuracy requirements of structural recognition and cannot identify anomalies such as broken anti-loosening markings, loose nuts, or missing nuts. In contrast, instance segmentation models can output pixel-level masks for each component, facilitating more detailed structural analysis and subsequent anomaly identification. Therefore, this study prioritizes instance segmentation models, which can more accurately characterize the morphology of each bolt component and provide a solid foundation for subsequent S3 anomaly identification, improving the accuracy and practical value of anomaly detection.

[0086] This step uses the YOLOv11-Seg model as a foundation, and trains a deep learning model for high-strength bolt instance segmentation on the training dataset of typical high-strength bolt failure models for large amusement facilities obtained in step S13. The specific steps are as follows:

[0087] S21. Deep learning model structure for high-strength bolt instance segmentation;

[0088] In this embodiment, the deep learning model structure for high-strength bolt instance segmentation is based on the YOLOv11-Seg instance segmentation model. The overall structure of this model is mainly divided into three parts: the backbone network, the neck network, and the head network. These three main networks are composed of multiple smaller modules, which together constitute the basic structure of the entire model, such as... Figure 5 As shown.

[0089] The backbone network mainly consists of several CBS units (Conv+BN+SiLU) and an improved C3k2 module (cross-stage partial connections + dual / small kernel convolutions, balancing lightweight and expressive power), progressively compressing spatial resolution, expanding channel depth, and accumulating multi-scale speech. C3k2, for example... Figure 6 As shown. In the C3K2 module, when the parameter of the C3 module is True, the Bottleneck module is replaced by the C3 module, enabling deeper and more complex feature extraction. When the parameter is False, C3K2 uses the ordinary Bottleneck module. SPPF is introduced in the high-level semantic stage to quickly converge different receptive field contexts. After the SPPF module, the C2PSA module is added. This module is essentially an improvement of C2f, replacing the Bottleneck in C2f with PSA. PSA includes a feedforward neural network (FFN) that can further extract features, enabling the model to capture more complex nonlinear patterns, and a multi-head attention mechanism that allows the model to simultaneously focus on different regions of the input features, thereby obtaining richer feature information.

[0090] In the neck network, starting with the deepest features of the backbone, upsampling is performed and then concatenated with corresponding scale features from shallower layers. This concatenation is then refined using C3k2 to suppress redundancy and enhance cross-layer interactions. This process is repeated multiple times from top to bottom to recover high-resolution details. From bottom to top, the detailed high-resolution features are fed back into the low-resolution semantic map, forming a semantic-detail bidirectional flow. The neck network ultimately outputs several scale-fused feature maps, which are then sent to the segmentation head.

[0091] The head network comprises three segment nodes. In the location regression branch, two levels of standard convolutional layers (preferably a CBS structure of convolution + batch normalization + SiLU activation) are executed sequentially to fuse features, followed by location prediction via a convolutional layer. In the classification branch, depthwise separable convolution (DWConv) is used for feature fusion, and pointwise convolution is used to achieve information transfer between channels. Finally, a convolutional layer is used for classification prediction. The last branch is the mask coefficient branch, which generates the coefficient vector required for a linear combination with the prototype mask, supporting the construction of the instance segmentation results.

[0092] S22. Training the deep learning model structure for high-strength bolt instance segmentation;

[0093] Based on the high-strength bolt instance segmentation model structure designed in S21, the model is trained on the training dataset of typical high-strength bolt failure models of large amusement facilities obtained in S13. The specific steps are as follows:

[0094] The deep learning model for high-strength bolt instance segmentation is initialized to accelerate the subsequent training process and improve model performance.

[0095] The training dataset for the typical high-strength bolt failure model of large amusement facilities obtained in S13 was input into the model. This dataset contains a large number of labeled high-strength bolt images, which are used to train the model to segment the various components in the images.

[0096] Data Augmentation: Based on the characteristics of high-strength bolt images, five image augmentation methods are employed. Random rotation, scaling, and translation of the image are used to expand the relative position of the high-strength bolt area captured by the camera. Gaussian blur, brightness enhancement, and saturation enhancement are used to simulate shooting under different exposure conditions. Radiometric transformation is used to simulate the imaging effect of the camera on the high-strength bolt target at different angles or viewpoints.

[0097] a. Random rotation, scaling, and translation;

[0098] b. Gaussian blur;

[0099] Gaussian blur is used to reduce image noise and detail. Its principle is to convolve the image with a Gaussian distribution, essentially using the weighted average of surrounding pixels as the current pixel value. Since images are two-dimensional, a two-dimensional Gaussian distribution is needed, defined as follows:

[0100] ;

[0101] Using a 3x3 convolution kernel, a normalized weight matrix is ​​obtained.

[0102] c. Enhanced brightness and saturation;

[0103] The brightness and saturation of an image are randomly adjusted by converting its RGB three-channel model to an HSV three-channel model. In the HSV model, H (Hue) represents chromaticity, measured in degrees; S (Saturation) represents saturation, indicating how close the color is to the spectral color; and V (Value) represents brightness, indicating the lightness or darkness of the image colors. Converting from RGB to HSV requires first dividing the R, G, and B component values ​​by 255 to scale them to between 0 and 1, then performing the conversion using the following formula:

[0104] ;

[0105] ;

[0106] ;

[0107] After conversion, values ​​are assigned to the H, S, and V variables according to the random magnification factor, and finally converted back to the RGB model to complete the image enhancement of contrast and saturation.

[0108] d. Affine transformation;

[0109] Affine transformation is a common geometric transformation of images. It combines linear transformations such as scaling, rotation, translation, and shearing on an image while preserving the relationships between points, lines, and surfaces.

[0110] ;

[0111] Model training is performed on a training dataset of typical high-strength bolt failure models in large amusement facilities. In this embodiment, during training, the model calculates the prediction result and prediction error for each input image through forward propagation, and uses a loss function to evaluate the model's current performance. Next, the gradient of the loss function with respect to the model weights is calculated using the backpropagation algorithm. This algorithm updates the model weights through gradient descent, aiming to minimize the value of the loss function. Through a series of iterations, the model continuously learns and adjusts its parameters to more accurately segment high-strength bolt components.

[0112] In this embodiment, considering the class imbalance and complex morphology between the high-strength bolt component instance area and the background, a composite loss function consisting of binary cross-entropy loss and Dice loss is adopted, defined as follows:

[0113] ;

[0114] Throughout the training process, regularization techniques are employed to avoid overfitting and improve the model's generalization ability. A certain number of epochs is set, with the model processing the entire training dataset once in each epoch. Ultimately, a well-trained deep learning model for segmenting high-strength bolt component instances is obtained.

[0115] S3, High-strength bolt connection structure marking status analysis and failure mode judgment module;

[0116] S31. Segmentation and identification of high-strength bolt components;

[0117] A pre-trained deep learning model for high-strength bolt component segmentation was used. A validation dataset of typical high-strength bolt failure models in large amusement facilities was used as inference input. The model automatically identified and segmented various components in the high-strength bolt images. After inference, the model output a list containing all identified components. Each object not only has a clear category label but also includes a confidence score predicted by the model, representing the reliability of its classification result. Simultaneously, each target is accompanied by a pixel-level mask, a binary image with the same size as the input image, used to accurately describe the target's spatial location and contour boundaries in the image. The existence of this mask provides a foundation for subsequent structural relationship analysis and anomaly detection.

[0118] Output format:

[0119] Component category label: Indicates the type of high-strength bolted structural component to which the target belongs, all represented by identifiers from a predefined set of category labels;

[0120] Confidence score: The level of confidence the model gives in classifying the target, expressed as a floating-point number;

[0121] Pixel mask: A binary image with the same dimensions as the input image, used to accurately depict the spatial distribution and contour boundaries of the target in the original image. Each pixel mask uniquely corresponds to a target being identified, providing instance-level discrimination capability.

[0122] S32. Continuity analysis of high-strength bolt markings;

[0123] To achieve accurate assessment of the connection status of high-strength bolts, this embodiment aims to identify any abnormalities such as loose bolts, loose nuts, or detached bolts based on the continuity of the marking structure in high-strength bolt images. The markings, serving as auxiliary identification structures between nuts, washers, and bolts, directly reflect the relative positional changes of the physical structure through their geometric shape and topological integrity. Geometric connectivity analysis of the marking masks in high-strength bolt images provides effective input for subsequent spatial structure assessment and failure reasoning.

[0124] S321. Using the pixel mask output in step S31 as input, determine that each detailed component of the high-strength bolt has a pixel mask.

[0125] First, the model retrieves a complete list of target masks from S31, filtering out mask data with the component category label "marking_line". This mask is typically a binary image and needs to be uniformly processed into a single-channel binary mask image. If multiple independent marking line objects exist in the image, the model merges all marking line masks to generate a unified marking line structure mask image, which serves as input for subsequent connectivity analysis. In this step, the model checks whether each key component in the bolt system has a corresponding segmentation mask, ensuring the spatial relationship between marking lines and components is traceable. If some key components are found to be unidentified, a system alarm will be triggered to indicate potential segmentation anomalies.

[0126] S322. Use the "connected component analysis" algorithm to detect the continuity of high-strength bolt markings.

[0127] Connected component analysis (CFI) is an image region extraction method used to detect all independent sub-regions composed of adjacent pixels in an image. In step S321, after obtaining the uniform map mask, the model performs CFI to identify whether the map is a complete and continuous geometric structure. The specific steps are as follows:

[0128] 1) Use the marking mask obtained in step S31 as the input image, and use opencv.connectedComponents() to set the marking_line as an 8-bit single-channel image.

[0129] 2) The model performs connected component labeling on the marking line using the 8-adjacency criterion, and calls cv2.connectedComponentsWithStats(mask,connectivity=8) to obtain the label graph, the number of connected components, region statistics (including the size and area of ​​the circumscribed rectangle), and centroids. The choice of 8-adjacency allows pixel contacts along the diagonal direction to be considered continuous, which is more consistent with the true shape of the painted marking line and reduces the risk of misjudging the gaps due to narrow diagonal connections.

[0130] 3) After the initial connected component labeling is completed, the model traverses all regions except the background and removes small regions suspected of being noise based on the area threshold; then, it reassigns continuous label numbers (segment_1_mask, segment_2_mask,...) to the retained regions, generates a label map, and updates the region statistics table.

[0131] After executing the algorithm, the model extracts all disconnected marking regions in the image. If only one connected component exists, the marking can be considered as a continuous whole, indicating that the bolt system structure is stable; if two or more connected regions exist, it indicates that the marking has broken, which may reflect relative displacement or structural loosening between physical components.

[0132] S323, Structured output of the continuity analysis results of high-strength bolt markings;

[0133] After completing the connectivity analysis, the model outputs the results in a structured format. First, it outputs the number of segments into which the marking is broken; this number reflects the integrity of the marking and is crucial for subsequent fault diagnosis. Second, the model extracts the pixel mask corresponding to each connected region, forming a set of structured output marking mask data (segment_1_mask, segment_2_mask,...), used for spatial contact determination with components in subsequent spatial relationship analysis (S33). These masks not only serve as the basis for image-level structural analysis but also provide visual support, facilitating manual verification and anomaly review.

[0134] The specific format of the output result is as follows:

[0135] Number of segments: The number of line segments;

[0136] Label mask set: The model extracts the pixel positions corresponding to each non-background connected region from the label image and generates independent binary mask images segment_i_mask (where i=1,2,...,num_segments). These masks have the same size as the original image, and each mask image only contains the foreground pixels of the corresponding connected region (value 1, others 0). The output is a set of indexable masks.

[0137] {

[0138] "num_segments":N,

[0139] "segment_masks":{

[0140] "segment_1_mask":[H×W binary image],

[0141] "segment_2_mask":[H×W binary image],

[0142] ...,

[0143] "segment_N_mask": [H×W binary image]

[0144] }

[0145] }

[0146] Marker mask diagram.

[0147] S33, Spatial relationship mapping of high-strength bolt detailed components;

[0148] After detecting the continuity of the markings, the spatial relationship mapping of each component of the high-strength bolt is performed. If the markings are continuous, there is no need to perform spatial relationship mapping. If the output markings are not connected, it is necessary to determine which physical components each segment of the marking is attached to.

[0149] After detecting a fracture in the marking structure, relying solely on segment number information is insufficient to infer whether the high-strength bolts exhibit actual physical structural anomalies. To achieve more discriminative model state recognition, this embodiment further analyzes the spatial relationship between each marking segment and other components in the bolt system, thereby establishing a clear structural contact mapping. This spatial relationship mapping not only reveals the component information to which the fractured marking is attached but also infers whether potential failure behaviors such as bolt loosening, nut loosening, or detachment exist. It serves as an important intermediate expression for high-level judgment by the rule engine. The specific steps are as follows:

[0150] S331. Calculate the overlap between the gradation mask and the component mask;

[0151] The model iterates through each segment mask (segment_i_mask) output in step S323, using it as a reference object and comparing it with all other component masks in the image. For each pair of segment and component masks, the spatial overlap is calculated. The overlap is evaluated using the absolute number of overlapping pixels.

[0152] The absolute number of overlapping pixels refers to the actual number of pixels that overlap between masks. The model sets a threshold (overlapping pixels > 50) to determine whether a line segment "touches" a component. Then, a redundant judgment mechanism is introduced to allow a line segment to touch multiple components simultaneously, in order to avoid judgment errors caused by boundary blurring.

[0153] S332. Establish the structural mapping relationship between the marking segments and bolt components to obtain the spatial relationship mapping table.

[0154] After completing the overlap analysis of all marking segments and components, the model summarizes the relationship between each marking segment and the component it contacts into a structured spatial mapping table.

[0155] For example, if segment_1_mask overlaps with both top_nut and washer, while segment_2_mask only touches bolt_head, then the corresponding mapping table can be represented as follows:

[0156] {

[0157] "segment_1_mask":["top_nut","washer"],

[0158] "segment_2_mask":["bolt_head"]

[0159] }

[0160] This mapping table clearly expresses the specific set of components to which each line is "attached," and serves as an important intermediate representation for the model to determine failure modes.

[0161] After the S333 spatial relationship mapping table is constructed, the model uses it as input and combines it with a predefined structural rationality model (normal state) to automatically analyze the mapping results.

[0162] In a standard structural configuration, the markings should be a single, continuous mask that simultaneously contacts all major components. If multiple marking mask segments or missing component masks exist, it indicates an abnormal state. The model can define several structural anomaly patterns, which are automatically categorized and judged based on the attachment relationships of the marking segments.

[0163] Through this step, the system completes the intermediate reasoning loop from "marking breakage" to "structural status determination", providing highly discriminative structural contact evidence for the final rule engine judgment (S34), significantly enhancing the accuracy and reliability of image-level bolt system anomaly detection.

[0164] S34, rule engine decision-making;

[0165] This step, based on the aforementioned segmentation recognition (S31), marking breakage detection (S32), and spatial relationship mapping (S33) results, combined with the high-strength bolt failure mode knowledge base, uses logical rules to determine whether the bolt in the current image exhibits failure behavior and its failure type. Failure Mode Reference Figure 2 As shown, the following typical scenarios are included:

[0166] The model first analyzes the number of broken road markings, and then constructs a component status map by combining the specific components to which each road marking is attached. For example, it determines whether a certain road marking is still in contact with a nut, washer or screw, whether there is a suspended mask that is not in contact with any component, and whether there are signs of physical structure detachment (mask missing).

[0167] Based on five components—the bolt top, upper nut, lower nut, washer, and equipment base—the marking is divided into five segments for continuous rule engine specification. The rule engine combines the following core reasoning paths:

[0168] If only a single marker segment can be detected, and it continuously covers all components from top_nut to washer to bolt_head, then the current structure is considered to be in a normal state.

[0169] When the model detects a break in the gradation mask, with one segment primarily in contact with the upper part of the bolt while other segments maintain normal attachment to the nut and washer, it can be inferred that the bolt has undergone axial displacement, leading to loosening. Figure 2 (a);

[0170] If the marking is detected to be divided into three segments, with the first segment attached to the upper area of ​​the screw, the second segment attached to the upper nut, and the third segment attached to the lower nut and washer, and the first and third segments are not offset, while the second segment is significantly offset from the other two segments, then it can be inferred that one of the nuts in the current double-nut structure has become loose. Figure 2 (b);

[0171] When it is detected that the marking is divided into three segments, and multiple marking segments form discontinuous contact with the upper and lower nuts respectively, and the vertical spacing between the segments is abnormally increased, it indicates that both nuts may have become loose simultaneously. Figure 2 (c);

[0172] When a marking is detected to be split into two segments, and the marking mask has normal contact with all components, but the vertical height of the structure is detected to significantly exceed the normal standard, it is inferred that the screw has been elongated. Figure 2 (d);

[0173] If a nut mask is missing from the test results, and a section of the marking mask does not contact any component mask, it indicates that the nut has detached from the structure. Figure 2 (e);

[0174] If the test results show that the masks for both nuts are completely missing, and only the screw and washer structure are visible, and the marking mask is divided into multiple segments, with almost all of these segments not in contact with any component mask, it is determined that both nuts have fallen off simultaneously. Figure 2 (f).

[0175] Rule pseudocode:

[0176] IF Only one entire line mask is detected AND the line continuously covers top_nut→washer→bolt_head THEN

[0177] Status = Normal

[0178] / / ELSEIF line mask is broken into two AND segments

[0179] A section that primarily contacts the upper region of bolt_head AND

[0180] The other section remains attached to the nut and washer.

[0181] Status = Single nut loose

[0182] / / ELSEIF markings are divided into three segments AND

[0183] The first part contacts the upper part of bolt_head AND

[0184] The second part involves top_nutAND

[0185] The third paragraph touches on bottom_nut and washerAND

[0186] The first and third segments show no obvious shift, but the second segment is significantly shifted from the other two segments.

[0187] State = One of the nuts in the double-nut structure is loose

[0188] / / ELSEIF markings are divided into three segments AND

[0189] The marker segments touch the top_nut and bottom_nut respectively, but the vertical spacing between them is abnormally increased.

[0190] Status = Both nuts are loose

[0191] / / ELSEIF lines are split into two segments AND

[0192] Each segment is properly attached to the component (without offset) BUT

[0193] The overall vertical height exceeds the standard threshold.

[0194] State = Screw elongation

[0195] / / ELSEIF missing a nut mask AND

[0196] There exists a section of the gamut mask that does not contact any components.

[0197] Status = Single nut detached

[0198] / / ELSEIF missing two nut masks AND

[0199] The markings are divided into multiple segments AND

[0200] Most of the marking segments did not contact any components.

[0201] Status = Both nuts have fallen off

[0202] ELSE

[0203] State = Abnormal Mode

[0204] ENDFUNCTION

[0205] During rule execution, the contact determination between masks uses an IoU greater than 0.5 as the effective contact threshold, the inter-segment position offset is greater than 2% of the image width as the basis for judging significant offset, the structure height exceeds 1.2 times the normal value as the screw elongation occurs, and the proportion of the marking segment that does not contact any component exceeds 70% is considered a serious structural failure.

[0206] S35 Result Output

[0207] Based on the structural state judgment results obtained from the rule engine decision module (S34), the analysis results are formatted and output, generating a visual diagnostic chart and a text report. Output results:

[0208] 1) Structural condition diagnosis conclusions

[0209] Output the current status label of the bolt system, including categories such as "structural normal", "screw loose", "one nut loose", "two nuts loose", "one nut detached", "two nuts detached simultaneously", or "screw elongated".

[0210] 2) Display of segmentation and spatial relationship results

[0211] Output the following image-level intermediate results:

[0212] Original input image;

[0213] Instance segmentation result image (all parts are labeled with category and mask outline);

[0214] Pitch breakage analysis diagram (each pitch segment is marked with a different color and labeled with numbered labels such as segment_1_mask, segment_2_mask, etc.);

[0215] Diagram showing the spatial contact relationship between the markings and components (using arrows to represent segment-to-part connections).

[0216] 3) Structured data output;

[0217] Generate a standardized JSON format diagnostic report:

[0218] JSON output example:

[0219] {

[0220] "image_id":"image***",

[0221] "failure_mode":"***",

[0222] "num_segments":***,

[0223] "missing_parts":[],

[0224] "attachment_map":{

[0225] "segment_1_mask":["***","***"],

[0226] "segment_2_mask":["***"]

[0227] },

[0228] "structure_height":***

[0229] }

[0230] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some or all of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. A machine vision-based high-strength bolt inspection system for large amusement facilities, characterized in that, include: The system comprises an image information acquisition unit, a data processing and storage module, a real-time data display module, a defect alarm module, a defect identification module, a defect marking module, a data communication module, and a remote expert system. These modules work in tandem to achieve the acquisition, processing, identification, alarm, and subsequent maintenance guidance of defects in high-strength bolts.

2. The machine vision-based high-strength bolt inspection system for large amusement facilities according to claim 1, characterized in that, The image information acquisition unit is connected to the data processing and storage module, the data processing and storage module is connected to the real-time data display module, the real-time data display module is connected to the defect alarm module, the defect alarm module is connected to the defect identification module, the defect identification module is connected to the defect information marking module and the defect information clearing and resetting module, the defect information clearing and marking module is connected to the data communication module and transmits the defect information to the remote expert system, and the remote expert system provides maintenance solutions and suggestions.

3. The machine vision-based high-strength bolt detection system for large amusement facilities according to claim 1, characterized in that, The image information acquisition unit includes a handheld image acquisition component and a drone image acquisition component. The handheld image acquisition component includes a handheld camera bracket, an industrial camera, an electronic display screen, and a light source. It is mainly used to acquire images of high-strength bolts that are relatively low in position and easy to shoot at close range. The drone image acquisition component includes a drone and an onboard industrial camera and a light source. It is mainly used to acquire images of high-strength bolts that are high in position and cannot be easily shot by the handheld component, so as to achieve full-scene coverage acquisition.

4. The machine vision-based high-strength bolt inspection system for large amusement facilities according to claim 1, characterized in that, The defect alarm module automatically alarms when the data processing and storage module initially determines that the "relative movement of the anti-loosening line position" is abnormal, triggering the defect identification module to further confirm the abnormal event.

5. The machine vision-based high-strength bolt detection system for large amusement facilities according to claim 1, characterized in that, The defect identification module accurately identifies the alarm information pushed by the defect alarm module. Through image feature analysis, it eliminates false alarms caused by non-bolt failure factors, such as unclear anti-loosening line marks, oil stains obscuring the anti-loosening line, and light reflection, and confirms and marks the real defects.

6. The machine vision-based high-strength bolt detection system for large amusement facilities according to claim 1, characterized in that, The remote expert system receives confirmed defect information transmitted by the data communication module, further analyzes the defect type and severity, judges the defect development trend, and generates detailed maintenance plans and operation suggestions. At the same time, defect information is transmitted to the control room to remind maintenance personnel to pay attention to the status changes of the corresponding high-strength bolts in real time.

7. A detection method for a machine vision-based high-strength bolt detection system for large amusement facilities as described in any one of claims 1 to 6, characterized in that it includes the following steps: S1. Preprocessing of database of typical high-strength bolt failure models for large amusement facilities; S2. Construct a deep learning model for high-strength bolt instance segmentation in large amusement facilities, and train the high-strength bolt instance segmentation model on a database of typical high-strength bolt failure models in large amusement facilities. S3, Module for analyzing the status and determining the failure mode of markings on high-strength bolted connection structures.

8. The detection method according to claim 7, characterized in that, Step S1 includes: S11. Generate a database of typical high-strength bolt failure models for large amusement facilities; S12, Label the failure model data of typical high-strength bolts in large amusement facilities; S13. Dataset partitioning and preprocessing augmentation to generate a training dataset for typical high-strength bolt failure models of large amusement facilities.

9. The detection method according to claim 7, characterized in that, Step S2 includes: S21. Deep learning model structure for high-strength bolt instance segmentation; S22. Training the deep learning model structure for high-strength bolt instance segmentation.

10. The detection method according to claim 7, characterized in that, Step S3 includes: S31. Segmentation and identification of high-strength bolt components; S32. Continuity analysis of high-strength bolt markings, specifically including: S321. Using the pixel mask output in step S31 as input, determine that each detailed component of the high-strength bolt has a pixel mask; S322. Using the "connected component analysis" algorithm to detect the continuity of the high-strength bolt markings; S323. Structured output of the high-strength bolt marking continuity analysis results; S33, Spatial relationship mapping of high-strength bolt detailed components, specifically including: S331, calculating the overlap between the marking mask and the component mask; S332, establishing the structural mapping relationship between the marking segment and the bolt component to obtain the spatial relationship mapping table; S333, after the spatial relationship mapping table is constructed, the model uses it as input and combines it with the predefined structural rationality model to automatically analyze the mapping results; S34, rule engine decision-making; S35 output results.

Images