A virtual maintenance data processing system for maintenance scenario optimization

Abstract

The present invention relates to the field of data processing technology, and specifically discloses a virtual maintenance data processing system for optimizing maintenance scenarios, comprising an optical contact probe, a scanning measurement system, an image recognition and defect analysis module, an image-assisted measurement module, a part visual tracking module, an augmented reality assembly guidance module, a data fusion and state modeling module, and a human-computer interaction and task incentive module. The present invention integrates optical contact measurement, image recognition, point cloud modeling, and augmented reality guidance to achieve intelligent and high-precision execution of defect detection, dimension measurement, assembly correction, and state modeling during the disassembly and reassembly of a gearbox.

Description

Technical Field

[0001] The present invention relates to the field of data processing technology, and more particularly to a virtual maintenance data processing system for optimizing maintenance scenarios. Background Art

[0002] The charging pump (high-pressure charging pump) is a critical safety system, and its gearbox is responsible for high-load, high-precision power transmission. Due to the complex long-term operating environment, the gearbox is prone to gear meshing anomalies (such as uneven wear, pitting, and spalling), bearing damage, unqualified fit clearances, and reinstallation errors. During maintenance, the disassembly and restoration of the gearbox involves typical high-difficulty processes such as precise disassembly and marking, multi-dimensional measurement and recording (of mating surfaces, tooth pitches, and meshing angles), condition assessment and damage judgment, part repair / replacement and reinstallation, and reinstallation accuracy confirmation and functional verification. The disassembly process is characterized by numerous parts, complex structures, confusing position markings, and a high risk of reinstallation errors. Traditional measurement tools are inefficient for measuring high-precision clearances, tooth profiles, and meshing angles. The measurement methods are complex, tooth surface defect identification is highly subjective, and relying on experience to achieve standardized maintenance is difficult. The complex assembly structure of the gearbox repair leads to difficulty in identifying reinstallation posture errors, low structural defect identification accuracy, scattered measurement data that is difficult to integrate and model, and manual task execution is prone to errors and lacks an incentive feedback mechanism, as well as intelligent adaptation and learning capabilities. Summary of the Invention

[0003] In order to overcome the above-mentioned defects of the prior art, the present invention provides a virtual maintenance data processing system for optimizing maintenance scenarios, which realizes the precision, intelligence and visualization of maintenance processes through virtual maintenance data processing that operates in collaboration with multi-source fusion, augmented reality guidance, state modeling and task incentives.

[0004] To achieve the above object, the present invention provides the following technical solutions:

[0005] A virtual maintenance data processing system for maintenance scenario optimization includes an optical contact probe, a scanning measurement system, an image recognition and defect analysis module, an image-assisted measurement module, a part visual tracking module, an augmented reality assembly guidance module, a data fusion and state modeling module, a human-computer interaction and task incentive module. The optical contact probe performs contact measurement of nanometer-level size and displacement data of key parts of the gear meshing surface, shaft shoulder, matching hole, and gasket in the gearbox. The scanning measurement system obtains three-dimensional point cloud data of key areas of the gearbox. The image recognition and defect analysis module collects image data of the target area of the gearbox, detects defects such as tooth surface wear, cracks, pitting, and spalling through a recognition algorithm based on deep learning, and jointly judges the posture deviation and poor contact of the assembly state. The image-assisted measurement module performs geometric analysis through the calibration benchmark and structural feature points in the figure, and forms a multi-source comparison with the point cloud model and contact measurement results. The part visual tracking module performs geometric analysis through the calibration benchmark and structural feature points in the figure, and forms a multi-source comparison with the point cloud model and contact measurement results. The image and unique identification of parts are recorded during the process, the whole process tracking and assembly sequence are managed through image registration, and the state model is combined to detect missing parts and verify assembly in the reassembly stage. The augmented reality assembly guidance module spatially aligns the standard 3D assembly model with the current on-site image, identifies assembly deviations in real time and generates a posture error heat map, combines image recognition and point cloud data to jointly drive assembly correction guidance, and performs multi-source information enhanced posture adjustment prompts. The data fusion and state modeling module fuses the optical probe measurement results, image recognition output and point cloud morphology data to construct a digital twin model of the gearbox assembly status, and generates personalized virtual maintenance suggestions through trend analysis, structural degradation judgment and residual life assessment. When the human-computer interaction and task incentive module identifies potential risk areas and key operation nodes, it actively generates measurement and assembly tasks, and combines task scoring, behavior confidence assessment and achievement incentive strategies to guide operators to complete the gearbox maintenance actions.

[0006] As a further technical solution of the present invention, the image recognition and defect analysis module includes a recognition unit based on a deep convolutional neural network. The network structure of the recognition unit includes a feature extraction layer, a region proposal network and a classification regression branch. By introducing an attention channel, weighted learning is performed on the spatial area of the gear surface image to extract crack edge and texture degradation features in the presence of local occlusion and oil interference.

[0007] As a further technical solution of the present invention, the image-assisted measurement module includes a two-dimensional image correction unit and a size conversion module, and is provided with image and point cloud error comparison control logic. The two-dimensional image correction unit is used to perform geometric distortion correction and angle projection correction on the collected original image, solve the image measurement error problem caused by lens distortion, shooting angle tilt or perspective distortion, correct fisheye distortion, barrel or pincushion deformation, and convert the tilted image into a standard front view through geometric transformation to ensure the geometric authenticity of the calibration reference and structure edge in the image, laying the foundation for subsequent accurate size extraction. The size conversion module converts the pixel unit in the image into the actual physical size unit (such as mm) and performs structural feature The size calculation is based on the calibration reference objects in the drawing (such as rulers, correction blocks) or preset scale factors, calculates the pixel-millimeter conversion coefficient, extracts the geometric distances between key structural edges, hole positions, center points and other features, and outputs two-dimensional measurement data for comparison with point cloud measurement results or probe measurement results. It supports dimensional anomaly warnings and preliminary judgments of assembly deviations. When the difference between the image measurement result and the point cloud measurement result is greater than 0.5mm, the image and point cloud error comparison control logic performs automatic image recalibration and size mapping coefficient adjustment operations until the measurement error is lower than the set tolerance range of 0.5mm. The scanning measurement system is a structured light scanner with a 0.05mm point pitch resolution and a scanning range of 300mm. 300mm. During the acquisition process, point cloud data from five perspectives are obtained through several angles. The point cloud data is aligned and used to generate a three-dimensional model of the gearbox surface.

[0008] As a further technical solution of the present invention, the augmented reality assembly guidance module includes an image feature point matching submodule and a point cloud registration submodule. The image feature point matching submodule (used to extract and match key feature points between the real-time collected assembly site image and the standard assembly model image to achieve preliminary positioning and posture estimation of the assembly parts in the two-dimensional image) and the point cloud registration submodule (used to spatially align the three-dimensional point cloud data obtained by structured light scanning or depth camera with the standard three-dimensional assembly model to accurately calculate the position and posture of the current part in the three-dimensional space) are connected to a heat map generation unit. The outputs of the image feature point matching submodule and the point cloud registration submodule are combined to calculate the three-dimensional rigid body transformation matrix of the assembly part relative to the standard assembly model through least squares registration. When the rotation amount of the three-dimensional rigid body transformation matrix is greater than 3° and the translation amount is greater than 2 mm, the heat map generation unit is triggered to superimpose a red warning block in the AR display.

[0009] As a further technical solution of the present invention, in the augmented reality assembly guidance module, the process of detecting the position and posture changes of the assembly parts in three-dimensional space when the operator performs the reassembly operation includes:

[0010] Step 1: Image acquisition and initial feature extraction: Collect the image of the reassembled component, apply the SHIF algorithm to extract the feature points of the reassembled component, and call the standard assembly model to establish the target area to be registered;

[0011] Step 2: Feature matching registration based on center point correction: The center point of the bolt on the gear in the detection target is calculated and corrected. If the overall component registration deviation exceeds the preset threshold within the set range, but the target center point alignment error is less than the set threshold, it is judged as surface misalignment and the transformation matrix is recalculated using the center point as the reference;

[0012] Step 3: Edge contour detection and line fitting: Extract the edge contour of the target component, use the Hough line fitting method to determine its boundary direction, and obtain the assembly rotation angle error by comparing the boundary direction with the standard direction.

[0013] Step 4: Point cloud-assisted spatial pose detection: Based on the point cloud data acquired by the scanning measurement system, the system automatically switches to 3D point cloud pose analysis after image registration. The region of interest is mapped to the point cloud space, the point cloud of the target area is intercepted, and RANSAC plane fitting is performed on the target assembly surface. The spatial distance, parallelism, and angle between the top and bottom surfaces are analyzed to determine whether there is any misalignment or incomplete insertion.

[0014] Step 5: Error Analysis and Type Judgment: Classify and judge assembly errors based on the image and point cloud information. Calculate translation errors using center point offset or point cloud center point difference. Calculate rotation errors using boundary line direction or fitted plane normal. Calculate vertical errors using point cloud surface distance to determine if the assembly is tight. Identify incomplete insertions when the edge contour difference area is greater than a set threshold or when an incomplete fit is not formed.

[0015] Step 6: Abnormal filtering and self-learning: For areas with abnormal center point distribution and abnormal contour disturbance, perform assembly error recognition optimization mechanism evaluation based on dynamic confidence weighting, and adaptively optimize the recognition algorithm.

[0016] As a further technical solution of the present invention, in the augmented reality assembly guidance module, an assembly error recognition optimization mechanism based on dynamic confidence weighting generates a confidence score for each assembly error recognition result. The confidence score is obtained by weighted summation of the number of matching points, boundary integrity, point cloud surface closure rate, center point offset and image clarity. If the confidence score is lower than the set threshold, the control pauses the task push and prompts the operator to re-collect the image. The assembly error recognition optimization mechanism based on dynamic confidence weighting includes a regression learning model based on operation behavior correction feedback, which records and marks the error correction between the operator's actual correction behavior and the recognition guidance result during the assembly process, and dynamically adjusts the parameter weights of image and point cloud recognition through a lightweight regression learning model to perform self-learning, self-optimization and confidence adaptive prompts of the assembly error recognition algorithm.

[0017] As a further technical solution of the present invention, the human-computer interaction and task incentive module includes a behavior data backtracking and strategy optimization mechanism, which is used to record the operator's execution path, assembly adjustment behavior, error level changes and confidence fluctuation process in each task. It generates a behavior log based on time series and performance labels, and adaptively optimizes and adjusts the recommended path, recognition parameter configuration and incentive level weight of subsequent tasks. The human-computer interaction and task incentive module uses the task scoring formula to calculate the task score. The task scoring formula is:

[0018] Where: Scoring tasks is used to update the operator's incentive level based on the score, and to control their task unlocking permissions and challenge prompt push. Scoring of task benchmarks, which are pre-set by administrators and engineers for each type of task. is the measurement error rate, which is equal to the mean absolute error rate of the optical touch probe, ,in is the number of measurement points involved in the task, The index of the measurement point for this task, 、 The tasks are The measured and expected values of the measurement points, To identify the confidence coefficient, the Softmax layer classification probability output by the deep learning network from the image recognition module is used. ,in is the normalized value of the matching points and the total points, To identify the mean of the pairwise residuals, is the normalized value of the boundary closure rate, is the normalized value of the center point offset ratio, is the normalized value for measuring image clarity, 、 、 、 、 is the weight factor, , automatically lock the task submission button and pop up a prompt box "Please collect again".

[0019] As a further technical solution of the present invention, in the data fusion and state modeling module, the digital twin model is based on the three-dimensional CAD model as the structural basis, and a parameter set that can update the status is set for each key component. The parameter set includes wear level, participating thickness, cumulative loading time and deviation history. The model update cycle is refreshed immediately after each measurement and identification task, and is used to determine whether to enter the maintenance window. The wear level includes 0 to 5 levels, the unit of participating thickness is mm, and the unit of cumulative loading time is h.

[0020] As a further technical solution of the present invention, the parts visual tracking module uses image texture hash coding to compare with the preset parts sequence dictionary, and verifies in real time during the reassembly process whether the image ID of the current component matches its preset position. In the event of misalignment, sequence jump, and direction flipping, the maintenance task is controlled to be paused and a prompt box of "assembly sequence abnormality" is popped up.

[0021] The technical effects of the virtual maintenance data processing system for optimizing maintenance scenarios of the present invention are as follows:

[0022] The present invention effectively improves the intelligence level of assembly precision control, structural defect detection, maintenance data modeling and operation task management through the deep integration of multi-source data collection, intelligent identification, three-dimensional alignment, error judgment and guidance incentives throughout the entire process of gearbox disassembly and reassembly. In the assembly guidance process, the present invention combines image recognition and point cloud posture recognition algorithms to dynamically obtain the spatial posture changes of components. By identifying translation errors, rotation errors and misfit states, it accurately generates hot zone prompts and corrects them in real time to ensure high-precision assembly. At the same time, it uses deep learning algorithms to achieve high-robust identification of tooth surface cracks, wear, and pitting defects, enhances the structural health assessment capability of the maintenance process, and achieves quantitative feedback and strategy optimization of operation quality through scoring mechanisms and behavioral data retrospective analysis, guiding maintenance personnel to actively complete high-quality operations, effectively avoiding human problems such as wrong installation and missing installation, and realizing the transformation of gearbox maintenance from experience-driven to data-driven, significantly improving maintenance efficiency, safety and intelligence. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] Figure 1 This is a schematic diagram of the optical contact probe of the present invention measuring the inner diameter of a cylinder;

[0024] Figure 2 This is an explosion demonstration interface diagram of the gearbox disassembly and overhaul of the present invention;

[0025] Figure 3 This is a diagram of the disassembly interface of the gearbox base during the disassembly and overhaul of the gearbox according to the present invention;

[0026] Figure 4 This is a diagram of the disassembly interface of the gearbox upper cover during the disassembly and overhaul of the gearbox according to the present invention;

[0027] Figure 5 Continuously play interface diagrams for the gearbox disassembly and repair steps of the present invention;

[0028] Figure 6 It is the comparison interface between the measurement model of the present invention and the standard model;

[0029] Figure 7 This is the gearbox component defect identification interface of the present invention. DETAILED DESCRIPTION

[0030] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of the present invention.

[0031] Example 1

[0032] The present invention proposes a virtual maintenance data processing system for optimizing maintenance scenarios, which includes an optical contact probe, a scanning measurement system, an image recognition and defect analysis module, an image-assisted measurement module, a part visual tracking module, an augmented reality assembly guidance module, a data fusion and state modeling module, and a human-computer interaction and task incentive module. The optical contact probe performs contact measurement of nanometer-level dimensions and displacement data on key parts of the gear meshing surface, shaft shoulder, mating hole, and gasket in the gearbox (such as Figure 1 As shown in the figure, the scanning measurement system obtains 3D point cloud data of the key area of the gearbox, and the image recognition and defect analysis module collects image data of the target area of the gearbox, and detects defects such as tooth surface wear, cracks, pitting and spalling through a recognition algorithm based on deep learning (as shown in the figure). Figure 7The image-assisted measurement module performs geometric analysis using calibrated datums and structural feature points in the image, and then performs multi-source comparison with the point cloud model and contact measurement results. The part visual tracking module records images and unique identifiers of components during gearbox disassembly, manages the entire tracking and assembly sequence through image registration, and combines the state model for missing part detection and assembly verification during the reassembly phase. The augmented reality assembly guidance module spatially aligns the standard 3D assembly model with the current on-site image, identifies assembly deviations in real time, and generates a pose error heatmap. It combines image recognition and point cloud data to drive assembly correction guidance, providing pose adjustment prompts enhanced by multi-source information. The data fusion and state modeling module integrates optical probe measurement results, image recognition output, and point cloud topography data to construct a digital twin model of the gearbox assembly state. It then generates personalized virtual maintenance recommendations through trend analysis, structural degradation judgment, and residual life assessment. The human-computer interaction and task incentive module proactively generates measurement and assembly tasks when identifying potential risk areas and key operation nodes. It then combines task scoring, behavior confidence assessment, and achievement incentive strategies to guide operators through gearbox maintenance actions.

[0033] It should be noted that the image recognition and defect analysis module includes a recognition unit based on a deep convolutional neural network. The network structure of the recognition unit includes a feature extraction layer, a region proposal network and a classification regression branch. By introducing an attention channel, weighted learning is performed on the spatial area of the gear surface image to extract crack edge and texture degradation features in the presence of local occlusion and oil interference.

[0034] First, the collected gearbox images are processed with grayscale equalization, noise reduction and edge enhancement. The training data set is expanded using rotation, scaling and brightness perturbation to improve the generalization ability of the model. The oil-covered and occluded defect images are labeled with key points or bounding boxes for supervised training. The overall recognition model adopts the improved Faster R-CNN structure. ResNet-50 is used as the backbone to extract features in the feature extraction layer. The texture, edge and morphological features of the gearbox images are extracted from several set scales, and the detection of small cracks is supported. In the region proposal network (RPN), a sliding window is generated on the feature map to generate candidate regions. The candidate frames are generated by combining anchor frames with preset sizes and aspect ratios. The candidate defect regions are generated using foreground / background classification and regression calculations. In the classification and regression branches, each proposed region is cropped and sent to the RoI The Pooling layer unifies the scale. Branch one performs defect classification (cracks, spalling, pitting), and branch two regresses and predicts the precise position of the bounding box (for subsequent defect positioning and measurement). The SE (Squeeze-and-Excitation) channel attention mechanism is introduced into the feature extraction backbone to perform global average pooling on the responses of each channel, learn the weight relationship between channels, weightedly enhance important channels, and weaken the influence of non-critical textures. Spatial attention can also be introduced to guide the model to focus on shallow crack areas and fuzzy areas. The detection defect targets extracted and output from abnormal areas include defect classification labels, defect bounding boxes, defect area confidence (Softmax output), crack length, area and other structured data.

[0035] It should be noted that the image-assisted measurement module includes a two-dimensional image correction unit and a size conversion module, and is equipped with an image and point cloud error comparison control logic. When the difference between the image measurement result and the point cloud measurement result is greater than 0.5mm, the image and point cloud error comparison control logic performs automatic image recalibration and size mapping coefficient adjustment operations until the measurement error is lower than the set tolerance range of 0.5mm. The scanning measurement system is a structured light scanner with a 0.05mm point pitch resolution and a scanning range of 300mm. 300mm. During the acquisition process, point cloud data from five perspectives are obtained through several angles. The point cloud data is aligned and used to generate a three-dimensional model of the gearbox surface.

[0036] The charging pump gearbox is a critical piece of equipment in nuclear power plants. It contains multiple gear sets, bearings, shaft shoulders, gaskets, and other high-precision components. During disassembly and reassembly, precise measurements of dimensions such as the mating aperture, axial clearance, and gasket thickness are required. Image measurement is subject to interference from factors such as structural obstruction and non-orthogonal viewing angles, making error control particularly critical. Setting up image and point cloud error comparison control logic allows for dynamic correction of image measurements to ensure reliable measurements of key areas. The charging pump gearbox is fixed in position and limited space, making it difficult for maintenance personnel to adjust lighting or posture. While point cloud data is highly accurate, it is prone to voids in grooves and blind spots around edges. Image measurement offers the advantages of texture completion and contour tracking, but point cloud calibration is required for reliable measurements. Using structured light scanning results as a trusted benchmark enables self-calibration of the image measurement process. Failure of the transmission mechanism of the charging pump gearbox can result in serious production safety accidents. Setting a 0.5mm error comparison trigger threshold is a rigid requirement based on nuclear equipment maintenance technical standards and is not an arbitrary parameter.

[0037] In the disassembly and maintenance scenario of the charging pump gearbox, due to the complex equipment structure, compact internal space, limited operating space, and the presence of a large number of closely fitting bearing holes, meshing surfaces, and gasket parts, it is difficult for the traditional single measurement method to simultaneously take into account both measurement efficiency and accuracy requirements. In the present invention, the image-assisted measurement module realizes rapid dimension extraction through a two-dimensional image correction unit and a dimension conversion module, and sets up image and point cloud error comparison control logic. With the help of the high-resolution point cloud data obtained by the structured light scanning system as the error benchmark, when the difference between the image measurement result and the point cloud result exceeds 0.5mm, the image recalibration and mapping coefficient adjustment operation are automatically triggered until the error converges to the set tolerance range within the nuclear-grade maintenance requirements. This mechanism is particularly suitable for key scenarios such as nuclear power charging pump gearboxes where high-precision measurement and reinstallation verification need to be completed in a limited space. It significantly improves the availability of image measurement and data fusion consistency, and meets the high reliability requirements of nuclear equipment for dimensional control.

[0038] It should be noted that the augmented reality assembly guidance module includes an image feature point matching submodule and a point cloud registration submodule. The image feature point matching submodule and the point cloud registration submodule are connected to a heat map generation unit. The outputs of the image feature point matching submodule and the point cloud registration submodule are combined to calculate the three-dimensional rigid body transformation matrix of the assembly relative to the standard assembly model through least squares registration. When the rotation amount of the three-dimensional rigid body transformation matrix is greater than 3° and the translation amount is greater than 2mm, the heat map generation unit is triggered to superimpose a red warning block in the AR display.

[0039] It should be specifically explained that, in the augmented reality assembly guidance module, the process of detecting the position and posture changes of the assembly parts in three-dimensional space when the operator performs the reassembly operation includes:

[0040] Step 1: Image acquisition and initial feature extraction: Collect the image of the reassembled component, apply the SHIF algorithm to extract the feature points of the reassembled component, and call the standard assembly model to establish the target area to be registered;

[0041] Step 2: Feature matching registration based on center point correction: The center point of the bolt on the gear in the detection target is calculated and corrected. If the overall component registration deviation exceeds the preset threshold within the set range, but the target center point alignment error is less than the set threshold, it is judged as surface misalignment and the transformation matrix is recalculated using the center point as the reference;

[0042] Step 3: Edge contour detection and line fitting: Extract the edge contour of the target component, use the Hough line fitting method to determine its boundary direction, and obtain the assembly rotation angle error by comparing the boundary direction with the standard direction.

[0043] Step 4: Point cloud-assisted spatial pose detection: Based on the point cloud data acquired by the scanning measurement system, the system automatically switches to 3D point cloud pose analysis after image registration. The region of interest is mapped to the point cloud space, the point cloud of the target area is intercepted, and RANSAC plane fitting is performed on the target assembly surface. The spatial distance, parallelism, and angle between the top and bottom surfaces are analyzed to determine whether there is any misalignment or incomplete insertion.

[0044] Step 5: Error Analysis and Type Judgment: Classify and judge assembly errors based on the image and point cloud information. Calculate translation errors using center point offset or point cloud center point difference. Calculate rotation errors using boundary line direction or fitted plane normal. Calculate vertical errors using point cloud surface distance to determine if the assembly is tight. Identify incomplete insertions when the edge contour difference area is greater than a set threshold or when an incomplete fit is not formed.

[0045] Step 6: Abnormal filtering and self-learning: For areas with abnormal center point distribution and abnormal contour disturbance, perform assembly error recognition optimization mechanism evaluation based on dynamic confidence weighting, and adaptively optimize the recognition algorithm.

[0046] For example, during regular maintenance at a nuclear power plant, the driving gear set in the charging pump gearbox needs to be disassembled and cleaned, and restored to its standard assembly state during the reinstallation process. The comparison diagram of the measured value of the driving gear assembly and the standard assembly 3D model is as follows: Figure 6 As shown, the driving gear assembly is precisely matched with the spindle shoulder and bearing inner ring. During reinstallation, the rotational error must not exceed ±3°, and the translation error must not exceed ±2mm. Failure to do so will result in poor meshing or premature wear. Maintenance personnel use head-mounted AR glasses for assembly guidance, and the system activates the augmented reality assembly guidance module to detect and correct assembly posture errors.

[0047] The operation flow and testing process are as follows:

[0048] Step 1: Image acquisition and initial feature extraction

[0049] (1) The operator aligns the active gear assembly position, and the system collects real-time images;

[0050] (2) Use SIFT algorithm to extract key feature points such as gear profile and bolt holes;

[0051] (3) Load the standard assembly model at the same time and extract the reference feature map of the corresponding parts in the model.

[0052] Step 2: Feature matching based on center point correction

[0053] (1) Preliminary registration showed an overall deviation of 4.3 mm;

[0054] (2) Further identify the alignment between the center point of the bolt hole on the driving gear and the model;

[0055] (3) If the center point error is less than 1 mm, the system will determine it as “surface misalignment” and automatically recalculate the rigid body transformation matrix based on the center point.

[0056] Step 3: Edge contour and angle extraction

[0057] (1) Extract the gear boundary direction through Canny edge detection + Hough line fitting;

[0058] (2) Determine that the angle between this direction and the standard direction is 4.7°;

[0059] (3) Determine whether the assembly rotation error exceeds the limit.

[0060] Step 4: Point cloud-assisted spatial posture detection

[0061] (1) Synchronously call the structured light scanner to obtain point cloud data of the target area;

[0062] (2) Map the gear area in the image to the point cloud space and perform RANSAC fitting;

[0063] (3) It was identified that there was a height difference of 1.3 mm between the assembly surface and the base surface of the shell, and the angle deviation was about 1.8°, which was judged to be an untightened assembly.

[0064] Step 5: Comprehensive error analysis

[0065] (1) Comprehensive judgment: the rotation error exceeds the limit (>3°), the compaction is incomplete, and there are areas where the boundaries are not aligned;

[0066] (2) Give the error type: "rotation direction deviation + assembly not tightened";

[0067] (3) Correction instructions are displayed: "Rotate clockwise about 3° and press down 1.5 mm."

[0068] Step 6: Abnormal area filtering and identification optimization

[0069] (1) At the same time, the edge area is detected to cause contour jitter due to reflection, which reduces the recognition confidence;

[0070] (2) Mark the area as a low-confidence image area and automatically reduce its recognition weight;

[0071] (3) Call historical successful assembly data to fine-tune and optimize the current recognition weight parameters.

[0072] Final result: The operator makes adjustments based on the heat map prompts; the recognized pose error falls below the set threshold (rotation angle 1.2°, translation error 0.9mm); the heat map automatically disappears, prompting "Assembly Passed"; the current operation record is stored in the digital twin assembly log for subsequent tracing and learning training.

[0073] It should be noted that in the data fusion and state modeling module, the digital twin model is based on the three-dimensional CAD model as the structural basis, and a parameter set that can update the status is set for each key component. The parameter set includes wear level, participating thickness, cumulative loading time and deviation history. The model update cycle is refreshed immediately after each measurement and identification task, and is used to determine whether to enter the maintenance window. The wear level includes 0 to 5 levels, the unit of participating thickness is mm, and the unit of cumulative loading time is h.

[0074] The lack of component state evolution records before and after disassembly and assembly of the charging pump gearbox necessitates manual judgment, which is highly subjective and unable to quantify the impact of wear, assembly deviation, or operating time on reliability. Maintenance decisions are often based on experience and lack scientific parameter support, making it difficult to implement trend prediction and timely maintenance strategies. Data from image recognition, optical contact probe measurement, and point cloud registration are scattered and lack structured fusion, resulting in a complete description of the life cycle of the same part. By using the 3D CAD model as the structural foundation, a digital twin structure is constructed with key components as nodes. Each node is configured with state parameters such as wear level (0-5), thickness (in millimeters), cumulative loading time (in hours), and historical deviation trajectory. The parameter set is automatically refreshed after each image recognition, probe measurement, or point cloud registration. This mechanism significantly improves the dynamic and interpretable nature of assembly state modeling, enabling maintenance personnel to understand the current status of the equipment in real time and conduct trend predictions, assisting in determining whether to enter the maintenance window. This supports the formulation of intelligent state-based maintenance strategies and achieves a technological leap from passive maintenance to proactive predictive operation and maintenance.

[0075] It should be noted that the parts visual tracking module uses image texture hash encoding to compare with the preset parts sequence dictionary, and verifies in real time during the reinstallation process whether the image ID of the current component matches its preset position. If there is misalignment, sequence jump, or direction flipping, the maintenance task is controlled to be paused and a prompt box of "abnormal assembly sequence" is popped up.

[0076] like Figures 2 to 5 As shown in the figure, the gearbox shows the step-by-step instructions for lifting the gearbox during the gearbox disassembly and overhaul process. Figures 2 to 5 There is a disassembly and maintenance step guidance task bar on the left side of the interface. Figure 2 A gearbox disintegration and explosion demonstration was shown. Figure 3 The gearbox base for gearbox hoisting is described with part numbers and precautions. The gearbox base part number is 56, and it contains 4 oil supply pipelines to avoid blockage. The data intelligence is displayed on the right. Figure 4 The gearbox cover is shown as part number 52, which contains the oil supply pipeline. Before reinstalling, check for foreign matter to avoid oil pipe blockage and bearing burnout. Figure 5 The disassembly and inspection steps for the gearbox are shown along with an animated demonstration. The disassembly process of the charging pump gearbox requires that multiple gaskets, gears, bearings, etc. be numbered and saved in a fixed sequence. During reinstallation, it is easy for components to be misordered or installed in the wrong direction, especially in symmetrical structures such as spiral directions and bevel angles. Manual paper records or alignment of drawings are labor-intensive, and once an error occurs, it is difficult to locate which link or component has a problem. Misinstalled bearings or gaskets can lead to poor meshing and abnormal preload, affecting transmission efficiency or even causing mechanical failure. The parts visual tracking module uses image texture hash encoding technology to create a unique visual identifier for each disassembled component, and compares and verifies the real-time image with the preset parts sequence dictionary during reinstallation. When it is identified that the current component position does not match its expected number, or if the assembly direction is reversed or the sequence is incorrectly installed, the current maintenance task process is immediately interrupted, and a prompt box indicating "Abnormal Assembly Sequence" pops up in the interactive interface. This mechanism effectively avoids low-level errors such as component misalignment and reverse assembly caused by operational negligence, greatly improving the accuracy, safety and traceability of the precision structure assembly process. It is particularly suitable for multi-level, highly sequence-dependent assembly scenarios of nuclear power key components such as charging pump gearboxes.

[0077] Example 2

[0078] The difference between Example 2 of the present invention and Example 1 is that this example introduces an augmented reality assembly guidance module in a virtual maintenance data processing system for optimizing maintenance scenarios.

[0079] It should be noted that in the augmented reality assembly guidance module, the assembly error recognition optimization mechanism based on dynamic confidence weighting generates a confidence score for each assembly error recognition result. The confidence score is obtained by weighted summation of the number of matching points, boundary integrity, point cloud surface closure rate, center point offset and image clarity. If the confidence score is lower than the set threshold, the control pauses the task push and prompts the operator to re-collect the image. The assembly error recognition optimization mechanism based on dynamic confidence weighting includes a regression learning model based on operation behavior correction feedback, which records and annotates the error correction between the operator's actual correction behavior and the recognition guidance results during the assembly process, and dynamically adjusts the parameter weights of image and point cloud recognition through a lightweight regression learning model to perform self-learning, self-optimization and confidence adaptive prompts of the assembly error recognition algorithm.

[0080] It should be specifically noted that the human-computer interaction and task incentive module includes a behavioral data backtracking and strategy optimization mechanism, which is used to record the operator's execution path, assembly adjustment behavior, error level changes, and confidence fluctuation process in each task. It generates a behavior log based on time series and performance labels, and adaptively optimizes and adjusts the recommended path, recognition parameter configuration, and incentive level weight of subsequent tasks. The human-computer interaction and task incentive module uses the task scoring formula to calculate the task score. The task scoring formula is:

[0081] Where: Scoring tasks is used to update the operator's incentive level based on the score, and to control their task unlocking permissions and challenge prompt push. Scoring of task benchmarks, which are pre-set by administrators and engineers for each type of task. is the measurement error rate, which is equal to the mean absolute error rate of the optical touch probe, ,in is the number of measurement points involved in the task, The index of the measurement point for this task, 、 The tasks are The measured and expected values of the measurement points, To identify the confidence coefficient, the Softmax layer classification probability output by the deep learning network from the image recognition module is used. ,in is the normalized value of the matching points and the total points, To identify the mean of the pairwise residuals, is the normalized value of the boundary closure rate, is the normalized value of the center point offset ratio, is the normalized value for measuring image clarity, 、 、 、 、 is the weight factor, , automatically lock the task submission button and pop up a prompt box "Please collect again".

[0082] For example, during the reinstallation of the charging pump gearbox, a maintenance task involves the reinstallation of the active gasket assembly. This gasket fits tightly against the shaft shoulder, and thickness variations significantly affect the meshing clearance. Therefore, the gasket assembly must not be eccentric, uncompacted, or reversed. Operators must also accurately identify assembly posture errors and correct them promptly. Maintenance personnel use head-mounted AR devices to assist in assembly, and the system simultaneously activates the "Augmented Reality Assembly Guidance Module" and the "Human-Computer Interaction and Task Incentive Module." The specific implementation process is detailed below:

[0083] Step 1: The system identifies the error and calculates the confidence score

[0084] The gasket outline is identified by matching image feature points;

[0085] Point cloud registration obtains the spatial position of the assembly surface;

[0086] The system analysis results are as follows:

[0087] The matching point ratio (FMP) is 0.91;

[0088] The residual mean error (RMSE) is 0.08;

[0089] The boundary closure rate (B_cl) is 0.85;

[0090] The center offset (ΔC) is 0.07;

[0091] The image clarity index (I) is 0.78;

[0092] The weight coefficient is preset to ~ =[0.25,0.25,0.2,0.15,0.15]

[0093] Calculate the confidence score: =0.25×0.91+0.25×(1-0.08)+0.2×0.85+0.15×(1-0.07)+0.15×0.78≈0.73;

[0094] If the system-set threshold (0.7) is not reached, the recognition result is judged to be of insufficient confidence, the task submission is locked, and a prompt "Please re-collect the image" is displayed.

[0095] Step 2: The operator fine-tunes the image and recognizes it again

[0096] The operator adjusts the assembly posture according to the red hot zone map prompts;

[0097] After the second image acquisition, the confidence score increased to 0.85;

[0098] The translation error of the identified assembly is 1.5 mm and the rotation error is 2.2°, which passes the identification threshold and the task continues.

[0099] Step 3: Behavior Backtracking and Recognition Weight Learning

[0100] The system records:

[0101] Initial identification failed;

[0102] The operator's actual fine-tuning direction is consistent with the hot zone guidance direction;

[0103] After adjustment, the confidence of assembly recognition is improved.

[0104] Mark this behavior as "Effective Guidance + Consistent Operation Correction":

[0105] The regression learning module updates the weight of the "boundary closure rate B_cl" in the current task scenario from 0.2 to 0.25;

[0106] Next time we encounter a similar task, we will pay more attention to contour integrity and implement dynamic learning and adaptive optimization.

[0107] Step 4: Task scoring and incentive feedback mechanism

[0108] This task involves the number of measurement points

[0109] =5, the difference between the actual measurement and the expected value of the optical contact probe is shown in Table 1:

[0110] Table 1 Differences between actual measurements and expected values of optical contact probes

[0111]

[0112] Calculate the measurement error rate, which is the mean absolute error rate of the optical touch probe, , task benchmark score , final task score =100 (1-0.014) 0.85≈83.8. The system updates the operator's points based on the score and pushes the achievement feedback "Rating of this task: excellent, high operating precision, and good assembly quality."

[0113] Through the dynamic confidence-weighted recognition mechanism and behavioral feedback self-learning model in the augmented reality assembly guidance module, the system can evaluate the credibility of recognition results in real time during the assembly error identification process, avoiding misjudgments caused by poor image quality or recognition bias. When the confidence level is insufficient, task submission is promptly terminated to ensure the accuracy of assembly data. At the same time, regression learning is performed based on the operator's actual error correction behavior, continuously optimizing the recognition parameter weights and achieving adaptive updates of the assembly recognition algorithm. Combined with the task scoring and behavior backtracking mechanisms in human-computer interaction, this not only improves recognition accuracy and assembly success rate, but also enhances operator initiative and system intelligence, significantly improving the assembly reliability and maintenance efficiency of high-precision components such as gearboxes.

[0114] Combining Example 1 and Example 2, the present invention effectively improves the intelligence level of assembly precision control, structural defect detection, maintenance data modeling and operation task management through the deep integration of multi-source data collection, intelligent identification, three-dimensional alignment, error judgment and guidance incentives throughout the entire process of gearbox disassembly and reassembly. In the assembly guidance process, image recognition and point cloud posture recognition algorithms are combined to dynamically obtain the spatial posture changes of components. By identifying translation errors, rotation errors and misfit states, hot zone prompts are accurately generated and corrected in real time to ensure high-precision assembly. At the same time, deep learning algorithms are used to achieve high-robust identification of tooth surface cracks, wear, and pitting defects, enhance the structural health assessment capability of the maintenance process, and through a scoring mechanism and behavioral data retrospective analysis, quantitative feedback and strategy optimization of operation quality are achieved, guiding maintenance personnel to actively complete high-quality operations, effectively avoiding human problems such as wrong installation and missing installation, and realizing the transformation of gearbox maintenance from experience-driven to data-driven, significantly improving maintenance efficiency, safety and intelligence.

[0115] The above description is merely a specific embodiment of the present application, but the scope of protection of the present application is not limited thereto. Any changes or substitutions that can be easily conceived by a person skilled in the art within the technical scope disclosed in this application should be included in the scope of protection of this application. Therefore, the scope of protection of this application should be based on the scope of protection of the claims.

[0116] Finally: The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the scope of protection of the present invention.

Claims

1. A virtual maintenance data processing system for optimizing maintenance scenarios, comprising an optical contact probe and a scanning measurement system, characterized in that: It also includes image recognition and defect analysis module, image-assisted measurement module, part visual tracking module, augmented reality assembly guidance module, data fusion and state modeling module, human-computer interaction and task incentive module, optical contact probe to measure data of key parts of gearbox, scanning measurement system to obtain three-dimensional point cloud data of key areas of gearbox, image recognition and defect analysis module to collect images of target area of gearbox, recognition algorithm based on deep learning to detect defects such as tooth surface wear, cracks, pitting and spalling, and jointly judge assembly posture deviation and poor contact, image-assisted measurement module to perform geometric analysis of calibration benchmarks and structural feature points in the image, and compare them with point cloud models and measurement results, part visual tracking module to record images and identifications of disassembled parts of gearbox, and manage the whole process tracking and assembly sequence The augmented reality assembly guidance module aligns the standard 3D assembly model with the current on-site image space, identifies assembly deviations and posture error heat maps in real time, and combines image recognition with point cloud data to jointly drive assembly correction guidance, prompting multi-source information enhanced posture adjustment. The data fusion and state modeling module fuses measurement results, image recognition output, and point cloud morphology to build a digital twin model of the gearbox assembly status. Through trend analysis, structural degradation judgment, and residual life assessment, it generates virtual maintenance suggestions. When the human-computer interaction and task incentive module identifies potential risk areas and key operation nodes, it actively generates measurement and assembly tasks, and combines task scores, behavior confidence, and incentive achievements to help guide the maintenance actions of the gearbox.

2. A virtual maintenance data processing system for optimizing maintenance scenarios according to claim 1, characterized in that: The image recognition and defect analysis module includes a recognition unit based on a deep convolutional neural network. The network structure of the recognition unit includes a feature extraction layer, a region proposal network and a classification regression branch. By introducing an attention channel, weighted learning is performed on the spatial area of the gear surface image to extract crack edge and texture degradation features in the presence of local occlusion and oil interference.

3. The virtual maintenance data processing system for optimizing maintenance scenarios according to claim 1, characterized in that: The image-assisted measurement module includes a two-dimensional image correction unit and a size conversion module, and is equipped with image and point cloud error comparison control logic. When the difference between the image measurement result and the point cloud measurement result is greater than 0.5mm, the image and point cloud error comparison control logic performs automatic image recalibration and size mapping coefficient adjustment operations until the measurement error is lower than the set tolerance range of 0.5mm. The scanning measurement system is a structured light scanner with a point pitch resolution of 0.05mm and a scanning range of 300mm*300mm. During the acquisition process, point cloud data from five perspectives is obtained through several angles. The point cloud data is used to generate a three-dimensional model of the gearbox surface after posture uniform alignment.

4. The virtual maintenance data processing system for optimizing maintenance scenarios according to claim 1, characterized in that: The augmented reality assembly guidance module includes an image feature point matching submodule and a point cloud registration submodule. The image feature point matching submodule and the point cloud registration submodule are connected to a heat map generation unit. The outputs of the image feature point matching submodule and the point cloud registration submodule are combined to calculate the three-dimensional rigid body transformation matrix of the assembly relative to the standard assembly model through least squares registration. When the rotation amount of the three-dimensional rigid body transformation matrix is greater than 3° and the translation amount is greater than 2mm, the heat map generation unit is triggered to superimpose a red warning block in the AR display.

5. A virtual maintenance data processing system for optimizing maintenance scenarios according to claim 4, characterized in that: In the augmented reality assembly guidance module, the process of detecting the position and posture changes of the assembly parts in three-dimensional space when the operator performs the reassembly operation includes: Step 1: Image acquisition and initial feature extraction: Collect the image of the reassembled component, apply the SHIF algorithm to extract the feature points of the reassembled component, and call the standard assembly model to establish the target area to be registered; Step 2: Feature matching registration based on center point correction: The center point of the bolt on the gear in the detection target is calculated and corrected. If the overall component registration deviation exceeds the preset threshold within the set range, but the target center point alignment error is less than the set threshold, it is judged as surface misalignment and the transformation matrix is recalculated using the center point as the reference; Step 3: Edge contour detection and line fitting: Extract the edge contour of the target component, use the Hough line fitting method to determine its boundary direction, and obtain the assembly rotation angle error by comparing the boundary direction with the standard direction. Step 4: Point cloud-assisted spatial pose detection: Based on the point cloud data acquired by the scanning measurement system, the system automatically switches to 3D point cloud pose analysis after image registration. The region of interest is mapped to the point cloud space, the point cloud of the target area is intercepted, and RANSAC plane fitting is performed on the target assembly surface. The spatial distance, parallelism, and angle between the top and bottom surfaces are analyzed to determine whether there is any misalignment or incomplete insertion. Step 5: Error Analysis and Type Judgment: Classify and judge assembly errors based on the image and point cloud information. Calculate translation errors using center point offset or point cloud center point difference. Calculate rotation errors using boundary line direction or fitted plane normal. Calculate vertical errors using point cloud surface distance to determine if the assembly is tight. Identify incomplete insertions when the edge contour difference area is greater than a set threshold or when an incomplete fit is not formed. Step 6: Abnormal filtering and self-learning: For areas with abnormal center point distribution and abnormal contour disturbance, perform assembly error recognition optimization mechanism evaluation based on dynamic confidence weighting, and adaptively optimize the recognition algorithm.

6. A virtual maintenance data processing system for optimizing maintenance scenarios according to claim 5, characterized in that: In the augmented reality assembly guidance module, the assembly error recognition optimization mechanism based on dynamic confidence weighting generates a confidence score for each assembly error recognition result. The confidence score is obtained by weighted summation of the number of matching points, boundary integrity, point cloud surface closure rate, center point offset and image clarity. If the confidence score is lower than the set threshold, the control pauses the task push and prompts the operator to re-collect the image. The assembly error recognition optimization mechanism based on dynamic confidence weighting includes a regression learning model based on operation behavior correction feedback, which records and annotates the error correction between the operator's actual correction behavior and the recognition guidance results during the assembly process, and dynamically adjusts the parameter weights of image and point cloud recognition through a lightweight regression learning model to perform self-learning, self-optimization and confidence adaptive prompts of the assembly error recognition algorithm.

7. A virtual maintenance data processing system for optimizing maintenance scenarios according to claim 6, characterized in that: The human-computer interaction and task incentive module includes a behavioral data backtracking and strategy optimization mechanism, which is used to record the operator's execution path, assembly adjustment behavior, error level changes, and confidence fluctuation process in each task. It generates a behavior log based on time series and performance labels, and adaptively optimizes and adjusts the recommended path, recognition parameter configuration, and incentive level weight for subsequent tasks. The human-computer interaction and task incentive module uses the task scoring formula to calculate the task score. The task scoring formula is: ； Where: Scoring tasks is used to update the operator's incentive level based on the score, and to control their task unlocking permissions and challenge prompt push. Scoring of task benchmarks, which are pre-set by administrators and engineers for each type of task. is the measurement error rate, which is equal to the mean absolute error rate of the optical touch probe, ,in is the number of measurement points involved in the task, The index of the measurement point for this task, 、 The tasks are The measured and expected values of the measurement points, To identify the confidence coefficient, the Softmax layer classification probability output by the deep learning network from the image recognition module is used. ,in is the normalized value of the matching points and the total points, To identify the mean of the pairwise residuals, is the normalized value of the boundary closure rate, is the normalized value of the center point offset ratio, is the normalized value for measuring image clarity, 、 、 、 、 are the weight factors of the normalized values of the number of matching points and the total number of points, the mean of the residual error of the recognition point pairs, the boundary closure rate, and the center point offset ratio, respectively. , automatically lock the task submission button and pop up a "Please collect again" prompt box.

8. The virtual maintenance data processing system for optimizing maintenance scenarios according to claim 1, characterized in that: In the data fusion and state modeling module, the digital twin model uses the three-dimensional CAD model as its structural basis, and sets a parameter set for each key component that can update the status. The parameter set includes wear level, participating thickness, cumulative loading time and deviation history. The model update cycle is refreshed immediately after each measurement and identification task, and is used to determine whether to enter the maintenance window. The wear level includes 0 to 5 levels, the unit of participating thickness is mm, and the unit of cumulative loading time is h.

9. The virtual maintenance data processing system for optimizing maintenance scenarios according to claim 1, characterized in that: The parts visual tracking module uses image texture hash encoding to compare with a preset part sequence dictionary. During the reassembly process, it verifies in real time whether the image ID of the current component matches its preset position. If there is any misalignment, sequence jump, or direction flip, the maintenance task will be suspended and a prompt box "Assembly sequence abnormality" will be popped up.

Images