Method for automatically identifying assembly quality of prefabricated station components based on machine vision

Abstract

The application relates to the technical field of artificial intelligence, and discloses an assembly type station component assembly quality automatic identification method based on machine vision. The method comprises the following steps: collecting a video stream through an industrial camera array; dynamically selecting a key frame according to a motion change rate and structural saliency; obtaining a component multi-scale space feature by using a light-weight multi-scale feature extraction network; performing projection matching in combination with a three-dimensional geometric constraint model constructed based on a BIM model to calculate a pose deviation; and performing compliance determination according to an assembly process rule library, and generating a structured warning information when the component is unqualified. Through the above technical means, the processing speed is improved while the recognition accuracy is ensured, and the field construction rhythm requirement is met.

Description

TECHNICAL FIELD

[0001] The application belongs to the field of artificial intelligence, and particularly relates to a method for automatically identifying assembly quality of a fabricated station component based on machine vision. BACKGROUND

[0002] With the rapid advancement of intelligent construction technology in the field of rail transit infrastructure, the fabricated station has become an important mode of modern urban underground space development due to its short construction period, small environmental impact and high industrialization level. In the process of fabricated construction, the assembly quality of prefabricated components is directly related to the structural safety and service life. The traditional quality inspection method mainly relies on manual visual inspection or contact measurement, which has strong subjectivity, low efficiency and incomplete coverage.

[0003] In recent years, automatic identification technology based on machine vision has been introduced into this scene, which realizes quantitative evaluation of key indicators such as component pose deviation, joint misalignment and surface defects through image acquisition and algorithm analysis, and improves the objectivity and traceability of detection.

[0004] The visual recognition method based on deep learning has become the mainstream technical path, which usually uses convolutional neural networks to extract features and make quality judgments on single or multiple frames of images. However, in actual assembly operations, the components move slowly and the background environment is relatively static, resulting in a high degree of redundant information between consecutive video frames. Existing systems generally perform a complete forward inference process on each frame of image, including repeated low-level feature extraction and background region processing, resulting in a waste of a large amount of computing resources.

[0005] Although some schemes try to introduce lightweight networks such as MobileNet series to reduce the single inference overhead, they do not fundamentally solve the problem of temporal redundancy, and the overall processing delay is still difficult to meet the real-time requirements of the "assembly, feedback and immediate correction" in the construction process.

[0006] The existing technology lacks effective use of spatiotemporal correlation at the algorithm architecture level, neither designs a cross-frame reuse mechanism that shares features, nor establishes a dynamic trigger calculation skipping strategy, resulting in continuous full calculation of the system in static or slowly changing scenes.

[0007] The high-dimensional feature channels and redundant convolutional layers reserved for precision further exacerbate the inference burden. In the industrial field such as the fabricated station which is sensitive to response speed, such methods are often forced to reduce the sampling frequency or simplify the model due to the high time consumption of single-frame processing, sacrificing detection accuracy or coverage density, ultimately restricting the simultaneous improvement of construction efficiency and quality control level.

[0008] Therefore, there is an urgent need for a component assembly quality automatic identification method that can deeply integrate temporal information, eliminate redundant calculations and compress inference delay. SUMMARY

[0009] The application provides a machine vision-based prefabricated station component assembly quality automatic identification method, aiming to solve the technical problem of high time consumption of existing systems in processing single-frame images, inability to meet real-time feedback requirements in the assembly process, and resulting in reduced construction efficiency. The method combines a lightweight multi-scale feature extraction network with a dynamic key frame selection mechanism to form a real-time visual analysis architecture, and combines component geometric constraint priors and an assembly process rule library to achieve millisecond-level identification and determination of key quality indicators such as component pose deviation, joint misalignment, and embedded part alignment state, thereby improving processing speed while ensuring identification accuracy to meet the requirements of the construction rhythm.

[0010] The application provides a machine vision-based prefabricated station component assembly quality automatic identification method, which includes:

[0011] An industrial camera array deployed at the construction site continuously acquires video stream data of the area to be assembled at a fixed frame rate;

[0012] From the video stream data, key frames are dynamically selected according to motion change rate and structural saliency criteria to form a sparse but information-complete key frame sequence;

[0013] The key frame sequence is input into a pre-trained lightweight multi-scale feature extraction network, which uses a backbone structure combining depth separable convolution and channel attention mechanism to extract multi-scale spatial features of component edge contours, corner distributions, and surface textures layer by layer;

[0014] A three-dimensional geometric constraint model is established based on component design drawings, which includes component shape size tolerance band, connection hole coordinate tolerance range, and assembly surface flatness threshold;

[0015] The multi-scale spatial features are projected and matched with the three-dimensional geometric constraint model to calculate the pose deviation between the actual assembly state and the theoretical design state;

[0016] The pose deviation is determined for compliance according to a pre-set assembly process rule library, which defines the allowed misalignment amount, allowed twist angle, and upper limit of embedded part center offset for different component types;

[0017] When the determination result is unqualified, structured quality warning information containing deviation type, value, and spatial position is generated and pushed to the on-site construction terminal.

[0018] As an embodiment of the present application, the industrial camera array is composed of more than three global shutter type industrial cameras, each camera is installed at a height of 4-6 meters, the field of view angle covers the entire component assembly operation area, the frame rate is set to 30 Hz, and the synchronous trigger signal is uniformly issued by the central controller to ensure the time consistency of the multi-view images.

[0019] As an embodiment of the present application, the dynamic selection of key frames specifically includes:

[0020] The root mean square of the optical flow amplitude between the current frame and the previous key frame is calculated, and if the value is greater than the preset motion threshold, it is determined that the current frame has component displacement;

[0021] The Laplacian operator response variance of the current frame is calculated at the same time, and if the variance is greater than the preset structural saliency threshold, it is determined that the current frame contains clear component edge information; only when the above two conditions are met at the same time, the current frame is marked as a key frame and added to the key frame sequence.

[0022] As an embodiment of the present application, the lightweight multi-scale feature extraction network contains four down-sampling stages, each stage is composed of 3 deep separable convolution layers in series, the convolution kernel size is 3x3, 5x5, 7x7 in turn, and the output channel number is 32, 64, 128, 256 respectively;

[0023] Coordinate attention modules are connected at the output ends of the third and fourth stages of the network, which respectively generate position-aware weights along the horizontal and vertical directions to enhance the feature response strength of the key parts of the component.

[0024] As an embodiment of the present application, the three-dimensional geometric constraint model is generated by analyzing the component BIM model, which discretizes the component outer contour into a point cloud set with a spacing of less than 10 mm, and assigns each point with a normal vector and curvature information; the connecting hole is modeled as a cylindrical entity with its axis direction consistent with the design drawing, and the radius is expanded by 5% as the tolerance radius; the assembly surface is represented by a plane equation, and the flatness tolerance is set to ±0.5 mm.

[0025] As an embodiment of the present application, the projection matching specifically includes:

[0026] The three-dimensional geometric constraint model is projected onto the key frame image plane according to the current estimated pose to generate a theoretical contour mask; the actual component edge line segments are detected in the key frame using Hough transform;

[0027] The Hausdorff distance between the actual edge line segments and the theoretical contour mask is calculated, and if the distance is greater than the allowed misalignment amount of the corresponding component type, it is recorded as an out-of-tolerance area;

[0028] Sub-pixel level circle center positioning is performed on the connection hole area, the Euclidean distance between the measured circle center and the theoretical projection circle center is calculated, and if the distance is greater than the upper limit of the embedded part center deviation, it is marked as alignment failure.

[0029] As an embodiment of the application, the assembly process rule library is stored in a local embedded database, indexed by component codes, and each rule record contains component category, applicable working condition, allowed misalignment amount, allowed torsion angle, upper limit of embedded part center deviation, and judgment priority.

[0030] The compliance judgment sequentially performs each rule check according to the judgment priority, and any check item exceeding the limit terminates the subsequent judgment and outputs an unqualified conclusion.

[0031] As an embodiment of the application, the structured quality alarm information is packaged in JSON format, including timestamp, component unique identifier, deviation type code, deviation value, spatial coordinates, and adjustment direction.

[0032] After receiving the construction terminal, the over-limit part is highlighted on the three-dimensional construction model, and the adjustment instruction is voice broadcasted.

[0033] The application also provides a machine vision-based prefabricated station component assembly quality automatic identification system, which comprises:

[0034] A video stream acquisition unit is used to continuously acquire video stream data of the area to be assembled at a fixed frame rate through an industrial camera array deployed at the construction site.

[0035] A key frame dynamic selection unit is used to dynamically select key frames from the video stream data according to motion change rate and structural saliency dual criteria, forming a sparse but information complete key frame sequence.

[0036] A multi-scale feature extraction unit is used to input the key frame sequence into a pre-trained lightweight multi-scale feature extraction network to extract multi-scale spatial features of component edge contours, corner point distribution and surface texture.

[0037] A three-dimensional geometric constraint modeling unit is used to establish a three-dimensional geometric constraint model containing shape and size tolerance bands, connection hole coordinate tolerance ranges and assembly surface flatness thresholds based on component design drawings.

[0038] A pose deviation calculation unit is used to project and match the multi-scale spatial features with the three-dimensional geometric constraint model to calculate the pose deviation amount between the actual assembly state and the theoretical design state.

[0039] A compliance judgment unit is used to perform compliance judgment on the pose deviation amount according to a pre-set assembly process rule library.

[0040] A quality alarm generating unit is configured to generate structured quality alarm information including a deviation type, a value and a spatial position when the determination result is unqualified, and push the information to a field construction terminal.

[0041] As an embodiment of the present application, the industrial camera in the video stream acquisition unit is a global shutter CMOS sensor with a resolution of 2480*2048 pixels and a lens focal length of 16 mm, equipped with an infrared light supplement lamp set with a working wavelength of 850 nm, ensuring that clear images can still be obtained in low-illumination environments.

[0042] As an embodiment of the present application, the multi-scale feature extraction unit runs on an edge computing device, which is equipped with a special neural network inference chip and supports INT8 quantization operation, with a network inference delay of less than 25 ms.

[0043] As an embodiment of the present application, the pose deviation calculation unit uses the iterative closest point algorithm to optimize the initial pose estimation, with an upper limit of 20 iterations and a convergence threshold of zero point zero one millimeter, ensuring that the pose solution accuracy meets the engineering acceptance standard.

[0044] Compared with the prior art, the present application has the following advantages:

[0045] 1. The dynamic key frame selection mechanism is introduced to avoid invalid processing of redundant still frames in the video stream, reducing the number of images to be analyzed; the lightweight multi-scale feature extraction network uses depth separable convolution and coordinate attention mechanism to compress the calculation amount while maintaining the feature expression ability;

[0046] 2. The deep fusion of the three-dimensional geometric constraint model and the assembly process rule library makes the quality determination directly face the engineering acceptance index, without relying on a large number of labeled samples for end-to-end training;

[0047] 3. The overall system realizes single quality identification full-flow time consumption under typical working conditions, meets the assembly rhythm requirements of the assembled station, and improves the construction efficiency and quality control level. BRIEF DESCRIPTION OF DRAWINGS

[0048] Figure 1 is the overall technical scheme architecture schematic diagram of the machine vision-based assembled station component assembly quality automatic identification method proposed by the present application;

[0049] Figure 2 is the core principle framework schematic diagram of the fusion of dynamic key frame selection and lightweight multi-scale feature extraction in the present application;

[0050] Figure 3 is the video stream acquisition and key frame dynamic selection logic flow framework diagram in the present application;

[0051] Figure 4 is the logical flow framework diagram of multi-scale spatial feature extraction and three-dimensional geometric constraint model projection matching in the application;

[0052] Figure 5 is the logical flow framework diagram of compliance determination and quality alarm generation based on the assembly process rule library in the application;

[0053] Figure 6 is the multi-level interaction relationship and data flow diagram of the industrial camera array, the edge computing device and the construction terminal in the application. DETAILED DESCRIPTION

[0054] Please refer to Figures 1 to 6 The application provides an assembly station component assembly quality automatic identification method based on machine vision, aiming to solve the technical problems of high time consumption of existing systems in processing single frame images, inability to meet real-time feedback requirements in the assembly process, and resulting in low construction efficiency.

[0055] The method continuously collects video stream data by deploying an industrial camera array, and introduces a real-time visual analysis architecture combining a dynamic key frame selection mechanism and a lightweight multi-scale feature extraction network, combines component geometric constraint priors and assembly process rule libraries, and realizes millisecond-level identification and determination of key quality indicators such as component pose deviation, joint misalignment, and embedded part alignment state.

[0056] The method comprises the following steps:

[0057] S1, the industrial camera array deployed at the construction site continuously collects video stream data of the area to be assembled at a fixed frame rate;

[0058] S2, dynamically select key frames from the video stream data according to the motion change rate and structural saliency dual criteria to form a sparse but complete key frame sequence;

[0059] S3, input the key frame sequence into a pre-trained lightweight multi-scale feature extraction network, which adopts a backbone structure combining depth separable convolution and channel attention mechanism to extract multi-scale spatial features of component edge contours, corner distributions and surface textures layer by layer;

[0060] S4, establish a three-dimensional geometric constraint model based on component design drawings, which includes component shape size tolerance band, connection hole coordinate tolerance range and assembly surface flatness threshold;

[0061] S5, project match the multi-scale spatial features and the three-dimensional geometric constraint model to calculate the pose deviation between the actual assembly state and the theoretical design state;

[0062] S6, judging the pose deviation according to a preset assembly process rule base, the assembly process rule base defining an allowed misalignment, an allowed torsion angle and a pre-embedded part center offset upper limit corresponding to different component types;

[0063] S7, when the judgment result is unqualified, generating structured quality warning information containing the deviation type, value and spatial position, and pushing to the on-site construction terminal.

[0064] In step S1, the industrial camera array deployed at the construction site continuously collects video stream data of the area to be assembled at a fixed frame rate. The industrial camera array is composed of more than 3 global shutter type industrial cameras, each camera is installed at a height of 4-6 meters, the field of view angle covers the entire component assembly operation area, and the frame rate is set to 30 Hz.

[0065] All cameras are uniformly triggered by a central controller to ensure the time consistency of multi-view images. Each industrial camera uses a global shutter CMOS sensor with a resolution of 2480x2048 pixels, a lens focal length of 16 mm, and is equipped with an infrared light supplement lamp set with a working wavelength of 850 nm to ensure that clear and motion-free image data can be obtained in low light or night construction environment.

[0066] The video stream data is transmitted in real time to the edge computing device through gigabit Ethernet, and the transmission protocol uses a low-delay streaming media packaging format based on UDP to ensure that the image frame is transmitted within 10 milliseconds after being collected.

[0067] In step S2, key frames are dynamically selected from the video stream data according to the motion change rate and structural saliency double criteria to form a sparse but information complete key frame sequence.

[0068] Specifically, the system maintains a cache of the previous key frame, and initially marks the first frame of the video stream as the first key frame.

[0069] For each subsequent frame, first calculate the root mean square of the optical flow amplitude between the current frame and the previous key frame.

[0070] The optical flow calculation uses the Lucas-Kanade algorithm based on pyramid layering, with 4 layers of pyramid levels and a downsampling factor of 2 for each layer. The root mean square of the optical flow amplitude is defined as the square root of the average of the square of the modulus of the optical flow vector of all pixel points.

[0071] If the value is greater than a preset motion threshold, it is determined that the current frame has component displacement. The motion threshold is set according to the typical moving speed of the component, with a value range of 0.5-1.5 pixels per pixel per frame.

[0072] At the same time, the Laplacian operator response variance of the current frame is calculated.

[0073] The Laplacian operator adopts a 3x3 kernel, and its response value reflects the local second-order derivative intensity of the image, i.e., the edge sharpness.

[0074] The response variance is defined as the square of the standard deviation of the Laplacian response value of the entire image.

[0075] If the variance is greater than a preset structural saliency threshold, it is determined that the current frame contains clear component edge information.

[0076] The structural saliency threshold is set according to the field illumination and the component surface material, and a typical value is 100-300.

[0077] Only when the above two conditions are met at the same time, the current frame is marked as a key frame and added to the key frame sequence, and the previous key frame buffer is updated. This mechanism effectively filters out redundant frames generated by hoisting pauses, slight vibrations or environmental light disturbances, reduces the density of the key frame sequence to less than 30% of the original video stream, and greatly reduces the subsequent processing load.

[0078] In step S3, the key frame sequence is input into a pre-trained lightweight multi-scale feature extraction network, which adopts a backbone structure combining deep separable convolution and channel attention mechanism to extract multi-scale spatial features of component edge contours, corner point distribution and surface texture layer by layer.

[0079] The lightweight multi-scale feature extraction network includes 4 down-sampling stages, each of which is composed of 3 deep separable convolution layers in series.

[0080] The input resolution of the first stage is 1224x1024, and the output channel number is 32;

[0081] The input resolution of the second stage is 612x512, and the output channel number is 64;

[0082] The input resolution of the third stage is 306x256, and the output channel number is 128;

[0083] The input resolution of the fourth stage is 153x128, and the output channel number is 256.

[0084] The convolution kernel size of the three deep separable convolution layers in each stage is 3x3, 5x5, and 7x7, respectively, and the step size is 1.

[0085] Each convolution layer is followed by batch normalization and ReLU activation function.

[0086] The coordinate attention module is connected to the output end of the third and fourth stages of the network.

[0087] The coordinate attention module generates position-aware weights along the horizontal and vertical directions. The specific implementation is as follows:

[0088] The input feature map is globally averaged along the height direction to obtain a one-dimensional description vector in the width direction, and then a one-dimensional convolution and a Sigmoid activation are used to generate a horizontal attention weight;

[0089] Similarly, the global average pooling is performed along the width direction to obtain a one-dimensional description vector in the height direction, and a vertical attention weight is generated.

[0090] Finally, the weights in the two directions are multiplied and element-wise multiplied with the original feature map to obtain an enhanced feature map.

[0091] This design improves the response strength of the network to key geometric positions such as component corner points and edge intersection points.

[0092] The entire network is optimized using a synthetic dataset and real labeled data during the training phase, and the loss function includes the weighted sum of edge positioning loss, corner point detection loss, and classification cross-entropy loss.

[0093] The network is deployed on an edge computing device equipped with a special neural network inference chip, supports INT8 quantization operation, and the network inference delay is less than 25 milliseconds.

[0094] In step S4, a three-dimensional geometric constraint model is established based on the component design drawing, which includes component shape size tolerance band, connection hole coordinate tolerance range, and assembly surface flatness threshold. The three-dimensional geometric constraint model is generated by analyzing the component BIM model.

[0095] First, the component outer contour polygon mesh is extracted from the BIM model and discretized into a point cloud set with a spacing of less than 10 mm. Each point is assigned a normal vector and curvature information, the normal vector is calculated by weighted average of adjacent facets, and the curvature is solved by fitting a quadratic surface to the local neighborhood points.

[0096] Secondly, the connection hole is modeled as a cylindrical solid, with its axis direction consistent with the design drawing and its radius expanded by 5% of the design value as the tolerance radius. For example, if the design hole diameter is fifty millimeters, the modeling radius is 26.25 millimeters.

[0097] Finally, the assembly surface is represented by a plane equation, and the flatness tolerance is set to ±0.5 mm. The plane equation is obtained by least squares fitting of the assembly surface vertices in the BIM model.

[0098] All geometric elements and their tolerance parameters are stored in a structured data format in a local database, indexed by the unique identifier of the component, for subsequent projection matching calls.

[0099] In step S5, the multi-scale spatial features are projected and matched with the three-dimensional geometric constraint model to calculate the pose deviation between the actual assembly state and the theoretical design state.

[0100] The initial pose estimation is solved by PnP algorithm based on the detected corner points and the corresponding points of BIM model. Then the initial pose estimation is optimized by iterative closest point algorithm, the upper limit of iteration times is 20, and the convergence threshold is set to 0.01 millimeters.

[0101] In each iteration, the three-dimensional geometric constraint model is projected onto the key frame image plane according to the current estimated pose to generate a theoretical contour mask.

[0102] The actual component edge line segments in the key frame are detected by using the Hough transform, and the Hough transform parameter space resolution is set to an angle of 1 degree and a distance of 1 pixel. The Hausdorff distance between the actual edge line segments and the theoretical contour mask is calculated.

[0103] The Hausdorff distance is defined as the maximum minimum distance between two sets, which is used to measure the overall deviation of the contour.

[0104] If the distance is greater than the allowed misalignment amount corresponding to the component type, it is recorded as an out-of-tolerance area, and its centroid coordinates and maximum deviation value are stored.

[0105] For the connecting hole region, first locate the hole region ROI in the image, and then perform sub-pixel level center positioning.

[0106] The center positioning uses a gradient-based ellipse fitting algorithm with a fitting accuracy of 0.1 pixels.

[0107] The Euclidean distance between the measured center and the theoretical projected center is calculated, and if the distance is greater than the upper limit of the embedded part center offset, it is marked as a failure of alignment and the offset vector is recorded.

[0108] The pose deviation is finally represented as a six-degree-of-freedom vector, including 3 translation components and 3 rotation components, with units of millimeters and degrees, respectively.

[0109] In step S6, the pose deviation is determined for compliance according to a pre-set assembly process rule library, which defines the allowed misalignment amount, the allowed torsion angle, and the upper limit of the embedded part center offset corresponding to different component types.

[0110] The assembly process rule library is stored in a local embedded database and indexed by component code.

[0111] Each rule record contains component category, applicable working condition, allowed misalignment amount, allowed torsion angle, upper limit of embedded part center offset, and judgment priority.

[0112] The smaller the judgment priority value, the earlier the check order. The compliance determination performs each rule check in order of judgment priority.

[0113] Firstly, the alignment state of embedded parts is checked. If the center of any embedded part deviates more than the upper limit, the subsequent determination is terminated immediately and an unqualified conclusion is output.

[0114] Secondly, the amount of joint misalignment is checked. If the Hausdorff distance exceeds the limit, an unqualified output is output.

[0115] Finally, the overall torsion angle is checked. If the rotation angle around the vertical axis is greater than the allowed value, an unqualified output is output.

[0116] All determination thresholds come from the current railway engineering acceptance specification, for example, the allowed misalignment of the side wall plate is 2mm, and the allowed torsion angle of the top plate is 0.5 degrees.

[0117] In step S7, when the determination result is unqualified, structured quality warning information containing the deviation type, value and spatial position is generated and pushed to the field construction terminal.

[0118] The structured quality warning information is packaged in JSON format, including timestamp, component unique identifier, deviation type code, deviation value, spatial coordinates and adjustment direction.

[0119] The deviation type code uses three-digit coding, for example, 001 represents embedded part deviation, 002 represents joint misalignment, and 003 represents overall torsion.

[0120] The spatial coordinates are represented in the global coordinate system of the construction site, with units of millimeters.

[0121] The adjustment direction is obtained by back-propagating the deviation vector, for example, "move left 15mm" or "rotate clockwise 0.3 degrees".

[0122] After receiving the warning information, the construction terminal highlights the out-of-tolerance part on the three-dimensional construction model and broadcasts the adjustment instructions through the TTS engine. The broadcast content includes component name, deviation type and adjustment action.

[0123] The warning information is also stored in the quality traceability database for later statistical analysis.

[0124] The prefabricated station component assembly quality automatic identification system based on machine vision includes a video stream acquisition unit, a key frame dynamic selection unit, a multi-scale feature extraction unit, a three-dimensional geometric constraint modeling unit, a pose deviation calculation unit, a compliance determination unit and a quality warning generation unit.

[0125] The video stream acquisition unit is used to continuously acquire video stream data of the area to be assembled by deploying an industrial camera array at the construction site at a fixed frame rate.

[0126] The unit contains more than 3 global shutter CMOS industrial cameras, each with a resolution of 2480x2048 pixels and a lens focal length of 16mm, equipped with a set of 850nm infrared fill light.

[0127] All cameras are synchronized by a central controller through a hardware trigger line, with a fixed frame rate of 30Hz. Image data is transmitted to the edge computing device via Gigabit Ethernet, with a transmission delay controlled within 10ms.

[0128] The key frame dynamic selection unit is used to dynamically select key frames from the video stream data based on motion change rate and structural saliency criteria, forming a sparse but complete key frame sequence.

[0129] The unit contains an optical flow calculation module and a structural saliency analysis module.

[0130] The optical flow calculation module uses a four-layer pyramid Lucas-Kanade algorithm to output the optical flow field of each frame relative to the previous key frame.

[0131] The structural saliency analysis module applies a 3x3 Laplacian operator to each frame of image and calculates the response variance.

[0132] The logical judgment module compares the root mean square of optical flow amplitude with the motion threshold and the Laplacian variance with the structural saliency threshold, and only when both are exceeded will the current frame be added to the key frame sequence.

[0133] The multi-scale feature extraction unit is used to input the key frame sequence into a pre-trained lightweight multi-scale feature extraction network to extract multi-scale spatial features of the component edge profile, corner distribution and surface texture.

[0134] The unit runs on an edge computing device equipped with a dedicated neural network inference chip supporting INT8 quantization.

[0135] The network backbone contains 4 down-sampling stages, each consisting of 3 deep separable convolution layers with convolution kernel sizes of 3x3, 5x5 and 7x7.

[0136] The four-stage output end integrates a coordinate attention module, generating position-aware weights along the horizontal and vertical directions to enhance the feature response of key geometric parts.

[0137] The three-dimensional geometric constraint modeling unit is used to establish a three-dimensional geometric constraint model based on the component design drawings, including the shape size tolerance band, the connection hole coordinate tolerance range and the assembly surface flatness threshold.

[0138] The unit parses the BIM model, discretizes the outer contour into a point cloud with a spacing of less than 10 mm, assigns a normal vector and a curvature to each point, connects the hole positions to model a cylinder with a radius of 5% expansion, and assembles the surfaces to represent a plane equation with a flatness tolerance of ±0.5 mm.

[0139] All geometric elements and their tolerance parameters are stored in a local database according to the unique identifier of the component.

[0140] The pose deviation calculation unit is used to project match the multi-scale spatial features with the three-dimensional geometric constraint model, and calculate the pose deviation between the actual assembly state and the theoretical design state.

[0141] The unit first obtains the initial pose through the PnP algorithm, and then optimizes it using the iterative closest point algorithm, with an upper limit of 20 iterations and a convergence threshold of 0.01 mm.

[0142] During the projection matching process, the actual edge is detected using the Hough transform, and the Hausdorff distance from the theoretical contour is calculated; sub-pixel center positioning is performed on the hole area, and the Euclidean distance between the measured and theoretical center is calculated.

[0143] The compliance determination unit is used to determine the compliance of the pose deviation according to the preset assembly process rule library. The unit accesses the assembly process rule library in the local embedded database, and the rules are indexed by component code, including the allowed misalignment amount, the allowed twist angle, the upper limit of the embedded part center offset, and the determination priority. The determination is performed in order of priority, and if any check item exceeds the limit, the process is terminated and an unqualified conclusion is output.

[0144] The quality warning generation unit is used to generate structured quality warning information containing the deviation type, value and spatial position when the determination result is unqualified, and push it to the on-site construction terminal. The unit encapsulates the warning information in JSON format, including timestamp, component unique identifier, deviation type code, deviation value, spatial coordinates and adjustment direction. The information is pushed to the construction terminal through a wireless local area network, and the terminal highlights the over-limit part on the three-dimensional model and plays the adjustment instruction.

[0145] It should be noted that, in this text, relational terms such as first and second are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "contain" or any other variant thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements not only includes those elements, but also includes other elements not explicitly listed or inherent to such a process, method, article or device.

[0146] While embodiments of the application have been shown and described, it is to be understood that the embodiments described are merely exemplary of the principles and application of the present application. Numerous modifications and adaptions can be effected without departing from the spirit and scope of the present application, which is not limited to the exact construction and arrangement described. It is intended, therefore, to cover all modifications and adaptions that fall within the scope of the claims and their equivalents.

Claims

1. A machine vision-based automatic identification method for the assembly quality of prefabricated railway station components, characterized in that, include: An industrial camera array deployed at the construction site continuously acquires video stream data of the area to be assembled at a fixed frame rate; Keyframes are dynamically selected from the video stream data based on both motion change rate and structural saliency criteria to form a sparse but information-complete keyframe sequence. The keyframe sequence is input into a pre-trained lightweight multi-scale feature extraction network. This network adopts a backbone structure that combines depthwise separable convolution with channel attention mechanism to extract multi-scale spatial features of component edge contours, corner distribution and surface texture layer by layer. A three-dimensional geometric constraint model is established based on the component design drawings. The three-dimensional geometric constraint model includes the component's external dimension tolerance zone, the coordinate tolerance range of the connecting hole position, and the flatness threshold of the assembly surface. The multi-scale spatial features are projected and matched with the three-dimensional geometric constraint model to calculate the pose deviation between the actual assembly state and the theoretical design state. The compliance of the positional deviation is determined according to the preset assembly process rule library. The assembly process rule library defines the allowable misalignment, allowable torsion angle and upper limit of the center offset of the embedded part for different component types. When the judgment result is unqualified, a structured quality alarm message containing the deviation type, value and spatial location is generated and pushed to the on-site construction terminal. The synchronization trigger signal of the industrial camera array is uniformly issued by the central controller to achieve time consistency of multi-view images. The dynamic selection of keyframes from the video stream data based on both motion change rate and structural saliency criteria includes: Calculate the root mean square of the optical flow amplitude between the current frame and the previous keyframe. If the value is greater than the preset motion threshold, it is determined that there is component displacement in the current frame. Simultaneously, the variance of the Laplacian operator response of the current frame is calculated. If the variance is greater than the preset structural saliency threshold, the current frame is determined to contain clear component edge information. The current frame is marked as a keyframe and added to the keyframe sequence only when both of the above conditions are met. The lightweight multi-scale feature extraction network contains four downsampling stages, each consisting of three depthwise separable convolutional layers connected in series. A coordinate attention module is connected to the output of the third and fourth stages of the lightweight multi-scale feature extraction network. The coordinate attention module generates position-aware weights along the horizontal and vertical directions to enhance the feature response intensity of key parts of the component. The three-dimensional geometric constraint model is generated by analyzing the component BIM model, discretizing the outer contour of the component into a set of point clouds, and assigning normal vector and curvature information to each point; the connecting holes are modeled in the form of cylindrical solids, with their axial direction consistent with the design drawings; the assembly surface is represented by a plane equation.

2. The automatic identification method for assembly quality of prefabricated station components based on machine vision according to claim 1, characterized in that, The step of projecting and matching the multi-scale spatial features with the three-dimensional geometric constraint model includes: The three-dimensional geometric constraint model is projected onto the keyframe image plane according to the current estimated pose to generate a theoretical contour mask; Detecting actual component edge segments in keyframes using Hough transform; Calculate the Hausdorff distance between the actual edge line segment and the theoretical contour mask. If the distance is greater than the allowable misalignment for the corresponding component type, it is recorded as an out-of-tolerance area. Perform subpixel-level center positioning on the connecting hole area, calculate the Euclidean distance between the measured center and the theoretical projected center, and mark the alignment failure if the distance is greater than the upper limit of the center offset of the embedded part.

3. The method for automatic identification of assembly quality of prefabricated station components based on machine vision according to claim 2, characterized in that, The current estimated pose is obtained by using the PnP algorithm to obtain the initial pose, and then optimized by the iterative nearest point algorithm.

4. The automatic identification method for assembly quality of prefabricated station components based on machine vision according to claim 3, characterized in that, The assembly process rule base is stored in a local embedded database and indexed by component code. Each rule record includes component category, applicable working conditions, allowable misalignment, allowable torsion angle, upper limit of center offset of embedded part, and judgment priority. The compliance judgment is performed sequentially according to the judgment priority. If any check item exceeds the limit, the subsequent judgment is terminated and a non-compliance conclusion is output.

5. The automatic identification method for assembly quality of prefabricated station components based on machine vision according to claim 4, characterized in that, The structured quality alarm information is encapsulated in JSON format, including timestamp, unique component identifier, deviation type code, deviation value, spatial coordinates, and suggested adjustment direction. After receiving the information, the construction terminal highlights the out-of-tolerance parts on the three-dimensional construction model and broadcasts the adjustment instructions via voice.

Images