Visual positioning method for alignment assembly of display screen module

Abstract

The invention discloses a visual positioning method for alignment assembly of a display screen module, and particularly relates to the technical field of automatic assembly. Acquiring multi-source image data including an orthographic vision image, an oblique angle projection image and a near-infrared image; carrying out image enhancement and feature fusion processing, and constructing a multi-dimensional joint visual feature matrix of structure edges, illumination reflection textures, material mark points and the like; identifying module key identification points based on the feature information, and constructing an adaptive error compensation model to predict and correct a positioning error; dynamically adjusting the pose of the identification point through a real-time image sequence, and generating an optimal alignment control path; in the path execution process, image feedback and encoder information are fused to monitor the positioning precision in real time, and a dynamic path re-planning mechanism is triggered to realize micro correction; the method can be widely applied to a high-precision and high-consistency display module automatic assembly scene.

Description

Technical Field

[0001] This invention relates to the field of automated assembly technology, and more specifically to a visual positioning method for aligning and assembling display modules. Background Technology

[0002] In existing automated assembly processes, the alignment accuracy of modules directly affects the overall display effect and yield rate. Especially with the continuous development of new display modules such as narrow bezels, high resolution, and high refresh rates, the requirements for assembly accuracy are increasing.

[0003] Traditional assembly processes rely heavily on fixed mechanical fixtures and manual adjustments, which suffer from limited positioning accuracy, weak error compensation capabilities, and low efficiency. While some improved technologies have introduced image recognition and position closed-loop feedback mechanisms, they often depend on the extraction of single visual features (such as edges and corners) for matching. This makes them susceptible to complex working conditions such as reflections, occlusions, and background interference, resulting in poor recognition robustness and difficulty in achieving multi-dimensional, real-time alignment control of complex module components.

[0004] Furthermore, existing technologies lack effective means to distinguish the multi-layered structural characteristics between the module body and the substrate, which can easily lead to feature confusion during image processing, thus limiting the accuracy and adaptability of visual recognition. Summary of the Invention

[0005] The purpose of this invention is to provide a visual positioning method for the alignment and assembly of display modules, so as to overcome the shortcomings of the prior art.

[0006] To achieve the above objectives, the present invention provides the following technical solution: a visual positioning method for aligning and assembling display module components, comprising:

[0007] Acquire multi-source image data of the area of ​​the display module to be assembled;

[0008] The multi-source image data is subjected to image enhancement and feature fusion processing to construct a multi-dimensional joint visual feature matrix containing structural edges, illumination reflection textures and material marker points;

[0009] Based on the multi-dimensional joint visual feature matrix, key alignment markers of the module are extracted, and a spatial mapping relationship between them and the corresponding reference points on the substrate is established.

[0010] An adaptive error compensation model is constructed to dynamically correct the pose of the marker points based on the image sequence acquired in real time during the module assembly process, and to generate the alignment control path.

[0011] The alignment control path guides the module alignment operation and monitors and corrects positioning errors in real time.

[0012] Preferably, the multi-source image data includes orthophoto images, oblique projection images, and near-infrared images.

[0013] Preferably, the multi-source image data is subjected to image enhancement and feature fusion processing, including:

[0014] Illumination normalization and edge-preserving filtering are applied to orthophoto images, oblique projection images, and near-infrared images respectively, while enhancing the local contrast of structural contours and material textures.

[0015] Based on image spatial resolution and region of interest distribution, a multi-scale pyramid image set with modular structure is constructed. The image pyramid set is used to represent each source image at multiple resolutions and extract feature response maps.

[0016] Edge features, corner features, and texture gradient features extracted from different image sources are semantically paired and spatially fused to generate a unified feature response tensor. Redundant feature information is then weighted and fused based on a local self-attention mechanism.

[0017] A saliency region analysis is performed on the fused feature response tensor to extract stable and highly discriminative key structural regions of the module, and a feature mapping map for alignment control is constructed.

[0018] Preferably, a multi-dimensional joint visual feature matrix is ​​constructed, including structural edges, illumination reflection textures, and material marker points, comprising:

[0019] Edge detection and texture decomposition operations are performed on the enhanced multi-source images respectively. The edge detection uses the adaptive Canny algorithm to extract high-contrast structural boundaries, and the texture decomposition uses a multi-directional Gabor filter bank to extract the material surface reflection texture information.

[0020] By modeling and analyzing the response differences of each pixel in the image under different lighting conditions through reflectivity feature modeling, a light reflection coefficient layer that can distinguish material types is extracted, and stable reflection texture anchor points are marked as material identification references.

[0021] By combining a prior image sample library, a material classification network model based on feature points is constructed to automatically identify material marker points in images, including FPC interfaces, fixing holes, and the edges of OLED transparent layers, and extract their position information and local texture descriptors.

[0022] The structural edges, light reflection textures, and material markers are uniformly mapped to a normalized coordinate system. A three-dimensional visual feature matrix is ​​formed by stacking feature channels, and spatial consistency and semantic similarity are used for convolutional fusion to construct a multi-dimensional joint visual feature matrix.

[0023] Preferably, the construction of the adaptive error compensation model includes:

[0024] Historical alignment deviation data and real-time image positioning error information are collected during the module assembly process. An error evolution trend curve is constructed based on time series modeling, and disturbance factors affecting alignment accuracy are identified, including mechanical backlash, platform vibration and illumination interference.

[0025] A residual prediction model based on multi-source error factors is constructed to predict the magnitude and direction of alignment error under different conditions.

[0026] Preferably, the step of dynamically correcting the pose of the marker points based on the image sequence acquired in real time during module assembly and generating the alignment control path includes:

[0027] Image sequences of the module alignment area are acquired at fixed time intervals, and marker point detection and pose estimation are performed on each frame of the image to obtain the changes in the position and orientation of the marker points in continuous time sequence.

[0028] A time sliding window mechanism is used to filter the pose data in the image sequence to obtain stable and continuous motion trajectories of marker points;

[0029] Based on the corrected pose information, an error vector field of the module relative to the assembly datum is constructed. The module structural parameters, kinematic constraints of the moving mechanism and machining tolerance requirements are integrated to generate a set of optimal alignment control path points in real time. The path forms include straight line segments, circular arc segments and spline curve segments.

[0030] The generated control path is input into the motion control system, and motion control of the actuator is achieved through path pre-interpolation and speed curve planning.

[0031] Preferably, the step of guiding the module alignment operation based on the alignment control path and monitoring and correcting the positioning error in real time includes:

[0032] The generated alignment control path is input into the motion control system, and trajectory interpolation and velocity curve optimization are performed on the path point sequence. A five-segment S-shaped velocity planning strategy based on acceleration is adopted.

[0033] The control actuator guides the display module to the target assembly position according to the optimized path trajectory. During the process, the real-time pose data of the module is periodically collected and compared with the path reference point to calculate the actual positioning error.

[0034] A dual-source error monitoring mechanism based on image feedback and encoder fusion is introduced, in which the image system provides global position and orientation assessment of the module, and the encoder system provides displacement accuracy information of the actuator. The two are compared collaboratively, and the error resolution is better than 3 micrometers.

[0035] If the detection error exceeds the preset tolerance threshold, the control path is dynamically adjusted and the path segment is replanned and corrected in real time.

[0036] Preferably, the dynamic adjustment of the control path and real-time replanning and correction of path segments includes:

[0037] Based on the residual error between the current module pose and the target path point, an error vector E is constructed, which includes displacement deviation and angle deviation. Dynamic path adjustment trigger thresholds are set, with the displacement threshold set to 10 micrometers and the angle threshold set to 0.05 degrees.

[0038] When the residual error is detected to exceed the set threshold range, the shortest correction path segment is generated based on the principle of priority of correction direction, with the current position as the starting point and the target position as the ending point.

[0039] A path quality factor Q is introduced as the optimization objective for path segments. Q comprehensively evaluates the displacement cost, angle turning complexity, and execution time of the corrected path segments.

[0040] The newly generated corrected path segment is dynamically spliced ​​into the original path trajectory, and the instruction sequence in the controller trajectory buffer is updated synchronously. Path switching is performed in the next control cycle.

[0041] The technical effects and advantages provided by the present invention in the above technical solution are as follows:

[0042] 1. The visual positioning method for display module alignment and assembly provided by this invention achieves closed-loop optimization of the entire process from image perception to path execution by constructing modules such as multi-source image enhancement processing, feature fusion recognition, adaptive error compensation, and dynamic path control. This method not only effectively improves the recognition stability of the module under complex lighting, structural occlusion, and minor disturbance conditions.

[0043] 2. This invention has strong environmental adaptability and assembly precision assurance capabilities. It can achieve precise optimization of alignment accuracy and path reconstruction response time on high-speed production lines, thereby improving module yield and consistency. It is particularly suitable for the automated assembly needs of new display modules such as high resolution and ultra-narrow bezel. Attached Figure Description

[0044] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments recorded in this invention. For those skilled in the art, other drawings can be obtained based on these drawings.

[0045] Figure 1 This is a mind map of the method of the present invention. Detailed Implementation

[0046] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0047] For examples, please refer to Figure 1 As shown, the visual positioning method for aligning and assembling display modules described in this embodiment includes:

[0048] Acquire multi-source image data of the area of ​​the display module to be assembled;

[0049] The multi-source image data is subjected to image enhancement and feature fusion processing to construct a multi-dimensional joint visual feature matrix containing structural edges, illumination reflection textures and material marker points;

[0050] Based on the multi-dimensional joint visual feature matrix, key alignment markers of the module are extracted, and a spatial mapping relationship between them and the corresponding reference points on the substrate is established.

[0051] An adaptive error compensation model is constructed to dynamically correct the pose of the marker points based on the image sequence acquired in real time during the module assembly process, and to generate the alignment control path.

[0052] The alignment control path guides the module alignment operation and monitors and corrects positioning errors in real time.

[0053] In this invention, the multi-source image data includes orthophoto images, oblique projection images, and near-infrared images. The three work together to form an image information redundancy mechanism for structure recognition, thereby improving the overall image quality and feature extraction stability.

[0054] First, orthophoto images are acquired using a high-resolution industrial camera positioned vertically to the display module. These images are primarily used to obtain the two-dimensional contour information of various structures on the module surface and the positions of key marker points. Compared to oblique imaging, orthophoto images effectively avoid perspective distortion problems and offer high precision and consistency in image registration and alignment reference extraction. They are the most fundamental and stable image source during module alignment.

[0055] Secondly, to address the feature weakening issues caused by the multi-layered structure of the module body, edge occlusion, or backlight transmission, this invention adds an oblique projection image acquisition path. Specifically, an auxiliary vision acquisition device with a preset tilt angle (e.g., 30 degrees or 45 degrees) is set on each side of the orthophoto camera to acquire oblique projection images of the module from an oblique viewing angle. This type of image can compensate for the shortcomings of orthophoto images in recognizing three-dimensional structures, especially showing significant advantages in recognizing structures with height differences, such as module borders, fixed protrusions, and interface slots. Simultaneously, the oblique image provides three-dimensional feature clues of the local structure, which helps in subsequent structural depth estimation and spatial fitting through image fusion, enhancing the geometric matching capability of module alignment.

[0056] Furthermore, to address the issue of blind spots in conventional visible light images under strong lighting, reflective module surfaces, or complex backgrounds, this invention introduces a near-infrared image acquisition mechanism. By integrating a near-infrared imaging sensor into the image acquisition module, specific wavelengths (such as 850 nm or 940 nm) of light are projected onto the module area, and its infrared reflection image information is acquired. Infrared images are sensitive to differences in material reflectivity, significantly enhancing the contrast between non-metallic materials and the background, thereby improving the structural segmentation capability in weakly textured areas at the module edges. Simultaneously, infrared imaging technology has a natural resistance to ambient light changes, providing stable and clear images, making it particularly suitable for complex environments with frequent lighting changes in automated production lines.

[0057] In this invention, to improve the feature quality and subsequent recognition stability of multi-source image data, a systematic image enhancement and feature fusion processing strategy is proposed. This strategy addresses the diversity and heterogeneity of three image sources—orthophoto images, oblique projection images, and near-infrared images—by systematically processing them from dimensions such as image preprocessing, feature response modeling, and fusion algorithm design, in order to construct a robust and highly discriminative joint feature representation.

[0058] First, the original image undergoes illumination normalization and edge-preserving filtering. Due to unstable lighting sources in the assembly line environment, images acquired at different times or angles exhibit significant differences in brightness and contrast. Therefore, this invention employs an illumination normalization method combining histogram equalization and the Retinex model to uniformly map different image sources to a similar lighting space. Based on this, an edge-preserving filter (such as bilateral filtering or guided filtering) is introduced to remove background noise while maintaining structural contour integrity, thereby enhancing the image's structural perception and local contrast of material textures.

[0059] Next, based on the spatial resolution characteristics of different images and the distribution features of the modules' regions of interest (ROIs), a multi-scale image pyramid model is constructed. Specifically, each type of image is downsampled and residual-enhanced using both Gaussian and Laplacian pyramids to obtain image copies at multiple resolution levels. The corresponding feature response maps are then extracted at each scale level. This method facilitates scale-invariant feature extraction, exhibiting higher robustness, especially for structural features within the module structure such as positioning markers of varying sizes, material segmentation lines, and edge intersections.

[0060] After acquiring various feature maps, this invention further proposes a semantic pairing and spatial fusion strategy to unify edge features, corner features, and texture gradient features extracted from different image sources. Specifically, feature alignment operations are performed on key points in different modal images in scale space, including matching based on SIFT (Scale Invariant Feature Transform) or ORB (Oriented Fast and Rotated BRIEF) feature descriptors, to establish semantically consistent feature point pairs. Spatially, affine transformation and image registration techniques are used to map local features from each image source to the same reference coordinate system. The fusion stage employs a local self-attention mechanism, assigning each feature point a dynamically calculated weight based on its spatial stability and semantic importance to achieve weighted fusion and suppression of redundant features, thereby improving the stability and discriminativeness of the fused feature representation tensor.

[0061] Finally, saliency region analysis is performed on the fused unified feature tensor. By calculating the response consistency and local maxima regions of the feature tensor at different scales and modalities, spatially saliency and semantically representative structural regions, such as alignment markers, boundary structures, and hierarchical transition regions, are identified in the module region and used as candidate regions for subsequent alignment control. Saliency regions not only enhance the separability of features but also significantly reduce the false recognition rate and computational overhead, improving overall localization efficiency.

[0062] To further enhance the understanding and adaptation capabilities of complex module structures, this invention, after image enhancement and fusion processing, establishes a three-dimensional feature representation with rich geometric structure, optical response, and semantic material information by fusing three representative image features: structural edge features, illumination reflection texture features, and material marker point features.

[0063] Specifically, edge detection and texture decomposition are first performed on the preprocessed multi-source images. The edge detection part employs an improved adaptive Canny algorithm, dynamically adjusting high and low thresholds based on local pixel gradients to accurately extract edge structures such as the module's outer contour, positioning reference lines, and FPC connection areas. Texture decomposition is based on a multi-directional Gabor filter bank, with each filter having a different direction and frequency response. This is used to extract the periodic micro-texture structures on the material surface, and is particularly suitable for extracting the optical response patterns of OLED module transparent layers, conductive adhesive materials, and special inkjet coatings.

[0064] Subsequently, by constructing a reflectivity feature model, the response changes of each pixel in the image under different illumination angles and wavelengths were analyzed to extract a reflectivity coefficient layer with material discrimination. This coefficient was calculated based on the brightness difference of pixels in visible light and near-infrared images, and regions with stable reflectivity were marked as reflectivity texture anchor points to identify the visual feature boundaries of different materials.

[0065] To achieve accurate identification of key material points, this invention further introduces a prior image sample library and a material classification network model. By constructing a few-shot classification network, known feature point categories (such as FPC interfaces, fixing holes, OLED transparent layer edges, etc.) are mapped to semantic marker points, and their position coordinates in the image and surrounding local texture descriptors are extracted. These descriptors can be extracted through convolutional feature pooling or patch-based feature clustering, exhibiting a certain degree of robustness and cross-sample transfer capability.

[0066] After extracting various features, the system maps structural edge features, illumination reflection texture features, and material marker point features to a normalized two-dimensional coordinate system, and stacks features along the third dimension (channel dimension) to construct a three-dimensional visual feature tensor. Each layer of this tensor represents a type of original or fused feature, and each pixel position corresponds to a complete multi-channel feature vector.

[0067] To optimize the combination effect of feature information, a convolutional fusion mechanism weighted by spatial consistency and semantic similarity is finally used to further process the feature tensor. Specifically, in the spatial dimension, a similarity matrix is ​​constructed using the geometric positional relationships between image regions; in the semantic dimension, a similarity weight map is constructed using the mutual information metric between feature channels. These similarity maps are used as convolution weight adjustment factors to perform convolutional fusion operations on the overall feature matrix.

[0068] In the high-precision alignment and assembly of display modules, various sources of error affect assembly accuracy, such as equipment mechanical clearance, vibration interference, image recognition errors, thermal drift of the assembly platform, and fluctuations in module material properties. To effectively suppress and dynamically compensate for these error factors, this invention proposes an adaptive error compensation model based on multi-source data fusion and dynamic prediction mechanisms. This model can adjust the compensation strategy in real time according to changes in operating conditions, thereby improving overall alignment accuracy and stability.

[0069] This model first collects data from two sources through the data acquisition module:

[0070] Alignment deviation data from historical assembly processes (e.g., displacement residual between the loading point and the final positioning point of a module, in micrometers);

[0071] The pose error information (including lateral, longitudinal, and angular errors, in micrometers and arcseconds, respectively) generated by the real-time image system in locating the current frame marker point.

[0072] The data is accessed via structured log files or industrial fieldbus interfaces (such as EtherCAT or Modbus) and timestamped. Subsequently, a time series analysis model is used to model the trend of the error data. A sliding window exponentially weighted moving average (EWMA) algorithm is preferably used, specifically: the predicted value at the current moment is equal to the weighted average of the historical observations and the current observation, with the weights decreasing over time.

[0073] For example, setting the sliding window size to 20 frames, the time step to 40 milliseconds, and the EWMA weight coefficient α to 0.25 can effectively filter out short-term sudden fluctuations and retain long-term trend characteristics.

[0074] In addition, during the data cleaning process, three types of disturbance factors were introduced as reference labels for analysis and modeling:

[0075] Mechanical backlash error: calculated by the difference between the reverse displacement of the actuator and the feedback position of the encoder, with a threshold of 5 micrometers;

[0076] Platform vibration and noise: Use gyroscopes / accelerometers to obtain the frequency of motion interference, focusing on the 50-200Hz frequency band;

[0077] Illumination interference: The presence of backlight reflection or unstable light source is detected by the offset of the image grayscale histogram. The grayscale fluctuation threshold is set to ±12%.

[0078] This modeling process not only extracts the statistical characteristics of the error itself, but also explicitly identifies its causes, which has a positive promoting effect on the generalization ability of subsequent error prediction models.

[0079] This invention proposes a method for predicting error residuals based on fusion sensing, constructing a prediction model based on multi-source error factors. The preferred model is an Ensemble Bayesian Regression Network (BERN), which balances prediction accuracy with uncertainty measurement.

[0080] The specific structure is as follows:

[0081] Input variables: historical error sequence, error causative factors (hysteresis, vibration, light fluctuation), current environmental conditions (temperature, speed);

[0082] Network structure: Input layer → Double hidden layer (64 nodes per layer, ReLU activation function) → Dropout layer (to prevent overfitting, retain rate 0.6) → Output layer;

[0083] Output: Expected value and confidence interval of the current position error (confidence level set to 95%).

[0084] To train the model, a labeled dataset of the most recent 500 assembly cycles was used. Backpropagation was performed using the Adam optimizer with an initial learning rate of 0.001 and 300 training epochs. After training, the model was deployed on a local industrial edge server, with an execution cycle of no more than 10 milliseconds, meeting real-time requirements.

[0085] The prediction result is "the theoretical position of the current module marker point and the actual offset vector after disturbance", that is, the prediction error vector ΔE(x,y,θ). This error vector is used as a correction quantity in the control path generation to ensure that alignment accuracy can be maintained even in the presence of external disturbances.

[0086] To adapt to the need for synchronous updates of pose changes, recognition errors, and path planning during module movement, this invention further proposes a method for dynamic pose correction and control path generation based on image sequences, achieving a balance between control accuracy and response speed.

[0087] A high-speed industrial camera (with a acquisition frequency of 25 frames / second and a resolution of 1920×1200 pixels) is installed on the assembly platform to acquire image sequences of the module alignment area in real time at fixed time intervals (40ms).

[0088] The pose of the marker points is obtained through the following processing flow for each frame of the image:

[0089] Marker point detection: Identifying module marker areas based on an improved FAST corner detection algorithm;

[0090] Subpixel fitting: Improving positional accuracy through Harris response function and subpixel gradient interpolation;

[0091] Pose calculation: Using the PnP (Perspective-n-Point) algorithm and calibration model, combined with camera intrinsic parameters and the coordinates of the marker point image, the position (x,y) and orientation angle θ of the module in three-dimensional space are estimated.

[0092] To reduce sporadic errors, this invention introduces a time-sliding window filtering mechanism. The window size is set to 5 frames, and a weighted median filter is used to remove outlier frames. The extended Kalman filter (EKF) method is then combined to predict the pose of the next frame.

[0093] After acquiring a stable pose trajectory, the system automatically calculates the error vector field between the current state of the module and the assembly reference. This vector field, centered on the module's key points, calculates the difference (including displacement and angle differences) between the current coordinates and the target alignment coordinates, forming a three-dimensional error description.

[0094] Based on the error field, module structure geometric parameters, kinematic model of the motion mechanism (such as four-axis platform, parallel arm system, etc.) and process tolerance requirements (e.g., within ±10 micrometers is acceptable), the path planning algorithm in the control module is invoked.

[0095] The path generation method used is as follows:

[0096] When the error is small (<5 micrometers): a linear correction path is generated using linear interpolation;

[0097] For moderate deviations (5–30 micrometers): smooth transitions are achieved by generating circular arc paths or quadratic B-spline curve segments through interpolation;

[0098] When there is a large offset or a sudden change in angle: call the fifth-order spline curve algorithm to ensure the continuity of terminal acceleration and velocity.

[0099] The generated path point set is a discrete path sequence (Pi, i = 1 to N), and each point contains spatial location and preset motion speed parameters.

[0100] The final generated control path is transmitted to the motion control system (such as a multi-axis servo drive) in real time, and the system performs the following tasks within each control cycle (2ms):

[0101] Pre-interpolate the path point sequence to generate higher density trajectory points;

[0102] Based on the current module speed and acceleration status, execute S-shaped speed curve planning to avoid startup impact or positioning overshoot;

[0103] It continuously acquires image feedback and evaluates pose changes. If a deviation is detected that exceeds a set threshold (e.g., 10 micrometers or 1 degree), it automatically triggers path recalculation to ensure that the module moves to the target position along the optimal path.

[0104] To achieve high-precision alignment control and dynamic error compensation capabilities for display module assembly, this invention introduces a path tracking execution system based on vision-guided path planning, and combines it with an image-encoder fusion error monitoring mechanism to achieve closed-loop control of positioning accuracy throughout the entire process.

[0105] The first step in module alignment is path guidance execution. First, the path point sequence (denoted as P) generated by the aforementioned visual positioning module is... 1～ P), where n is the total number of path points, is input to the motion control system. To ensure stable motion without mechanical impact and to achieve high-frequency control command response, this invention performs the following processing steps before path execution:

[0106] Trajectory interpolation: Interpolate discrete path points to ensure the spacing between them does not exceed 10 micrometers. A cubic spline-based interpolation method is preferred, which maintains path continuity while ensuring the continuity of the first and second derivatives of position, velocity, and acceleration.

[0107] Speed ​​Curve Planning: To prevent jitter and positioning deviations during module start-up / stopping, this invention employs a five-segment S-shaped speed curve planning strategy based on acceleration variation (Jerk) control, including five stages: acceleration segment, constant acceleration segment, uniform speed segment, constant deceleration segment, and deceleration segment. The duration and speed variation of each stage are controlled by the following conditions:

[0108] The jerk is set to a maximum of 20,000 mm / s². 3 ;

[0109] The maximum acceleration is limited to 2000 mm / s². 2 ;

[0110] The maximum speed limit is set according to the module type, and usually does not exceed 300 mm / s;

[0111] The system automatically generates a speed profile based on path length, deviation tolerance, and control cycle to ensure that the actuator remains stable under continuous control.

[0112] During the module's movement along the path, the system periodically collects its real-time pose data and compares it with predetermined path reference points to calculate the current actual positioning error vector E, specifically:

[0113] Image system monitoring: Real-time acquisition of module images via high-speed industrial cameras, with recognition and pose estimation completed every 40ms, recognition accuracy better than ±5 micrometers, and angle error less than 0.02 degrees.

[0114] Encoder feedback monitoring: The absolute encoder in the drive mechanism provides real-time feedback on the movement distance of each axis of the platform, with a sampling period of less than 2ms and a resolution better than 1 micrometer.

[0115] Image and encoder data are fused via hardware and software (e.g., in a PLC or embedded ARM controller), and error estimation is performed using a weighted fusion model. The weights are set as follows:

[0116] If the image recognition signal quality (image sharpness + illumination stability score) is higher than the threshold T = 85 (out of 100), the image signal weight is set to 0.7;

[0117] Otherwise, the encoder weight is increased to 0.6, and the image weight is decreased to 0.4.

[0118] The final fusion output shows the actual pose of the current module, which is then differentially analyzed with the pose of the path points to obtain the error vector E = (Δx, Δy, Δθ), where Δx and Δy are displacement deviations, and Δθ is the angular deviation. The system error resolution is better than 3 micrometers, which can be used for minor corrections.

[0119] To improve assembly error response speed and path tracking flexibility, the system will automatically trigger a path correction mechanism when any component in the error vector E exceeds a set threshold. The dynamic path adjustment trigger threshold defined in this invention is:

[0120] Δx or Δy exceeds 10 micrometers;

[0121] Δθ exceeds 0.05 degrees.

[0122] Upon triggering, the system immediately performs the following steps:

[0123] The system uses the spatial error vector E between the current actuator position Pcurr and the preset target point Ptarget as the starting condition for replanning. To ensure efficiency in the correction process, a "correction direction priority principle" is adopted, namely:

[0124] Prioritize the correction of the direction with the largest error (e.g., when Δx>Δy, correct the X-axis deviation first), and simultaneously perform fine-tuning of other dimensions during the correction of the main direction.

[0125] Starting with Pcurr and ending with Ptarget, a locally corrected path segment is generated. The path segment configuration selection automatically switches according to error characteristics.

[0126] For small deviations (<30 micrometers), quadratic B-spline curves are preferred;

[0127] For areas with complex angle corrections or abrupt curvature changes, flexible oblique line segments are introduced to improve path smoothness.

[0128] To ensure optimal performance of the corrected path segment, a path quality factor Q is proposed, with the following evaluation criteria: Q = w1 × L + w2 × |Δθ| + w3 × T; where: L is the length of the corrected path segment (unit: millimeters); |Δθ| is the total angle of turn (unit: degrees); T is the estimated time required for correction (unit: milliseconds); and the weight values ​​w1, w2, and w3 reflect the assembly priority (for example, w3 is set to the maximum if speed response is emphasized).

[0129] The system uses this evaluation model to filter multiple feasible path segments and finally retains the path scheme with the smallest Q value.

[0130] The selected corrected path segment is dynamically appended to the original control path sequence, and the trajectory buffer queue of the motion controller is updated in real time. The trajectory points of the new path segment are executed in the next control cycle (usually 2ms to 5ms) to complete the corrected closed loop.

[0131] Meanwhile, to ensure the smoothness of path switching, the system adopts curvature continuity constraints and coordinates the first and second derivatives at the splicing points to avoid path oscillations.

[0132] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any changes or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application.

Claims

1. A visual positioning method for aligning and assembling display modules, characterized in that: include: Acquire multi-source image data of the area of ​​the display module to be assembled; The multi-source image data is subjected to image enhancement and feature fusion processing to construct a multi-dimensional joint visual feature matrix containing structural edges, illumination reflection textures and material marker points; Based on the multi-dimensional joint visual feature matrix, key alignment markers of the module are extracted, and a spatial mapping relationship between them and the corresponding reference points on the substrate is established. An adaptive error compensation model is constructed to dynamically correct the pose of the marker points based on the image sequence acquired in real time during the module assembly process, and to generate the alignment control path. The alignment control path guides the module alignment operation and monitors and corrects positioning errors in real time.

2. The visual positioning method for aligning and assembling display modules according to claim 1, characterized in that: The multi-source image data includes orthophoto images, oblique projection images, and near-infrared images.

3. The visual positioning method for aligning and assembling display modules according to claim 2, characterized in that: The multi-source image data is subjected to image enhancement and feature fusion processing, including: Illumination normalization and edge-preserving filtering are applied to orthophoto images, oblique projection images, and near-infrared images respectively, while enhancing the local contrast of structural contours and material textures. Based on image spatial resolution and region of interest distribution, a multi-scale pyramid image set with modular structure is constructed. The image pyramid set is used to represent each source image at multiple resolutions and extract feature response maps. Edge features, corner features, and texture gradient features extracted from different image sources are semantically paired and spatially fused to generate a unified feature response tensor. Redundant feature information is then weighted and fused based on a local self-attention mechanism. A saliency region analysis is performed on the fused feature response tensor to extract stable and highly discriminative key structural regions of the module, and a feature mapping map for alignment control is constructed.

4. The visual positioning method for aligning and assembling display modules according to claim 3, characterized in that: Construct a multi-dimensional joint visual feature matrix that includes structural edges, lighting reflection textures, and material marker points, including: Edge detection and texture decomposition operations are performed on the enhanced multi-source images respectively. The edge detection uses the adaptive Canny algorithm to extract high-contrast structural boundaries, and the texture decomposition uses a multi-directional Gabor filter bank to extract the material surface reflection texture information. By modeling and analyzing the response differences of each pixel in the image under different lighting conditions through reflectivity feature modeling, a light reflection coefficient layer that can distinguish material types is extracted, and stable reflection texture anchor points are marked as material identification references. By combining a prior image sample library, a material classification network model based on feature points is constructed to automatically identify material marker points in images, including FPC interfaces, fixing holes, and the edges of OLED transparent layers, and extract their position information and local texture descriptors. The structural edges, light reflection textures, and material markers are uniformly mapped to a normalized coordinate system. A three-dimensional visual feature matrix is ​​formed by stacking feature channels, and spatial consistency and semantic similarity are used for convolutional fusion to construct a multi-dimensional joint visual feature matrix.

5. The visual positioning method for aligning and assembling display modules according to claim 4, characterized in that: The construction of the adaptive error compensation model includes: Historical alignment deviation data and real-time image positioning error information are collected during the module assembly process. An error evolution trend curve is constructed based on time series modeling, and disturbance factors affecting alignment accuracy are identified, including mechanical backlash, platform vibration and illumination interference. A residual prediction model based on multi-source error factors is constructed to predict the magnitude and direction of alignment error under different conditions.

6. The visual positioning method for aligning and assembling display modules according to claim 5, characterized in that: The step of dynamically correcting the pose of the marker points based on the image sequence acquired in real time during module assembly and generating the alignment control path includes: Image sequences of the module alignment area are acquired at fixed time intervals, and marker point detection and pose estimation are performed on each frame of the image to obtain the changes in the position and orientation of the marker points in continuous time sequence. A time sliding window mechanism is used to filter the pose data in the image sequence to obtain stable and continuous motion trajectories of marker points; Based on the corrected pose information, an error vector field of the module relative to the assembly datum is constructed. The module structural parameters, kinematic constraints of the moving mechanism and machining tolerance requirements are integrated to generate a set of optimal alignment control path points in real time. The path forms include straight line segments, circular arc segments and spline curve segments. The generated control path is input into the motion control system, and motion control of the actuator is achieved through path pre-interpolation and speed curve planning.

7. The visual positioning method for aligning and assembling display modules according to claim 6, characterized in that: The alignment operation of the module guided by the alignment control path, and the real-time monitoring and correction of positioning errors, includes: The generated alignment control path is input into the motion control system, and trajectory interpolation and velocity curve optimization are performed on the path point sequence. A five-segment S-shaped velocity planning strategy based on acceleration is adopted. The control actuator guides the display module to the target assembly position according to the optimized path trajectory. During the process, the real-time pose data of the module is periodically collected and compared with the path reference point to calculate the actual positioning error. A dual-source error monitoring mechanism based on image feedback and encoder fusion is introduced, in which the image system provides global position and orientation assessment of the module, and the encoder system provides displacement accuracy information of the actuator. The two are compared collaboratively, and the error resolution is better than 3 micrometers. If the detection error exceeds the preset tolerance threshold, the control path is dynamically adjusted and the path segment is replanned and corrected in real time.

8. The visual positioning method for aligning and assembling display modules according to claim 7, characterized in that: The dynamic adjustment of the control path and real-time replanning and correction of path segments include: Based on the residual error between the current module pose and the target path point, an error vector E is constructed, which includes displacement deviation and angle deviation. Dynamic path adjustment trigger thresholds are set, with the displacement threshold set to 10 micrometers and the angle threshold set to 0.05 degrees. When the residual error is detected to exceed the set threshold range, the shortest correction path segment is generated based on the principle of priority of correction direction, with the current position as the starting point and the target position as the ending point. A path quality factor Q is introduced as the optimization objective for path segments. Q comprehensively evaluates the displacement cost, angle turning complexity, and execution time of the corrected path segments. The newly generated corrected path segment is dynamically spliced ​​into the original path trajectory, and the instruction sequence in the controller trajectory buffer is updated synchronously. Path switching is performed in the next control cycle.

Images