An active optical alignment method based on a local light-weight language model

Abstract

The application discloses an active optical alignment method based on a local lightweight language model and particularly relates to the field of precision assembly and adjustment, and comprises the following steps: image indicators and platform attitude information are extracted through image acquisition, a unified input format is constructed, a language model is input after verification, a six-degree-of-freedom pose adjustment strategy and a credibility score are generated; the strategy is verified by a preset criterion and is executed by a platform control device; the strategy is scored again after execution, and whether the strategy is recorded as a characteristic attitude or triggers error backfill is judged according to the score result, which is used for optimizing subsequent input and strategy generation, improving alignment accuracy and stability; the application guarantees data consistency through a unified input format and a verification mechanism, improves alignment accuracy in combination with a dynamic scoring strategy, and enhances system stability and fault tolerance by introducing a characteristic attitude recording and error backfill mechanism, so that a strategy generation and execution process with high reliability, high adaptability and high robustness in an active optical alignment task is realized.

Description

Technical Field

[0001] This invention relates to the field of precision alignment technology, and more specifically, to an active optical alignment method based on a local lightweight language model. Background Technology

[0002] In fields such as optical module assembly, semiconductor manufacturing, and high-precision inspection, active optical alignment technology is a key process widely used in the coordinated control of six-degree-of-freedom platforms and image acquisition units. Traditional methods often rely on fixed rule algorithms or simple image feedback closed-loop systems for platform pose adjustment, but these methods are difficult to adapt to the actual needs of changing image features, process flow variations, and multi-task switching, especially exhibiting significant bottlenecks in accuracy control and alignment stability.

[0003] In recent years, with the development of natural language processing technology, lightweight language models, due to their excellent generalization ability and structured instruction generation capabilities, have been attempted to be introduced into optical alignment tasks to generate alignment strategies based on image metrics and platform pose information. In some solutions, locally deployed lightweight language models possess the ability to run and reason about strategies in real time in edge environments, improving the intelligence and adaptability of policy adjustments. However, a key vulnerability remains in existing technologies: when the language model receives erroneous, missing, or inconsistent contextual information (such as mismatches between image metrics and pose data, or outdated task parameters), it is highly susceptible to producing policy outputs with high confidence but logical errors. This "illusionary strategy," once executed by the platform controller via an interface, can lead to serious consequences such as the six-axis platform operating beyond its range, damage to optical components, process interruption, or batch product scrapping.

[0004] Furthermore, there are some hidden dangers in the management and utilization mechanism of historical feature pose records. When encountering poor alignment results or image drift trends, it is impossible to effectively compare the offset difference between the current state and the historical benchmark. Without a suitable utilization mechanism, the language model will continue to reason and adjust based on the current abnormal data, which may lead to a dead loop of policy offset and cause alignment failure. Therefore, this invention proposes an active optical alignment method based on a local lightweight language model to solve the above problems. Summary of the Invention

[0005] To achieve the above objectives, the present invention provides the following technical solution: An active optical alignment method based on a local lightweight language model includes the following steps: The image acquisition unit acquires images under the initial field of view, extracts sharpness and position offset values, and records the current platform's attitude parameters as image indicators and attitude information. Image metrics, attitude information, process stage identifiers, and historical task reference data are combined to construct a unified input format, while retaining key context fields; a verification mechanism is introduced to analyze the logical relationships and content matching between fields and eliminate input combinations that do not meet the input standards. The validated input format is fed into the locally running language model, and the output is a structured strategy containing suggestions for the adjustment direction and magnitude of the six degrees of freedom, as well as a corresponding credibility score. The strategy content output by the language model is verified according to the pre-set verification criteria. The platform control device is activated to perform adjustments only when all criteria are met. After the platform control device completes the pose adjustment, it acquires images again and calculates the score value. The score value is composed of the product of the sharpness value and the first weight, minus the product of the position offset value and the second weight. The first weight and the second weight are dynamically set according to the task type. When the score value meets the preset stability condition or the continuous change range is less than the set change threshold, the image index and platform attitude are combined and written into the feature attitude corresponding record table; if there is a difference in the image index before and after subsequent calibration that exceeds the preset tolerance, error compensation is performed based on the most recent record content, and the input format content is updated.

[0006] In a preferred embodiment, before extracting the sharpness value and position offset value, edge enhancement and region segmentation processing are performed on the image under the initial field of view. Gradient enhancement operator is used in the image processing unit to enhance the image edges. Based on the image's morphological features or gray level, the image is divided into multiple candidate regions. A rectangular region with the image center as the geometric center and a width and height each accounting for 20% of the image size, or the candidate region with the highest score in the previous round of scoring, is selected as the region of interest. The geometric center corresponding to the candidate region is defined as the average value of all pixel coordinates within the boundary of the candidate region, i.e., the centroid coordinates.

[0007] In a preferred embodiment, extracting the sharpness value and the positional offset value refers to: Multiple fixed sampling points are preset in the region of interest. The pixel grayscale of each sampling point is used as the sharpness index for that point. A weighted average is then calculated based on the distance of each sampling point from the geometric center of the region of interest to obtain the sharpness value. The spatial coordinate difference between the geometric center of the region of interest and the center of the image field of view is used as the position offset value, where the center of the field of view is the coordinate point corresponding to the midpoint of the image width and height.

[0008] In a preferred embodiment, the step of constructing a unified input format includes the following process: The image metrics and platform posture information are normalized into structured key-value pairs to form an image state set. Extract the process stage identifier and the current task batch number, and compare them with the recorded image indicators and platform attitude information in the historical task reference data to filter out reference samples with a matching degree higher than the preset similarity threshold. The image state set, matching samples, platform posture, and task identifier are encapsulated into a unified input format. The key context fields retained include the current image's sharpness value, position offset value, the platform posture's position information in six directions, and the historical task number and score value.

[0009] In a preferred embodiment, during the screening process, based on the process stage identifier and the current task batch number, the recorded image indicators and platform attitude information in the historical task reference data are searched and combined into a historical vector. The current image indicators and platform attitude parameters are combined into a current vector. Cosine similarity is used as the matching metric. The current vector and the historical vector are similar to each other, and a preset similarity threshold is used as the judgment standard.

[0010] In a preferred embodiment, the application logic of the verification mechanism is as follows: The image metrics, platform posture, process stage identifiers, and historical sample score values ​​in the completed input format are converted into structured triples with attribute labels one by one, and a logical dependency matrix between the fields is constructed. The system performs consistency calculations on the logical relationship between any two fields, constructs semantic comparison triples based on the attributes and values ​​of each field, outputs a support score through a preset rule base, and then compares it with a preset logical consistency threshold. Field combinations that are below the logical consistency threshold are marked as invalid and removed.

[0011] In a preferred embodiment, the pre-defined verification criteria include pose adjustment range, movement speed limit, acceleration limit, and process safety conditions.

[0012] In a preferred embodiment, the first weight and the second weight are obtained in the following way: Based on the current task type, retrieve the scoring strategy that matches the process stage identifier and task batch number from the preset task parameter table, and extract the first weight and second weight corresponding to the scoring strategy.

[0013] The locally running language model is built based on any of the lightweight pre-trained model structures of MiniLM, DistilBERT, or TinyBERT. It receives structured input consisting of image metrics and platform pose, performs input parsing and policy reasoning in the local device environment, and outputs a structured policy result containing pose adjustment suggestions in six directions, including displacement values ​​along the three axes and rotation angles around the three axes. Based on the matching strength between the structured policy result and the task objective, a corresponding credibility score is generated. The calculation logic for matching strength is as follows: for each set of candidate adjustment values ​​generated by the language model, construct an estimate of the image score value after its effect, and normalize the numerical difference between the estimate and the preset task target score value to obtain the matching strength between the set of candidate adjustment values ​​and the task target, which is then used as the credibility score.

[0014] In a preferred embodiment, error recovery refers to: when the score value meets the preset stability condition or its continuous change range is less than the set change threshold, the image index and platform posture are recorded in a pair into the feature posture corresponding record table, and the current task type and recording time are marked at the same time to form a traceable historical benchmark. If, during the subsequent alignment process, the difference between the image metrics before and after calibration exceeds the preset tolerance range, the offset between this difference and the most recent benchmark record is calculated to form an error vector. Based on the error vector, the input format content to be input into the language model is dynamically supplemented, and the error vector is superimposed on the current input content of the locally running language model as the standard input structure for subsequent inference and policy generation of the language model.

[0015] The technical effects and advantages of this invention are as follows: This invention acquires images through an image acquisition unit and extracts image metrics and pose information. These, combined with process stage identifiers and historical task reference data, are then used to construct a unified input format. A verification mechanism is introduced to analyze the logical relationships and content matching between fields, enabling timely identification of potential input anomalies or inconsistent combinations during the input phase. This mechanism effectively reduces language model strategy bias caused by data errors or inconsistent context. Especially in task switching, batch updates, or abnormal task scenarios, it maintains the semantic stability and structural integrity of the input, enhancing the accuracy and controllability of the language model generation strategy and ensuring stable operation of the active optical alignment task under different conditions.

[0016] This invention quantitatively evaluates the alignment effect by re-acquiring images and calculating a score after the platform control device completes the adjustment operation. The score is composed of the product of the sharpness value and a first weight, minus the product of the positional offset value and a second weight. This calculation method comprehensively considers both image quality and attitude accuracy, accurately reflecting the quality performance of the alignment result after platform adjustment. Simultaneously, the first and second weights are dynamically set according to the task type, ensuring that the scoring mechanism can flexibly adapt to the differentiated requirements of different task stages or process conditions. This avoids misjudgments caused by a rigid scoring mechanism, significantly improving the adaptability between the attitude adjustment strategy and the current task objective, and ensuring control accuracy and process consistency in the active optical alignment process.

[0017] This invention, under the premise that the score value meets the preset stability condition or the continuous change range of the score is less than the set change threshold, records the image index and platform posture in a feature posture corresponding record table to form a reference benchmark in the task process. In the subsequent alignment process, if the difference between the image index before and after calibration is detected to exceed the preset tolerance range, the error offset is calculated by comparing it with the most recently recorded content, and the input format content is dynamically supplemented to form a language model input that is closer to the target state. This mechanism ensures that the system has the ability to self-correct sudden disturbances while avoiding adjustment interruptions caused by the failure of a single strategy, improving the stability, continuity and adaptability of the entire active optical alignment process, and is especially suitable for precision manufacturing scenarios with high precision and high yield requirements. Attached Figure Description

[0018] To facilitate understanding by those skilled in the art, the present invention will be further described below with reference to the accompanying drawings; Figure 1 This is a schematic diagram of an active optical alignment method based on a local lightweight language model, as described in this invention. Detailed Implementation

[0019] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of the present invention.

[0020] Reference Figure 1 The following examples were obtained: Example 1: An active optical alignment method based on a local lightweight language model, comprising the following steps: acquiring an image under the initial field of view through an image acquisition unit, extracting sharpness and position offset values, and recording the current platform's attitude parameters as image indicators and attitude information; establishing the initial observation state for active alignment, acquiring image data under the current physical field of view through the image acquisition unit, and calculating sharpness and position offset values ​​based on a set image processing method. The platform's six-directional attitude parameters are recorded synchronously, enabling the subsequent inference model to have a complete input context, realizing the linked expression of image information and motion state, and laying the foundation for constructing a structured input format.

[0021] Image metrics, pose information, process stage identifiers, and historical task reference data are combined to construct a unified input format, while retaining key context fields. A validation mechanism is introduced to analyze the logical relationships and content matching between fields, eliminating input combinations that do not meet the input standards. Multidimensional data related to the current task are integrated, including current image metrics and platform pose, the current process stage status, and experience data from similar past tasks, and standardized into a structured format for model inference. Simultaneously, the validation mechanism checks the constructed input structure to analyze for logical conflicts or input anomalies between fields, avoiding errors in language model generation strategies due to context mismatch.

[0022] The validated input format is fed into a locally running language model, which outputs a structured strategy containing suggestions for adjustment directions and magnitudes across six degrees of freedom, along with a corresponding credibility score. Leveraging the model's reasoning capabilities, the current state is matched with historical knowledge to output physically executable six-degree-of-freedom pose adjustment suggestions, including translations along three axes and rotations around three axes. The model also outputs a credibility score, reflecting the degree of match between the suggested strategy and the task objective, providing a quantitative basis for subsequent judgments on the strategy's feasibility.

[0023] Based on pre-defined verification criteria, the strategy content output by the language model is verified. The platform control device is activated to perform adjustments only when all criteria are met. Before the strategy is executed by the platform, a complete set of criteria verification is conducted based on engineering safety and control constraints to ensure that the strategy recommendations are legal and compliant under pose range, velocity, acceleration, and system flow conditions. This rigorous screening avoids the risks of over-adjustment or nonlinear response, ensuring platform operational safety and alignment accuracy.

[0024] After the platform control device completes pose adjustment, it acquires images again and calculates a score. The score is composed of the product of sharpness and a first weight, minus the product of position offset and a second weight. The first and second weights are dynamically set according to the task type. By collecting real-time feedback on the adjustment results, the image state and position state after the pose adjustment are quantified and integrated into a unified score to reflect the effectiveness of the strategy. The scoring model integrates two core indicators, sharpness and position offset, and dynamically adjusts the weights to achieve adaptability between the scoring logic and the task objectives, thereby supporting subsequent stability assessment and optimization.

[0025] When the score meets the preset stability condition or the continuous change is less than the set change threshold, the image index and platform attitude are combined and written into the feature attitude corresponding record table. If there is a difference in the image index before and after subsequent calibration that exceeds the preset tolerance, error compensation is performed based on the most recent record, and the input format is updated. When the score shows a stable trend, the system records it as a set of reliable feature state data and writes it into the feature attitude corresponding record table for reference and comparison in subsequent calibration processes. If abnormal changes in the image index occur in the future, the system will actively trigger the error compensation process, construct an error vector based on the recorded data to correct the current input format, avoid the language model from becoming unstable due to drift or sudden changes, and ensure the robustness and fault tolerance of the alignment process.

[0026] In one embodiment of this invention, to obtain core data that can be used to construct image metrics, after the image acquisition unit completes image acquisition, the system performs edge enhancement and region segmentation processing on the image under the initial field of view through the image processing unit, and further performs operations to extract sharpness values ​​and positional offset values. Specifically, this includes the following steps: Edge enhancement processing is performed on the image under the initial field of view in the image processing unit, using a gradient enhancement operator to enhance the image edges. The gradient enhancement operator can be an improved Sobel operator, a Laplacian of Gaussian enhancement algorithm, or an edge sharpening method based on guided filtering, to improve the contrast between structurally clear regions and blurred regions in the image. For example, for an image with a size of 1024 x 768 pixels, the system performs three-channel gradient enhancement, resulting in an average increase of 20 percentage points in edge response intensity compared to the original image, effectively enhancing the sharpness of feature boundaries and providing a foundation for subsequent region segmentation.

[0027] Image region segmentation is performed based on morphological features or grayscale levels, dividing the entire image into several candidate regions, each with a bounded set of pixels. Region segmentation methods can be implemented using existing techniques such as region growing clustering or multi-scale segmentation map methods. Specifically, the implementation steps are as follows: The image undergoes preliminary preprocessing to unify brightness and contrast, constructing a unified grayscale image structure. For this grayscale image, a seed-point-based region growing clustering algorithm or a multi-scale segmentation map algorithm is executed to complete the image region segmentation.

[0028] In region-growing-based clustering, the system automatically selects multiple initial seed points in the image as the starting positions for region growth, and expands outwards to form regions based on the similarity of pixel grayscale values ​​or a threshold for gradient value changes. If the grayscale difference between the current pixel and a pixel in the grown region is lower than a set grayscale difference threshold (e.g., 10 grayscale levels), the pixel is assigned to that region; otherwise, a new grown region is generated, continuing until all pixels have been traversed. Each ultimately formed region has a closed boundary and constitutes a set of candidate regions with a set of pixels.

[0029] In multi-scale segmentation graph-based methods, the system constructs a graph representation of the image, treating each pixel as a node in the graph. Edges are established between nodes based on adjacency, with pixel gray-level differences serving as edge weights. By calculating the graph partitioning structure at different scales (such as image cutting based on graph cut algorithms, NormalizedCut, or SLIC superpixel segmentation), the image is divided into multiple visually coherent regions with strong internal consistency. After segmentation, the system assigns a number and attribute label to each region, including region boundaries, area, gray-level variance, and edge density. This region segmentation method can generate multiple candidate regions.

[0030] Based on this, the system selects the region of interest (ROI) from the candidate regions for subsequent image metric extraction. Two priority selection strategies are implemented: first, a rectangular region with its geometric center at the image center and a width and height each accounting for 20% of the image size is selected as a stable reference region; second, if the previous scoring result has already been generated, the candidate region with the highest score is prioritized as the ROI. For example, with an image width of 1024 pixels and a height of 768 pixels, the system will crop a rectangular region of approximately 205 x 153 pixels at the image center as the default ROI. The following logic can be set: if the score of the highest-scoring region in the previous round exceeds a switching threshold, then the region selected from the previous method (using a rectangular region with its geometric center at the image center and a width and height each accounting for 20% of the image size) is replaced with this region, which better reflects the actual situation.

[0031] For a defined region of interest (ROI), the system pre-defines multiple fixed sampling points within it. These sampling points are positioned with a fixed offset relative to the boundary coordinates of the ROI. For example, the ROI might be defined as a 5x5 grid, resulting in a total of twenty-five evenly distributed sampling points. The system uses the pixel grayscale value corresponding to each sampling point as a sharpness index for that point. Furthermore, it uses the Euclidean distance between each sampling point and the geometric center of the ROI as a weighting factor to calculate a weighted average of the sharpness indices for all sampling points, thus obtaining the sharpness value corresponding to the ROI. Sampling points closer to the geometric center receive a higher weighting coefficient. For example, points less than ten pixels from the center have a weighting coefficient of one, while points more than fifty pixels from the center have a weighting coefficient reduced to 0.25, effectively highlighting the image quality variations in the central region.

[0032] The relative positional offset of the region of interest (ROI) is calculated based on the spatial coordinate difference between its geometric center and the field of view center of the entire image. The field of view center is defined as the coordinate point corresponding to the midpoint of both the image width and height. In the aforementioned image size of 1024 x 768 pixels, the corresponding field of view center coordinates are (512, 384). Assuming the geometric center coordinates of the ROI are (480, 395), the positional offset is a vector distance of 32 pixels along the horizontal axis and 11 pixels along the vertical axis. This vector can be used for subsequent scoring calculations and is included as the positional offset portion of the image metrics in a unified input format.

[0033] In one embodiment of this invention, to enable the language model to accurately understand image states and task context information and improve the generation accuracy of posture adjustment strategies, after extracting image metrics and platform posture information, it is necessary to combine the image metrics, posture information, process stage identifiers, and historical task reference data into a unified input format, while retaining key context fields. The steps for constructing the unified input format include the following: Normalizing the fields of image metrics and platform posture information. Image metrics include sharpness values ​​and position offset values; platform posture information includes displacement values ​​along three axes and rotation angles around three axes, totaling position information in six directions. All field data are standardized and converted into structured key-value pairs to form an image state set. For example, sharpness values ​​are normalized to between zero and one, position offset values ​​are expressed as integers in pixels, and platform posture information is unified as floating-point values ​​in millimeters and angles, with accompanying field labels such as "Position_X" and "Rotation_Y" to ensure consistency in subsequent data parsing. After structuring, the following example can be constructed: {"Sharpness": 0.82, "Offset": 15, "Position_X": 1.2, "Position_Y": 0.8, ...}, which serves as the core part of the image state set input format.

[0034] Extract the process stage identifier and task batch number of the current task. These two fields, derived from the instruction information of the task execution control flow, are key indexes for constructing the context. Taking "Process stage identifier = glue dispensing completed, task batch number = BATCH_202512" as an example, samples with recorded image metrics and platform attitude information are filtered from the historical task reference data. By constructing feature vectors of the current image state set and historical sample data, and calculating their matching degree using cosine similarity, historical samples with similarity higher than a preset similarity threshold (e.g., 0.92) are selected as reference samples. For example, if a historical image metric has a sharpness of 0.79 and an offset of 17, and the platform attitude is a displacement of 1.0 mm on the X-axis and a rotation of 0.3 degrees around the Z-axis, its cosine similarity with the current sample is 0.94, which can be used as a valid reference.

[0035] The image state set, matching samples, platform pose information, and task identifier are encapsulated into a unified input format, serving as the standard input structure for the locally running language model. This unified input format is constructed using structured JSON or nested key-value pair arrays, with each field having a clear data label and value. The encapsulated structure explicitly retains key context fields, including the current image's sharpness value, position offset value, the platform's pose in six directions, and the historical task number and score. For example, the historical task number could be written as "BATCH_202403," and its corresponding score record would be "Score: 0.865," embedded within the unified format structure.

[0036] After the unified input format is constructed, a final verification of the integrity of key context fields is required. This ensures that the fields in the image state set are complete, the matching sample data is valid, the platform's position information in all six directions is complete, the task identifier is accurate, and the historical task numbers correspond one-to-one with the score values. This step guarantees that the unified input format has a closed-loop contextual logic, avoiding misinferences or abnormal outputs in the language model due to missing fields or semantic inconsistencies, thus providing a consistent information foundation for policy generation.

[0037] When selecting the closest reference sample to the current task from historical task reference data, to improve the contextual accuracy of language model inference, it is necessary to filter historical records based on the process stage identifier and the current task batch number, and to construct feature vectors based on image metrics and platform posture information for similarity calculation. This implementation includes the following steps: Based on the process stage identifier and the current task batch number obtained from the current task, index filtering is performed in the historical task reference data to limit the candidate reference range. For example, when the process stage identifier is "lens positioning stage" and the current task batch number is "BATCH_202601", all record samples under the same process stage are retrieved from the historical task reference data, such as retrieving one thousand historical records, each of which contains image metrics and platform posture information. Taking a certain historical record as an example, its sharpness value is 0.79, its position offset value is 16 pixels, and the platform posture information includes displacement values ​​of 1 mm, 0.8 mm, and -0.5 mm along the three axes, and rotation angles of 0.1 degree, -0.3 degrees, and 0.05 degrees around the three axes. By filtering historical records whose process stage identifiers match the current task batch number, the accuracy of the matching stage can be significantly improved, limiting subsequent calculations to the scope of historical tasks that are of similar significance.

[0038] The image metrics of the current task are combined with the platform attitude information to form a current vector, and the image metrics and platform attitude information from each historical task reference data are combined to form a historical vector. The current vector consists of a sharpness value, a positional offset value, and positional information in six directions. For example, the sharpness value of the current task is 0.82, the positional offset value is 15 pixels, and the platform attitude information is displacement along three axes of 1.2 mm, 0.8 mm, and -0.4 mm, and rotation around three axes of 0.1 degree, -0.3 degree, and 0 degree. Thus, the current vector is constructed as [0.82, 15, 1.2, 0.8, -0.4, 0.1, -0.3, 0.0]. Similarly, each historical record is converted into a historical vector with consistent dimensions, such as [0.79, 16, 1.0, 0.8, -0.5, 0.1, -0.3, 0.05]. Through vectorization, the image metrics and platform attitude information are transformed into computable mathematical objects, laying the foundation for cosine similarity calculation.

[0039] The similarity of the current vector and historical vectors is calculated one by one, using cosine similarity as the matching metric. The cosine similarity formula is the dot product of the current vector and the historical vector, divided by the product of the magnitudes of the two vectors. A value closer to one indicates greater similarity. For example, a cosine similarity of 0.942 between the current vector and a sample historical vector indicates high similarity in terms of sharpness trend, positional offset, and platform attitude variation. From one thousand historical vectors, one thousand similarity results are generated, forming a reference for ranking.

[0040] The similarity result of each historical vector is compared with a preset similarity threshold, and reference samples with similarity higher than the threshold are selected. For example, if the preset similarity threshold is set to 0.90, in the example above, the cosine similarity of 0.942 is higher than the threshold, so this historical record is included in the reference sample set. If 200 records out of 1,000 meet the threshold condition, these 200 records will serve as valid historical references for language model inference, providing high-quality contextual support for building a unified input format. The preset similarity threshold can be dynamically adjusted according to product characteristics to ensure that neither high-value historical data is ignored nor noisy data is introduced.

[0041] In one embodiment of this invention, to ensure that the constructed unified input format meets the language model's reasoning requirements under conditions of structural integrity, semantic coherence, and task constraints, a verification mechanism is introduced to analyze the logical relationships and content matching between image metrics, platform posture, process stage identifiers, and historical sample score values, and to exclude input combinations that do not meet the input standards. The specific implementation of this verification mechanism includes the following steps: Image metrics, platform posture, process stage identifiers, and historical sample score values ​​in the constructed input format are converted item by item into structured triples with attribute labels. Each field is reconstructed into a structure like <field name, field value, attribute label>. For example, a sharpness value can be converted into <sharpness value, 0.82, high-quality area>, a displacement in a certain direction in platform posture information can be converted into <Position_X, 1.2 mm, micro-displacement>, a process stage identifier such as "lens positioning stage" can be converted into <process stage identifier, lens positioning stage, high-precision requirement>, and a historical sample score value such as 0.865 can be converted into <score value, 0.865, high-confidence record>. After combining the triples of all fields, a logical dependency matrix is ​​further constructed between the fields. For example, the matrix defines a positive correlation between the clarity value and the score value, a weak coupling between the translation direction and the rotation direction in the platform attitude, and a strong constraint between the process stage identifier and the platform attitude quality. The logical dependency matrix makes the semantic relationships between the fields computable, laying the foundation for the next step of consistency judgment.

[0042] Based on the structured triples described above, consistency calculations are performed on the logical relationships between any two fields. This consistency calculation expresses the logical inference relationships between fields by constructing semantic contrast triples. For example, if the sharpness value is 0.82 and the historical sample score is 0.35, a logical relationship is established using the semantic contrast triple <sharpness value, high; score value, low; contradictory trend>. After the semantic contrast is input into the rule base, the rule base infers the logical consistency between field attributes and values ​​and outputs a support score. For example, if the rule base determines that a high sharpness value usually corresponds to a high score value, the support score for this contrast relationship might be 0.15, indicating a semantic conflict; conversely, if the sharpness value is 0.82 and the score value is 0.865, the support score might be 0.92, indicating logical consistency between the fields. The rule base contains domain knowledge rules, such as "high-resolution areas usually correspond to low positional offset values" and "high score values ​​often correspond to stable postures," and the support scores are generated based on these rules.

[0043] The support score of each pair of fields is compared with a preset logical consistency threshold to determine whether it meets the input criteria. The logical consistency threshold is set based on the stability requirements of the task scenario and can be calculated based on the statistical results of historical task reference data. For example, by statistically analyzing the field relationship support score distribution of one thousand successful alignment samples, it is determined that the field consistency of the vast majority of successful tasks is higher than 0.70, so the preset logical consistency threshold can be set to 0.70. If the support score obtained by the rule base analysis of a certain field combination is 0.65, which is lower than the preset logical consistency threshold, then the field combination is marked as invalid. For example, if the platform attitude information shows a rotation angle of more than five degrees, and the current task is in the precision positioning stage, the rule base may output a low support score, thus identifying it as a logical anomaly.

[0044] All field combinations below the logical consistency threshold are removed, and related input fields are removed or replaced in the unified input format. This ensures logical consistency, semantic coherence, and content consistency between image metrics, platform pose, process stage identifiers, and historical sample scores in the final unified input format. This guarantees that the language model receives a clear and complete input context, preventing incorrect strategies or abnormal outputs in the inference results. The unified input format, after removal, only contains field combinations validated by logical relationships, effectively improving the reliability and accuracy of subsequent pose adjustment suggestions.

[0045] In one embodiment of this invention, a validated input format is input into a locally running language model to generate a structured strategy containing suggestions for adjusting the direction and magnitude of six degrees of freedom, and a confidence score corresponding to the strategy is calculated. To ensure the controllability, security, and effectiveness of the strategy during physical execution, this invention further introduces pre-defined verification criteria to perform multi-dimensional verification of the strategy content before execution. Specifically, this includes the following steps: inputting the input format processed by the previous stage verification mechanism into a language model deployed in the device's local computing environment. This language model is built based on any one of the lightweight pre-trained model structures of MiniLM, DistilBERT, or TinyBERT. It parses the structured input format containing image metrics and platform pose information, executes the complete inference process locally, and outputs a set of structured strategy results containing six-directional pose adjustment suggestions. This structured strategy includes displacement values ​​along three axes and rotation angles around those axes. For example, the output suggestion might be a positive displacement of 0.12 mm in the X direction, a negative displacement of 0.08 mm in the Y direction, and a positive displacement of 0.05 mm in the Z direction, while simultaneously rotating 0.1 degrees clockwise around the X-axis, 0.05 degrees counterclockwise around the Y-axis, and 0.02 degrees clockwise around the Z-axis. The language model also calculates the matching strength between this set of suggestions and the task objective, obtaining a credibility score. For example, a score of 0.87 indicates that the adjustment strategy has high feasibility in the current input context.

[0046] Based on this structured strategy and credibility score, each adjustment parameter in the strategy undergoes format integrity checks and numerical boundary assessments to ensure that the basic data for each degree of freedom adjustment recommendation meets the execution requirements. To this end, this invention sets physical boundary ranges for each of the six degrees of freedom. For example, on a precision optical platform, the maximum allowable displacement in the X direction is ±0.5 mm, in the Y direction ±0.4 mm, in the Z direction ±0.3 mm, and the upper limit for the three-axis rotation angle is ±1.0 degree. If a recommended value for a degree of freedom exceeds the boundary, the strategy is directly deemed invalid and will not proceed to the subsequent verification stage. This step effectively filters out out-of-bounds strategies caused by input drift or inference anomalies, avoiding hardware risks during platform execution.

[0047] After boundary verification, the displacement values ​​and rotation angles included in the structured strategy are further verified through dynamic behavioral constraints, including movement speed limits, acceleration limits, and process safety condition verification. The movement speed limit, set to within 0.2 mm / s to prevent vibration caused by sudden rapid platform movement, is set to within 0.1 mm / s² to avoid mechanical shock. The adjustment range of each degree of freedom in the strategy generated by the language model is combined with the platform's kinematic model to estimate the expected velocity and acceleration during execution. If the estimated values ​​exceed the limits, the strategy will be marked as a high-risk strategy. Furthermore, process safety conditions include determining whether the current alignment task allows for repeated fine-tuning and whether the current stage is before glue pre-curing. For example, if the language model suggests a 0.3-degree rotation but the current process prohibits the activation of rotational degrees of freedom, the suggestion will be considered a conflict with the process and will not be executed.

[0048] Only after the structured strategy content output by the language model passes all verification criteria is the platform control device executed to execute the corresponding pose adjustment command, completing a policy-driven active optical alignment operation. The policy verification mechanism ensures a closed-loop safety channel between the language model output and platform actions, preventing slight inference deviations in the language model from being directly applied to actual platform adjustments. Through the pre-verification criteria set in this invention, including pose adjustment range, movement speed limits, acceleration limits, and process safety conditions, strong constraints on the physical execution layer of the policy can be achieved while maintaining policy flexibility, thereby significantly improving the stability, safety, and yield control capabilities of the active alignment process.

[0049] In one embodiment of the present invention, to achieve real-time evaluation of the imaging effect after the platform control device performs pose adjustment, an image scoring value is used as the evaluation basis. This scoring value is calculated by combining the image sharpness value and the positional offset value. The scoring process includes four steps: after the platform control device completes a complete pose adjustment command, it immediately re-acquires the image under the current field of view through the image acquisition unit, and extracts the sharpness value and positional offset value of the current image, which serve as the input basis for the scoring value.

[0050] A scoring function is constructed to convert image sharpness and positional offset values ​​into a quantifiable score. The scoring function employs the following structure: the sharpness value is multiplied by a first weight to obtain an image quality score; the positional offset value is multiplied by a second weight to obtain a positional error score; the two are subtracted to obtain the final score. This scoring structure allows the system to dynamically evaluate the image quality and center offset after alignment, providing a basis for subsequent adjustments or recording in the log table.

[0051] To ensure the scoring function's flexibility and adaptability across different task scenarios, the first and second weights are not set to fixed values ​​but are dynamically set based on the task type. Specifically, a task parameter table is constructed, recording the scoring strategies corresponding to different process stages and task batch numbers. During the execution of the current task, based on the process stage identifier and the current task batch number in the input format, the matching record in the task parameter table is retrieved, and the first and second weights corresponding to the current task are extracted. For example, if the current task is the second batch in the "lens bonding stage," and the corresponding record in the parameter table is: first weight set to 0.8, second weight set to 0.6; then, with a sharpness value of 0.85 and a positional offset value of 0.02, the score is 0.85 × 0.8 − 0.02 × 0.6 = 0.656.

[0052] In one embodiment of this invention, the language model is built upon a lightweight inference model deployed locally on the device. It is primarily used for deep analysis and policy inference of input image metrics and platform pose information to generate structured six-directional pose adjustment suggestions and output a credibility score as a basis for policy feasibility evaluation. This process includes the following steps: In terms of model structure selection, the language model is constructed based on any one of the lightweight pre-trained model structures: MiniLM, DistilBERT, or TinyBERT. All three structures are lightweight models optimized using knowledge distillation techniques. MiniLM compresses multi-head self-attention mechanisms and context representations, preserving high-level semantic features and significantly reducing model parameter size; DistilBERT uses partial network layer removal and pruning operations to achieve volume compression while maintaining contextual understanding capabilities; TinyBERT retains strong semantic inference capabilities with a very small model size through inter-layer alignment and multi-stage distillation. Based on these structures, this invention deploys the language model to an edge computing device with local inference capabilities, enabling real-time processing of input information without relying on network connectivity.

[0053] In terms of model input construction, the language model receives structured input consisting of image metrics and platform pose information. Image metrics include image sharpness and positional offset values, while platform pose information includes the current six degrees of freedom pose parameters: linear displacements in three directions and rotation angles in three directions. This data, after field normalization, is organized into key-value pairs and input into the language model. The model uses a self-attention mechanism to capture the dependencies and implicit relationships between fields and constructs a contextual representation vector. During inference, the language model evaluates the sensitivity of different input fields to adjustments based on internal semantic structure weights to deduce the optimal path required for the current pose adjustment.

[0054] In terms of output policy generation, after reasoning on the structured input, the language model outputs a set of structured policy results containing pose adjustment suggestions in six directions. Alternatively, during the training phase, it can be configured to generate multiple candidate policies, with each policy result consisting of displacement values ​​along three axes and rotation angles around those axes. For example, given an input image with a sharpness value of 0.78 and a positional offset value of 0.05, and current platform pose parameters including a displacement of 0.15 mm in the X direction and a rotation of 0.09 degrees in the Y direction, the model might output the following suggestions: a positive displacement of 0.08 mm in the X direction, a negative displacement of 0.04 mm in the Y direction, unchanged displacement in the Z direction, a clockwise rotation of 0.12 degrees around the X axis, a counterclockwise rotation of 0.07 degrees around the Y axis, and the angle around the Z axis remaining constant. This policy result is organized in a structured key-value format for easy subsequent verification and execution.

[0055] In terms of credibility score generation, to evaluate the degree of matching between the structured strategy results and the current task objective, the language model further scores based on the matching strength. The calculation logic for this score is as follows: First, for each set of candidate adjustment values, the language model uses its internal prediction mechanism to construct an estimate of the image score value after the adjustment. This estimate is calculated by weighting the sharpness value and the positional offset value. To improve the consistency and interpretability of the scoring system, the language model adopts the same scoring calculation logic as the subsequent actual scoring process when scoring the credibility of the candidate adjustment strategies. That is, the predicted image sharpness and positional offset derived from the candidate strategies are used as the basis, and substituted into the function corresponding to the score value to calculate the estimate.

[0056] The preset task target score is set based on statistical values ​​of image states that have met the alignment completion standard in historical tasks. For example, by statistically analyzing the score values ​​of images verified as "alignment complete" or "optimal state" in a large number of historical tasks, the average or median of the distribution is calculated as the target score for different task types. This score is used as a target reference when the language model generates its strategy. The numerical difference between the estimated value and the preset task target score is used as the evaluation benchmark. This difference is normalized and mapped to a continuous value between zero and one. The closer the value is to zero, the closer the adjusted strategy is to the task target. Finally, this normalized difference is converted into the matching strength between the current strategy and the task target. The matching strength value is the credibility score. For example, if the task target score is 0.75, the estimated value is 0.72, and the difference after normalization is 0.03, then the credibility score is 0.97.

[0057] It should be noted that when the language model generates multiple candidate policies (e.g., up to three groups), if the first group of policies fails the validation criteria, the second and third groups will be tried sequentially. Multiple candidate policies can be generated during language model training. If all candidate policies fail to meet the validation criteria, the current input will be deemed unexecutable, the platform adjustment process will not be triggered, and the current input will be marked as an "invalid round" and recorded in the state cache. This state record will be used for subsequent analysis and anomaly detection. A waiting period then begins, during which images are reacquired, and image metrics and pose information are reconstructed. The strategy output by the language model must pass preset verification criteria (such as pose range, velocity limits, acceleration limits, etc.) before it can be executed. If the strategy output does not meet these criteria, it indicates that the current strategy may have potential hardware risks or lead to misadjustment, such as: six-axis platform exceeding its limits; lens or workpiece collision; sudden acceleration causing focus failure. In this case, performing adjustment is dangerous. Therefore, pausing the current process is a safety and fault-tolerant mechanism. Moreover, optical alignment is not a task that must be forcibly adjusted for every frame of the image; rather, it is sufficient to reach a stable target state. Before adjustment, it is necessary to confirm that the direction and amplitude are reasonable. If the model cannot provide reasonable suggestions, it indicates that there may be problems or contextual conflicts in the current input (image, pose, etc.). In this case, pausing the adjustment, waiting for the next image acquisition and reconstructing the input is a safe strategy that conforms to engineering practice. In actual production lines, many automatic alignment systems have strategy output filtering mechanisms to avoid frequent malfunctions, and this invention is no exception.

[0058] In the implementation of the active optical alignment method, an error compensation mechanism with time-series backtracking capability was constructed to achieve continuous stability of optical alignment accuracy and dynamic suppression of error accumulation. This mechanism is triggered when the score value meets a preset stability condition or its continuous variation amplitude is less than a set variation threshold, and is automatically executed when image index changes exceed the tolerance range in subsequent stages of the alignment process. Its execution steps include the following: After the platform control device completes attitude adjustment, the system acquires images and calculates score values. When the fluctuation amplitude of the score value is lower than the set variation threshold in three consecutive acquisitions, or when a single score value reaches the absolute threshold set in the stability condition (e.g., greater than 8.5), the current image state and platform attitude combination is determined to be in a stable state. The variation threshold is set according to the standard deviation distribution of different task types in the historical score fluctuation range and dynamically adjusted in conjunction with the task stage to determine whether the variation amplitude of the score value in multiple consecutive rounds is in a stable state. The absolute threshold is set based on manual alignment experience and equipment performance evaluation results, as a percentage reduction of the maximum score value, used to determine whether a single score value has met the target state requirements. At this point, the image metrics and platform posture are combined and recorded in the feature posture corresponding record table, and the current task type, task batch number and recording time are also recorded to construct a traceable record sample with complete label information.

[0059] In subsequent alignment cycles, the system compares the current image metrics with those before the last image acquisition, calculating the differences in sharpness and positional offset. If this difference exceeds a preset tolerance range—for example, a sharpness change exceeding 0.8 pixels or a positional offset change exceeding 0.5 pixels—error compensation logic is triggered. The tolerance range is set based on statistical results of score fluctuations in historical task samples, taking into account the sensitivity of task type and key structures to alignment accuracy, and establishing a task-level floating tolerance table.

[0060] The error correction logic first calls the most recently recorded stable state sample, extracting the image metrics and platform attitude combination from the record. Then, it calculates the difference between the current image metrics and the recorded metrics, constructing an error vector containing changes in sharpness and positional offset. Further, it calculates the differences in six directions between the current platform attitude and the recorded platform attitude to obtain the attitude drift vector. The system integrates these two vectors into a single overall error vector and performs normalization to adapt to the key-value standard in subsequent input formats.

[0061] The input format to the language model is dynamically supplemented. Specifically, the components of the error vector are superimposed onto the image index field and the platform pose field to adjust the original input values. The updated input format retains the original image acquisition time and task identifier fields to ensure temporal and task continuity after correction. After the update, the input format is fed into the locally running language model to continue the policy generation and credibility scoring process, achieving closed-loop self-correction and stable recovery capabilities under abnormal conditions.

[0062] The above algorithms or formulas are all dimensionless and numerical calculations, and the results are obtained by software simulation based on a large amount of collected data to obtain the most recent real-world results. The preset parameters are set by those skilled in the art according to the actual situation.

[0063] It should be understood that in the various embodiments of this application, the order of the above-mentioned processes does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of this application.

[0064] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0065] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working processes of the devices and units described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here.

[0066] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. An active optical alignment method based on a local lightweight language model, characterized in that, Includes the following steps: The image acquisition unit acquires images under the initial field of view, extracts sharpness and position offset values, and records the current platform's attitude parameters as image indicators and attitude information. Image metrics, attitude information, process stage identifiers, and historical task reference data are combined to construct a unified input format, while retaining key context fields; a verification mechanism is introduced to analyze the logical relationships and content matching between fields and eliminate input combinations that do not meet the input standards. The validated input format is fed into the locally running language model, and the output is a structured strategy containing suggestions for the adjustment direction and magnitude of the six degrees of freedom, as well as a corresponding credibility score. The strategy content output by the language model is verified according to the pre-set verification criteria. The platform control device is activated to perform adjustments only when all criteria are met. After the platform control device completes the pose adjustment, it acquires images again and calculates the score value. The score value is composed of the product of the sharpness value and the first weight, minus the product of the position offset value and the second weight. The first weight and the second weight are dynamically set according to the task type. When the score value meets the preset stability condition or the continuous change range is less than the set change threshold, the image index and platform posture are combined and written into the feature posture corresponding record table. If there are differences in image metrics before and after subsequent calibration that exceed the preset tolerance, error compensation will be performed based on the most recent recorded content, and the input format content will be updated.

2. The active optical alignment method based on a local lightweight language model according to claim 1, characterized in that, Before extracting the sharpness and position offset values, edge enhancement and region segmentation are performed on the image under the initial field of view. Gradient enhancement operators are used in the image processing unit to enhance the image edges. Based on the image's morphological features or grayscale levels, the image is divided into multiple candidate regions. A rectangular region with the image center as the geometric center and a width and height each accounting for 20% of the image size, or the candidate region with the highest score in the previous round of scoring, is selected as the region of interest. The geometric center of the candidate region is defined as the average value of all pixel coordinates within the boundary of the candidate region, i.e., the centroid coordinates.

3. The active optical alignment method based on a local lightweight language model according to claim 2, characterized in that, Extracting the sharpness value and position offset value refers to: Multiple fixed sampling points are preset in the region of interest. The pixel gray level of each sampling point is used as the clarity index of that point. The Euclidean distance between each sampling point and the geometric center of the region of interest is used as a weighting factor. The clarity index of all sampling points is weighted and averaged to obtain the clarity value corresponding to the region of interest. The weighting coefficient of the sampling point is inversely proportional to the Euclidean distance. The spatial coordinate difference between the geometric center of the region of interest and the center of the image field of view is used as the position offset value, where the center of the field of view is the coordinate point corresponding to the midpoint of the image width and height.

4. The active optical alignment method based on a local lightweight language model according to claim 3, characterized in that, The steps to build a unified input format include the following: The image metrics and platform posture information are normalized into structured key-value pairs to form an image state set. Extract the process stage identifier and the current task batch number, and compare them with the recorded image indicators and platform posture information in the historical task reference data to filter out reference samples with a matching degree higher than the preset similarity threshold. The image state set, matching samples, platform posture, and task identifier are encapsulated into a unified input format. The key context fields retained include the current image's sharpness value, position offset value, the platform posture's position information in six directions, and the historical task number and score value.

5. The active optical alignment method based on a local lightweight language model according to claim 4, characterized in that, During the screening process, based on the process stage identifier and the current task batch number, the recorded image indicators and platform attitude information in the historical task reference data are searched and combined into a historical vector. The current image indicators and platform attitude parameters are combined into a current vector. Cosine similarity is used as the matching metric. The current vector and the historical vector are similar to each other, and a preset similarity threshold is used as the judgment standard.

6. The active optical alignment method based on a local lightweight language model according to claim 5, characterized in that, The application logic of the verification mechanism is as follows: The image metrics, platform posture, process stage identifiers, and historical sample score values ​​in the completed input format are converted into structured triples with attribute labels one by one, and a logical dependency matrix between the fields is constructed. The system performs consistency calculations on the logical relationship between any two fields, constructs semantic comparison triples based on the attributes and values ​​of each field, outputs a support score through a preset rule base, and then compares it with a preset logical consistency threshold. Field combinations that are below the logical consistency threshold are marked as invalid and removed.

7. The active optical alignment method based on a local lightweight language model according to claim 6, characterized in that, The pre-defined verification criteria include pose adjustment range, movement speed limit, acceleration limit, and process safety conditions.

8. The active optical alignment method based on a local lightweight language model according to claim 6, characterized in that, The methods for obtaining the first and second weights are as follows: Based on the current task type, retrieve the scoring strategy that matches the process stage identifier and task batch number from the preset task parameter table, and extract the first weight and second weight corresponding to the scoring strategy.

9. The active optical alignment method based on a local lightweight language model according to claim 8, characterized in that, The locally running language model is built based on any of the lightweight pre-trained model structures of MiniLM, DistilBERT, or TinyBERT. It receives structured input consisting of image metrics and platform pose, performs input parsing and policy reasoning in the local device environment, and outputs a structured policy result containing pose adjustment suggestions in six directions, including displacement values ​​along the three axes and rotation angles around the three axes. Based on the matching strength between the structured policy result and the task objective, a corresponding credibility score is generated. The calculation logic for matching strength is as follows: for each set of candidate adjustment values ​​generated by the language model, construct an estimate of the image score value after its effect, and normalize the numerical difference between the estimate and the preset task target score value to obtain the matching strength between the set of candidate adjustment values ​​and the task target, which is then used as the credibility score.

10. The active optical alignment method based on a local lightweight language model according to claim 9, characterized in that, Error recovery refers to the following: when the score value meets the preset stability condition or its continuous change is less than the set change threshold, the image index and platform attitude are recorded in a pair into the characteristic attitude corresponding record table, and the current task type and recording time are marked at the same time to form a traceable historical benchmark. If, during the subsequent alignment process, the difference between the image metrics before and after calibration exceeds the preset tolerance range, the offset between this difference and the most recent benchmark record is calculated to form an error vector. Based on the error vector, the input format content to be input into the language model is dynamically supplemented, and the error vector is superimposed on the current input content of the locally running language model as the standard input structure for subsequent inference and policy generation of the language model.

Images