A real-time visual image matching positioning method for optical communication module

Abstract

The application discloses a real-time visual image matching positioning method for an optical communication module, relates to the field of optical visual positioning, and aims to solve the problem of accurate matching positioning of the module. The application constructs a unified transformation relationship from a multi-source imaging unit image coordinate system to a device mechanical coordinate system, fuses template matching and feature matching technologies, combines a structure model preset area region of interest, and drives adaptive distribution of an imaging mode and collection parameters, performs high-quality feature extraction and redundant supplementary collection based on multi-source images, improves a transparent window in a complex area, introduces a geometric model constraint and a residual error elimination mechanism in a fusion process, realizes high-confidence fusion of features of each area, and finally obtains an overall six-degree-of-freedom attitude through a rigid body registration and pose solving algorithm and outputs key structure coordinates, so that the recognition precision and engineering implementability of the optical communication module in positioning, assembly and detection and other links are improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of optical vision positioning, more particularly, the present application relates to a real-time vision image matching positioning method for optical communication modules. BACKGROUND

[0002] In the production detection and automatic assembly scene of optical communication modules, three-dimensional accurate positioning of the module shell profile, optical window, alignment hole and metal contact array is required to ensure the stable execution of the processes such as plug-in docking, fiber coupling and end face cleaning. Current common systems mostly rely on single vision camera and fixed light source scheme, and estimate the attitude through template matching or simple edge extraction. When there is strong reflection on the transparent window of the module surface, mirror reflection on the metal contact or deep hole structure shielding, the image is prone to regional overexposure, edge breakage or deep hole "collapse into black hole", which makes it impossible to reliably extract key feature points. Only through repeated mechanical test insertion or manual reinspection can the position be corrected, which reduces the cycle and increases the risk of misassembly.

[0003] At the same time, although some production lines introduce profile light, depth camera and other imaging units, they are mostly used in independent stations or manually switched, lacking calibration and result fusion mechanism in the unified device mechanical coordinate system. Multi-source images are only passively recorded, and there is no closed-loop mapping from regional attributes to imaging mode to attitude solution. For optical communication modules of different models and different surface treatment processes, it is difficult to adaptively select imaging combination and acquisition parameters according to material and geometric differences, and it is impossible to actively adjust the image acquisition strategy when early signs of positioning deviation, reflection abnormalities or local shielding appear. It is difficult to meet the comprehensive requirements of positioning accuracy and adaptive ability in high-density port and high-yield assembly scenarios. SUMMARY

[0004] In order to overcome the above-mentioned defects of the prior art, the following scheme is provided to solve the problem of accurate matching positioning of the module in the background art.

[0005] To achieve the above-mentioned purpose, the present application provides the following technical scheme:

[0006] A real-time vision image matching positioning method for optical communication modules, comprising the following steps:

[0007] A global vision camera and at least two types of auxiliary imaging units are configured on an optical measurement platform, and a unified transformation relationship between the pixel coordinate system of each imaging unit and the device mechanical coordinate system is established through a calibration plate image;

[0008] When the optical communication module enters the detection area, the global vision camera acquires a global image and performs preprocessing, template matching or feature matching with a reference image to obtain the coarse attitude of the module in the image coordinate system, and divides the region of interest of the outer contour, transparent window, metal contact and deep hole according to the structure model.

[0009] Based on the grayscale distribution and reflection characteristics of each region of interest, a region attribute map is generated by combining the structural model, and the region image acquisition task is selected from the imaging mode mapping table for each region, which contains different imaging unit combinations and acquisition parameters.

[0010] The optical measurement platform is driven by the regional image acquisition task to adjust its attitude and viewing angle, acquire multi-source images of the same region under a unified coordinate system, perform feature extraction and quality evaluation on the multi-source images, and add supplementary images by calling unused imaging units when low-quality or missing observations occur. The effective observations are fused under regional geometric constraints to obtain the fused coordinates of regional feature points.

[0011] The coordinates of all feature points in the region are fused and registered with the 3D structural model of the optical communication module. The pose solving algorithm is used to calculate the overall position and orientation of the optical communication module in the mechanical coordinate system of the equipment, and the coordinates of the reference plane, alignment hole, optical window and metal contact array are output as the visual image matching and positioning results.

[0012] Furthermore, a global vision camera and at least two types of auxiliary imaging units are configured on the optical measurement platform. A unified transformation relationship between the pixel coordinate system of each imaging unit and the mechanical coordinate system of the equipment is established using calibration board images. Specific steps include:

[0013] Simultaneously acquire calibration images of the calibration board in the global vision camera and each auxiliary imaging unit under different postures;

[0014] Extract the pixel coordinates of calibration feature points from each calibration image and calculate the pose of the calibration board relative to the global vision camera;

[0015] Using the global vision camera as a reference, the pixel coordinates of the calibration feature points of each auxiliary imaging unit are registered with the pose of the calibration board. The spatial transformation parameters from each auxiliary imaging unit to the mechanical coordinate system of the equipment are obtained, and the spatial transformation parameters are recorded as a unified transformation relationship.

[0016] Furthermore, when the optical communication module enters the detection area, the global vision camera acquires a global image and performs preprocessing. It then performs template matching or feature matching with the reference image to obtain the module's coarse pose in the image coordinate system. Based on the structural model, it divides the region of interest into the outline, transparent window, metal contact, and deep hole regions. Specific steps include:

[0017] Based on the 3D structural model of the optical communication module, a reference image from a reference perspective is pre-generated, and structural unit markers are established for the outline, transparent window, metal contact and deep hole.

[0018] The acquired global image is subjected to grayscale normalization, noise suppression, and edge enhancement processing.

[0019] By combining template matching and feature point matching, the global image and the reference image are registered to obtain the coarse orientation of the optical communication module relative to the image coordinate system.

[0020] By projecting the structural unit labels from the reference image onto the current global image, the corresponding outlines, transparent windows, metal contacts, and deep hole regions of interest are obtained.

[0021] Furthermore, based on the grayscale distribution and reflectance characteristics of each region of interest, and combined with the structural model, a region attribute map is generated. This allows for the selection of regions from the imaging mode mapping table that contain different combinations of imaging units and acquisition parameters for image acquisition tasks. Specific steps include:

[0022] For each region of interest, calculate the gray-level histogram, edge gradient distribution, and saturation pixel ratio to obtain the reflectance feature index;

[0023] The reflection feature index is associated with the material type label and geometry type label in the structural model to form a region attribute record containing material attribute field, geometry field and the feature type field to be tested;

[0024] The region attribute records are input into the imaging mode mapping table to retrieve the corresponding imaging unit combination and acquisition parameters, generate a region image acquisition task, and assign different imaging unit combinations to transparent window regions, metal contact regions and deep hole regions.

[0025] Furthermore, the optical measurement platform is driven to adjust its attitude and viewing angle according to the regional image acquisition task, and multi-source images of the same region are acquired in a unified coordinate system. The specific steps include:

[0026] The optical measurement platform is controlled to move to the target orientation based on the viewpoint and position parameters in the regional image acquisition task.

[0027] The global vision camera and auxiliary imaging unit are triggered sequentially to acquire local images, contour images, or depth images of the target area;

[0028] The pixel coordinates of feature points in various images are transformed to the equipment mechanical coordinate system using a unified transformation relationship, and edges, corners, contours or depth extrema are extracted as multi-source feature point observations on each type of image.

[0029] Furthermore, feature extraction and quality assessment are performed on the multi-source images. When low-quality or missing observations are found, unused imaging units are used to acquire supplementary images. The specific steps include:

[0030] For each region, the contrast, contour continuity, depth jump count, saturation pixel ratio, and repeated observation position deviation are calculated from the observations of multi-source feature points to form quality indicators.

[0031] Compare the quality indicators with a preset threshold, and mark feature points whose quality is below the threshold or whose observation is missing.

[0032] For the marked area, select unused imaging unit combinations according to the imaging mode mapping table to re-acquire local images, and add the supplemented feature point observations to the multi-source observation set of the marked area.

[0033] Furthermore, the effective observations are fused under regional geometric constraints to obtain the fused coordinates of regional feature points. Specific steps include:

[0034] Select a straight line model, circular arc model, circular hole model, or planar model as the region's geometric constraints based on the geometric shape field in the region attribute map.

[0035] The observations marked as valid multi-source feature points are substituted into the corresponding geometric constraints, and the geometric model parameters are solved by minimizing the error.

[0036] The spatial positions of the region's feature points in the equipment's mechanical coordinate system are calculated using the obtained geometric model parameters, and these positions are used as the fused coordinates of the region's feature points.

[0037] Furthermore, the coordinates of all feature points in the region are fused and registered with the 3D structural model of the optical communication module. Specific steps include:

[0038] Collect the fused coordinates of feature points in each region, and associate each feature point with the corresponding structural unit in the 3D structural model based on the region attribute map;

[0039] Using the coarse pose of the optical communication module in the image coordinate system as the initial pose, a one-to-one correspondence is established between the fused coordinates of the feature points and the three-dimensional reference points of the corresponding structural units.

[0040] By minimizing the spatial distance error between the feature point fusion coordinates and the 3D reference point in the equipment's mechanical coordinate system, the rigid body transformation parameters used for registration are obtained.

[0041] Furthermore, the pose calculation algorithm is used to calculate the overall position and orientation of the optical communication module in the equipment's mechanical coordinate system. Specific steps include:

[0042] The rigid body transformation parameters are decomposed into translational and rotational components to represent the six-degree-of-freedom pose of the optical communication module relative to the device's mechanical coordinate system.

[0043] Based on the initial rigid body transformation parameters, the translation and rotation components are updated using an iterative optimization method to gradually reduce the residual distance of all feature points in the equipment's mechanical coordinate system.

[0044] During the iteration process, abnormal feature points with residuals exceeding a preset threshold are removed, and the remaining feature points are used to recalculate the overall position and orientation until convergence.

[0045] Furthermore, the coordinates of the reference plane, alignment hole, optical window, and metal contact array are output as the visual image matching and positioning results. Specific steps include:

[0046] Based on the obtained overall position and orientation, the reference surface point set in the three-dimensional structural model is transformed into the equipment mechanical coordinate system. The geometric parameters of the reference surface are obtained through plane fitting, and the reference surface coordinates are output.

[0047] The coordinate transformation of the structural unit reference points corresponding to the alignment hole and optical window is performed according to the overall position and orientation, and the coordinates of the center of the alignment hole and the center of the optical window are output.

[0048] The corresponding reference points of the metal contact array are extracted according to the contact index order. After coordinate transformation, the spatial coordinates of each contact in the mechanical coordinate system of the equipment are obtained, and the coordinate set of the metal contact array is formed according to the index order.

[0049] The technical effects and advantages of the real-time visual image matching and localization method for optical communication modules of the present invention are as follows:

[0050] This invention constructs a region-adaptive visual measurement process based on multiple imaging units to achieve real-time image matching and precise positioning of the outer contour of the optical communication module, transparent window, metal contact point and deep hole in a unified equipment mechanical coordinate system. Using a three-dimensional structural model as a priori, the global image is divided into regions of interest and a region attribute map is generated. The material type, geometric shape and the features to be measured are mapped to the imaging mode and acquisition parameters. The global vision camera, structured light contour imaging unit, depth imaging unit and confocal reflection imaging unit are called as needed to effectively cover the measurement blind spots of transparent high reflective surfaces, metal strong reflective surfaces and deep cavity obstruction parts.

[0051] By employing a multi-source image quality evaluation and re-acquisition mechanism, and combining linear, circular hole, and planar geometric constraints to fuse and solve feature points, the stability and anti-interference capability of local features are improved. Then, by utilizing the registration calculation module of the fused feature points and three-dimensional reference points, the overall pose is calculated, and the spatial coordinates of the reference plane, alignment hole, optical window, and metal contact array are uniformly output. This provides high-confidence positioning results for automatic insertion and removal, precision assembly, and online dimensional inspection, enhancing the versatility and measurement reliability of the optical communication module detection line under complex surface and batch variation conditions. Attached Figure Description

[0052] Figure 1 This is a flowchart illustrating a real-time visual image matching and positioning method for an optical communication module according to the present invention. Detailed Implementation

[0053] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0054] In order to achieve the above objectives, Figure 1 A schematic diagram of a real-time visual image matching and localization method for an optical communication module according to the present invention is provided, which specifically includes the following steps;

[0055] Configure a global vision camera and at least two types of auxiliary imaging units on an optical measurement platform, and establish a unified transformation relationship between the pixel coordinate system of each imaging unit and the mechanical coordinate system of the equipment through the calibration plate image;

[0056] When the optical communication module enters the detection area, the global vision camera acquires a global image and performs preprocessing. It then performs template matching or feature matching with the reference image to obtain the coarse pose of the module in the image coordinate system. The region of interest is divided into the outline, transparent window, metal contact and deep hole according to the structural model.

[0057] Based on the grayscale distribution and reflection characteristics of each region of interest, a region attribute map is generated by combining the structural model, and the region image acquisition task is selected from the imaging mode mapping table for each region, which contains different imaging unit combinations and acquisition parameters.

[0058] The optical measurement platform is driven by the regional image acquisition task to adjust its attitude and viewing angle, acquire multi-source images of the same region under a unified coordinate system, perform feature extraction and quality evaluation on the multi-source images, and add supplementary images by calling unused imaging units when low-quality or missing observations occur. The effective observations are fused under regional geometric constraints to obtain the fused coordinates of regional feature points.

[0059] The coordinates of all feature points in the region are fused and registered with the 3D structural model of the optical communication module. The pose solving algorithm is used to calculate the overall position and orientation of the optical communication module in the mechanical coordinate system of the equipment, and the coordinates of the reference plane, alignment hole, optical window and metal contact array are output as the visual image matching and positioning results.

[0060] Step 1: Configure a global vision camera and at least two types of auxiliary imaging units on the optical measurement platform. Establish a unified transformation relationship between the pixel coordinate system of each imaging unit and the mechanical coordinate system of the equipment using the calibration board image. Specifically, this is implemented as follows:

[0061] The optical measurement platform is equipped with a global vision camera fixed above the platform, and two or more types of auxiliary imaging units. The auxiliary imaging units may include structured light contour imaging units, depth imaging units, and confocal reflection imaging units. Each type of auxiliary imaging unit is installed around the optical measurement platform via an independent bracket. The optical measurement platform is equipped with a mechanical coordinate system. The origin of the mechanical coordinate system is associated with the mechanical reference of the platform working surface through a mechanical reference block, positioning pin, or positioning hole, and the directions of the three coordinate axes are defined. One coordinate axis is along the length of the platform, one coordinate axis is along the width of the platform, and the remaining coordinate axis is perpendicular to the platform working surface. A calibration plate is fixedly installed on the platform working surface. Multiple calibration feature points are printed or engraved on the surface of the calibration plate. The spatial position of each calibration feature point in the coordinate system of the calibration plate itself is given in advance through high-precision machining or high-precision measurement. The calibration plate is fixed to the mechanical coordinate system of the equipment through positioning pins, so that the spatial coordinates of each calibration feature point in the mechanical coordinate system of the equipment can be predetermined through mechanical design and measurement.

[0062] During the calibration process, the calibration board is driven by the optical measurement platform or by the tooling that mounts the calibration board, so that the calibration board is in multiple different postures in the mechanical coordinate system of the equipment. Different postures include different positions and translation states as well as different tilt angles. Under each posture, the calibration images of the calibration board in the global vision camera and each auxiliary imaging unit are acquired simultaneously using the synchronous triggering method.

[0063] For each calibration image, the position of calibration feature points in the image is detected using image processing algorithms. Detection methods include corner detection, center detection, or center point detection of the calibration board pattern to obtain the pixel coordinates of each calibration feature point in the image. The pixel coordinates are recorded in the form of row and column numbers, where the row number represents the integer index from the top boundary of the image to the row position of the pixel, and the column number represents the integer index from the left boundary of the image to the column position of the pixel. For a global vision camera calibration image in a certain pose, the pixel coordinates of all calibration feature points in the image are paired with the spatial coordinates of the corresponding calibration feature points in the device's mechanical coordinate system. The control unit executes the solution process, adjusting a set of pose parameters and position parameters iteratively to minimize the sum of the distance errors between the projected position of the calibration feature point on the global vision camera's imaging plane after parameter transformation and the corresponding pixel coordinates. The pose parameters represent the rotation state, including three rotation angle components, and the position parameters represent the translation state, including three translation components along the three coordinate axes of the device's mechanical coordinate system. Through this process, the pose of the calibration board relative to the global vision camera in the current pose can be obtained.

[0064] By repeating the above process for multiple different poses, a set of pose data of the calibration board relative to the global vision camera under different poses can be obtained, which can be used to improve the stability and accuracy of pose solution.

[0065] After obtaining the pose of the calibration board relative to the global vision camera, the global vision camera is used as the reference imaging unit. Registration is performed on each type of auxiliary imaging unit. For a given auxiliary imaging unit, the pixel coordinates of calibration feature points are extracted from the calibration image acquired by that unit and paired with the spatial coordinates of the calibration board in the equipment's mechanical coordinate system. The control unit establishes constraints between the pixel coordinates of the auxiliary imaging unit, the spatial coordinates of the calibration board, and the coordinates of the equipment's mechanical coordinate system based on the calibration board pose calculated by the global vision camera under the same orientation. By adjusting the spatial transformation parameters from the auxiliary imaging unit to the equipment's mechanical coordinate system, the total distance error between the spatial position of the calibration feature points derived from the pixel coordinates in the calibration image of the auxiliary imaging unit and the spatial coordinates of the calibration board in the equipment's mechanical coordinate system is minimized. These spatial transformation parameters include three rotational angle components and three translational components. The above solution steps are repeated for calibration images acquired by the auxiliary imaging unit under multiple different orientations. All orientation data are used together to constrain the spatial transformation parameters from the auxiliary imaging unit to the equipment's mechanical coordinate system, ensuring the uniformity and stability of these spatial transformation parameters within the platform's workspace.

[0066] The same registration and solution process is performed on each type of auxiliary imaging unit to obtain the set of spatial transformation parameters of the global vision camera and each auxiliary imaging unit relative to the device's mechanical coordinate system.

[0067] The spatial transformation parameters from each imaging unit to the device's mechanical coordinate system are stored in the parameter storage area of ​​the control unit in a unified structure. Each set of spatial transformation parameters includes three rotational angular components representing the imaging unit's mounting attitude and three translational components representing the imaging unit's mounting position. The control unit defines this set of parameters as a unified transformation relationship.

[0068] In subsequent testing, when any imaging unit acquires an image of the optical communication module, the pixel coordinates of any feature point in the image can be converted into spatial coordinates in the mechanical coordinate system of the device through a unified transformation relationship. This achieves a consistent spatial mapping relationship between the pixel coordinate system of the global vision camera and each auxiliary imaging unit and the mechanical coordinate system of the device within the entire optical measurement platform.

[0069] Step 2: When the optical communication module enters the detection area, the global vision camera acquires a global image and performs preprocessing. Template matching or feature matching is then performed with the reference image to obtain the module's coarse pose in the image coordinate system. The region of interest (ROI) is then divided according to the structural model, including the outline, transparent window, metal contact, and deep hole. Specifically, the implementation is as follows:

[0070] First, a reference image from a reference perspective is pre-generated based on the three-dimensional structural model of the optical communication module, and structural unit markings are established for the outline, transparent window, metal contact and deep hole. In the three-dimensional structural model, each geometric unit has a clear three-dimensional spatial position and geometric contour information in the model coordinate system. During the design stage, it can be defined by three-dimensional modeling software, and the outline unit, transparent window unit, metal contact array unit and deep hole unit are marked in the model respectively.

[0071] Based on the installation posture of the global vision camera on the optical measurement platform, multiple reference viewpoints that are consistent with or approximately consistent with the camera's viewpoint are selected. The three-dimensional structural model of the optical communication module is virtually imaged under these reference viewpoints to obtain a set of reference images containing geometric contours and texture information. When generating reference images, the projection contour of each structural unit on the imaging plane is marked with a polygonal or rectangular region, and four types of structural units are assigned labels: outline, transparent window, metal contact, and deep hole, forming a reference image library with structural unit labels.

[0072] When the optical communication module enters the detection area through the conveyor, the global vision camera on the optical measurement platform acquires a frame of global image after the module reaches the preset trigger position. The global image is represented by an image coordinate system, which uses pixel row number and pixel column number as coordinate components. In order to improve the stability of subsequent matching, the acquired global image is subjected to grayscale normalization, noise suppression and edge enhancement processing.

[0073] The grayscale normalization process maps the minimum grayscale value in the original image to a preset lower limit grayscale level and the maximum grayscale value to a preset upper limit grayscale level through linear stretching. Interpolation is performed on intermediate grayscale values ​​according to a ratio, thereby reducing the brightness differences between different batches of optical communication modules and under different lighting conditions. The noise suppression process performs smoothing filtering on the global image. The size of the filtering window is selected according to the typical feature size of the module in the image, so that random noise is weakened while structural edges are preserved. The edge enhancement process enhances the contrast of areas with large grayscale changes, further amplifying the grayscale gradient along the outline, transparent window boundary, metal contact edge, and deep hole contour, so that subsequent feature point detection and contour matching are more stable.

[0074] After completing the global image preprocessing, the control unit selects a set of reference images from the reference image library that corresponds to the current workstation geometry. Using a combination of template matching and feature point matching, the global image and the reference image are registered to obtain the coarse orientation of the optical communication module relative to the image coordinate system. The template matching part uses the overall grayscale distribution of the outer contour region and transparent window region in the reference image as the template. It searches in the global image according to the preset translation step size and rotation angle step size. For each candidate translation position and rotation angle combination, the similarity score between the template region and the corresponding region in the global image is calculated. The similarity score can be characterized by comparing the sum of squares of pixel grayscale differences or correlation indicators. The combination with the best score is selected as the coarse position and rotation angle estimate.

[0075] The feature point matching part extracts local feature points such as corners, sharp corners and intersections in the reference image and the global image respectively, and establishes feature description vectors according to the gray-scale neighborhood feature description method. By measuring the degree of difference between feature description vectors, the feature points in the reference image are paired with the feature points in the global image.

[0076] The control unit integrates the coarse position obtained by template matching and the point-to-point correspondence obtained by feature point matching, and adjusts the translation and rotation components of the optical communication module in the image coordinate system through iterative method. This gradually reduces the distance and direction errors between the projection position of the structural unit contour in the reference image and the corresponding edge position in the global image. When the error converges to a preset threshold range, the translation and rotation components at this time are taken as the coarse orientation of the optical communication module in the image coordinate system.

[0077] After obtaining the coarse pose of the optical communication module, the structural unit markers already established in the reference image are projected from the reference image onto the current global image to obtain the corresponding outline, transparent window, metal contact, and deep hole region of interest. Specifically, for each structural unit, the vertex set of that structural unit is obtained from the 3D structural model of the optical communication module. The vertex set is then converted into vertex positions in the image coordinate system using the viewpoint projection relationship corresponding to the coarse pose. Connecting these vertices forms the 2D contour region of that structural unit under the current viewpoint. The contour region is expanded or contracted by a certain number of pixels as a safety margin to obtain the region boundary containing all effective pixels of that structural unit.

[0078] For the outer contour structure unit, an outer contour region of interest is formed that surrounds the entire outline of the optical communication module; for the transparent window structure unit, a transparent window region of interest is formed for extracting optical window features; for the metal contact structure unit, a metal contact region of interest is formed that covers the metal contact array; for the deep hole structure unit, a deep hole region of interest is formed that covers the deep hole opening outline. The control unit records each region of interest as a set of regions described by the four vertices of a polygon or a rectangle in the image coordinate system, providing spatial range constraints for subsequently calling different imaging units for different regions of interest and performing multi-source image feature extraction.

[0079] Step 3: Based on the grayscale distribution and reflectance characteristics of each region of interest, and combined with the structural model, generate a region attribute map. Then, select regions from the imaging mode mapping table that contain different combinations of imaging units and acquisition parameters for image acquisition tasks. The specific implementation is as follows:

[0080] Based on each region of interest obtained, the control unit first extracts the corresponding sub-image in the global image according to the boundary range of the region of interest, and calculates the gray-level histogram, edge gradient distribution and saturation pixel ratio based on the sub-image to form the reflection feature index.

[0081] The grayscale histogram is calculated as follows:

[0082] The gray level of each pixel in the sub-image is counted, and the number of pixels is accumulated into the corresponding gray level interval according to the preset gray level interval. For example, the gray level range is divided into several consecutive gray level intervals, and the number of pixels contained in each interval is recorded. At the same time, the proportion of pixels in the low gray level interval, the proportion of pixels in the near-middle gray level interval, and the proportion of pixels in the high gray level interval are calculated to characterize the overall brightness distribution characteristics of the region.

[0083] The edge gradient distribution is calculated as follows:

[0084] In the sub-image, a local gray-level change is constructed for each pixel based on the gray-level difference in the horizontal and vertical directions. The gradient magnitude of the pixel is obtained based on the local gray-level change, and the number of pixels in different gradient magnitude intervals and the distribution ratio of high gradient pixels in the region are counted to characterize the richness of the region edge and the clarity of the boundary.

[0085] The saturation pixel ratio is calculated as follows:

[0086] The number of pixels in the sub-image whose gray level is close to the upper gray level threshold and the number of pixels whose gray level is close to the lower gray level threshold are counted. The sum of these two pixel counts is then used as the ratio of the total number of pixels in the sub-image. This ratio is used as the saturated pixel ratio of the region of interest, which is used to characterize the high reflectance saturation and low reflectance underexposure conditions in the region.

[0087] The control unit records the feature values ​​of the grayscale histogram, the feature values ​​of the edge gradient distribution, and the proportion of saturated pixels as reflection feature indicators. The grayscale histogram feature values ​​may include the average grayscale representing the overall brightness level, the grayscale variance representing the contrast, and the proportion of high grayscale intervals representing the proportion of bright areas. The edge gradient distribution feature values ​​may include the proportion of high gradient pixels representing the number of edges and the gradient mean representing the texture intensity.

[0088] After obtaining the reflection feature index of each region of interest, the control unit associates the reflection feature index with the material type label and geometry type label in the 3D structural model of the optical communication module to form a region attribute record containing material attribute field, geometry field and the feature type field to be measured.

[0089] The material type labels are derived from the material definitions of each structural unit in the 3D structural model of the optical communication module. For example, the shell area is labeled as metal material, the transparent window area is labeled as glass or transparent plastic material, the deep hole area is labeled as metal guide hole or plastic guide hole, and the substrate area is labeled as plastic material.

[0090] The control unit assigns the corresponding material definition to the material attribute field in the attribute record of the region of interest based on the structural unit marker in the structural model. The geometric type label also comes from the definition of geometric shape in the 3D structural model. For example, the outer contour region is the overall outer contour planar region, the transparent window region is the rectangular opening region or the circular opening region, the metal contact region is the regular array region, and the deep hole region is the circular hole or the elliptical hole opening region. The control unit assigns geometric shapes such as planar region, rectangular opening, circular opening, and hole structure to the geometric shape field accordingly.

[0091] The "Feature Type to be Measured" field describes the type of features that need to be extracted from this region. For example, the feature types to be measured in the outer contour region are the outer contour and the reference edge position; the feature types to be measured in the transparent window region are the window boundary position and the window center position; the feature types to be measured in the metal contact region are the contact center position and the contact array row and column structure; and the feature types to be measured in the deep hole region are the hole center position.

[0092] The control unit combines the material attribute field, geometric shape field, the type of feature to be measured field, and the reflection feature index into a region attribute record, and assigns a unique region identifier to each region of interest. All region attribute records are organized into a region attribute map according to the region identifier. The region attribute map is stored in the control unit in the form of a data table for subsequent imaging mode selection and image acquisition task configuration.

[0093] After generating the region attribute map, the control unit inputs each region attribute record into the imaging mode mapping table, retrieves the corresponding imaging unit combination and acquisition parameters, generates a region image acquisition task, and assigns different imaging unit combinations to the transparent window region, metal contact region and deep hole region.

[0094] The imaging mode mapping table is pre-established during the system debugging phase and contains multiple mapping entries. Each mapping entry contains a set of condition fields and corresponding imaging unit combinations and acquisition parameters. The condition fields include the value range of the material attribute field, the value range of the geometric shape field, and the value range of the feature type field to be measured. The imaging unit combination field records the type of imaging unit to be called, such as global vision camera, structured light contour imaging unit, depth imaging unit, and confocal reflection imaging unit. The acquisition parameter field records the exposure time, light source brightness, light source incident angle, image resolution, scanning path, or scanning step distance for each imaging unit.

[0095] When processing attribute records for transparent window areas, since the material attribute field is transparent material, the geometry field is the window opening area, and the target feature type field is the window boundary position and window center position, the control unit retrieves the configured imaging unit combination from the imaging mode mapping table, including a global vision camera and a structured light contour imaging unit or a global vision camera and a confocal reflection imaging unit, and selects exposure time and light source brightness parameters suitable for suppressing strong reflections and improving boundary contrast. When processing attribute records for metal contact areas, the material attribute field is metal, the geometry field is a regular array area, and the target feature type field is the contact center position. The imaging unit combination retrieved by the control unit includes a global vision camera and a structured light contour imaging unit. The light source incident angle in the acquisition parameters is configured as oblique incident to reduce specular reflection, and the exposure time is set to prevent overexposure of the contact area.

[0096] When processing the attribute records of deep borehole regions, the material attribute field is the borehole material, the geometric morphology field is the borehole structure, and the target feature type field is the borehole center position. The imaging unit combination retrieved from the imaging mode mapping table by the control unit includes depth imaging units and confocal reflection imaging units. The exposure time and scanning step distance in the acquisition parameters are set to capture the bottom reflection and borehole wall contour. The control unit assembles the region identifier, imaging unit combination, and corresponding acquisition parameters into a region image acquisition task list. Each task clearly indicates the target region of interest, the type of imaging unit to be activated, and the corresponding acquisition parameters, providing parameter basis for subsequent multi-source image acquisition and feature extraction by region.

[0097] Step 4: Drive the optical measurement platform to adjust its attitude and viewing angle according to the regional image acquisition task, acquire multi-source images of the same region under a unified coordinate system, perform feature extraction and quality evaluation on the multi-source images, and add supplementary images by calling unused imaging units when low-quality or missing observations are found. Then, fuse the valid observations under the regional geometric constraints to obtain the fused coordinates of regional feature points. The specific implementation is as follows:

[0098] After the optical measurement platform generates the list of regional image acquisition tasks, the control unit reads the regional identifier, viewing angle parameters and position parameters in the regional image acquisition tasks one by one, and drives the motion axis of the optical measurement platform to move the optical communication module to the target observation posture of the corresponding region.

[0099] Based on the viewing angle and position parameters recorded in the regional image acquisition task, the control unit controls the linear motion axis and rotational motion axis of the platform to sequentially complete position and attitude adjustments, so that the target region of interest is located at the center of the effective field of view of the global vision camera and the corresponding auxiliary imaging unit. After reaching the target attitude, the control unit triggers the global vision camera and each auxiliary imaging unit to complete image acquisition in the order of the imaging unit combination in the regional image acquisition task. Among them, the global vision camera acquires the local image of the target region, the structured light contour imaging unit acquires the contour image of the target region, the depth imaging unit acquires the depth image of the target region, and the confocal reflection imaging unit acquires the reflection intensity distribution image of the target region.

[0100] After the data acquisition is completed, the control unit calls the aforementioned unified transformation relationship to convert the pixel coordinates of the feature points extracted from various images into spatial coordinates in the equipment's mechanical coordinate system. It extracts the region's edges and corners on the local image, extracts the feature points on the contour line on the contour image, extracts the depth extreme points on the depth image, and extracts the reflection intensity extreme points on the confocal reflection image. It then stores the spatial coordinates of these feature points, which are represented in the equipment's mechanical coordinate system, as a multi-source feature point observation set for the region and attaches an image type label to each observation record.

[0101] For each region's multi-source feature point observation set, the control unit sequentially calculates contrast, contour continuity, depth jump count, saturation pixel ratio, and repeated observation position deviation according to the preset quality evaluation process, and forms quality indicators accordingly. Contrast is used to reflect the brightness contrast of the image in the region. Specifically, it is to calculate the average gray level of all pixels in the local image of the target region and the difference degree representing the degree of gray level dispersion. The difference degree can be characterized by calculating the average absolute value of the difference between each pixel gray level and the average gray level of the region. The difference degree is compared with the contrast threshold obtained in advance based on sample data during the system debugging phase. If the difference degree is lower than the threshold, the contrast is considered insufficient.

[0102] Contour continuity is used to reflect whether the edges or contour lines are complete. By tracking the pixel connectivity of the contour lines in the contour image, the actual continuous contour length is compared with the expected contour length of the region in the structural model. When the continuous contour length is significantly shorter than the expected length, it is judged that the contour is broken. The depth jump count is used to reflect whether there are abnormal depth changes in the depth image. By comparing the depth difference of adjacent points on the sampling path of the depth image, when the depth difference exceeds the preset depth change threshold, it is counted once. All count results are used as the depth jump count. If the depth jump count is too large, it indicates that the depth data is unstable.

[0103] The saturation pixel ratio is used to reflect the bright saturation or underexposure in a region. It is calculated by counting the number of pixels close to the upper and lower grayscale limits in a local image and then comparing this number with the total number of pixels in the region. When the saturation pixel ratio exceeds a preset saturation threshold, the region is considered to have a serious saturation problem. The repeated observation position deviation is used to reflect the spatial position difference of the same physical feature observed under different imaging units. It is calculated by calculating the difference in the position of the same feature point observed from different image types in the equipment's mechanical coordinate system. The spatial distance of the difference is compared with the repeated observation deviation threshold. When the distance exceeds the deviation threshold, the repeated observation is considered inconsistent.

[0104] The control unit assigns a quality label to each multi-source feature point observation based on the comparison results of the above quality indicators and their respective thresholds. If at least one quality indicator of an observation fails to meet the requirements or no effective feature point is detected in the region under a certain imaging unit, the observation or the observation corresponding to the imaging unit is marked as a low-quality observation or a missing observation.

[0105] For areas marked as low-quality observations or missing observations, the control unit consults the imaging mode mapping table again based on the material attribute field, geometric shape field, and target feature type field in the area attribute map, selects the imaging unit combination that has not been used in this round of acquisition and the matching acquisition parameters for the area, and regenerates the area image acquisition task for supplementary acquisition.

[0106] During the supplementary acquisition process, the optical measurement platform fine-tunes the observation posture of the target area again based on the viewpoint and position parameters in the supplementary task, triggering the newly added auxiliary imaging unit to acquire local images, contour images, or depth images of the area. The control unit uses the same feature extraction method as described above to convert the pixel coordinates of feature points in the supplementary acquisition images into spatial coordinates in the equipment's mechanical coordinate system through a unified transformation relationship, and adds these supplementary feature point observations to the multi-source observation set of the area. To prevent infinite repeated acquisition, the control unit can set a maximum number of supplementary acquisitions for each area. If some feature point observations still fail to meet the quality requirements after reaching the maximum number of supplementary acquisitions, the area will be marked as a low-reliability area in the subsequent pose solution, serving only as an auxiliary constraint and not as the main solution basis. This repeated evaluation and supplementary acquisition mechanism improves the effective observation coverage of transparent window areas, metal contact areas, and deep hole areas under multi-source imaging conditions.

[0107] After completing the quality evaluation and supplementary acquisition of the multi-source feature point observation set, the control unit performs geometrically constrained fusion on the feature point set marked as valid observation in each region of interest to obtain the fused coordinates of the regional feature points.

[0108] The control unit first determines the geometric model type corresponding to the current region based on the geometric shape field in the region attribute map. When the geometric shape field indicates that the region is an outer edge, a straight line model or a circular arc model is selected as the geometric constraint. When the geometric shape field indicates that the region is a deep hole or an aligned hole, a circular hole model is selected as the geometric constraint. When the geometric shape field indicates that the region is a reference plane or a window plane, a planar model is selected as the geometric constraint.

[0109] Subsequently, the control unit sequentially substitutes the spatial coordinates of the multi-source feature point observations marked as valid into the corresponding geometric model. By adjusting the geometric model parameters, the sum of the spatial distances from all valid feature points to the geometric model is minimized. This process is performed iteratively. In each iteration, the distance from each feature point to the geometric model is calculated based on the current geometric model parameters. These distances are summed according to preset weights, and the geometric model parameters are adjusted to gradually reduce the summed value until the change in the summed distance value is less than a predetermined convergence threshold. The geometric model parameters include the direction vector and the point of passage in the linear model, the center position and radius in the arc model or circular hole model, and the normal vector and the position of a point on the plane in the planar model.

[0110] After the geometric model parameters converge, the control unit calculates the spatial position of the corresponding regional feature points in the equipment's mechanical coordinate system based on the geometric model parameters. For example, for a circular hole model, it outputs the center position of the circular hole; for a straight line model, it outputs the intersection of the baseline and the specified reference plane; and for a planar model, it outputs the intersection of the plane and the reference coordinate system of the optical communication module or the geometric center position. Finally, the spatial position is written as the fused coordinate of the regional feature points in the region into the fusion result field of the region attribute map, and provides a high-confidence local feature input for the overall pose solution.

[0111] It should be noted that the depth change threshold, saturation threshold, and repeated observation deviation threshold were all determined during the system debugging phase by statistically analyzing multiple sets of collected data from qualified optical communication module samples, based on the allowable measurement error range and module structural tolerance range, and the parameters can be adjusted for different module models.

[0112] Step 5: Register the fused coordinates of all feature points in the region with the 3D structural model of the optical communication module. Use a pose solving algorithm to calculate the overall position and orientation of the optical communication module in the equipment's mechanical coordinate system, and output the coordinates of the reference plane, alignment hole, optical window, and metal contact array as the visual image matching and positioning results. The specific implementation is as follows:

[0113] After generating the fused coordinates of regional feature points in each region of interest, the control unit first collects the fused coordinates of all regional feature points and associates the fused coordinates of each regional feature point with the corresponding structural unit in the 3D structural model of the optical communication module according to the regional attribute map.

[0114] The region attribute map has recorded the region identifier and corresponding structural unit marker for each region of interest, including the outer contour structural unit, transparent window structural unit, metal contact structural unit, and deep hole structural unit. In the three-dimensional structural model, one or more three-dimensional reference points are predefined for each structural unit. The spatial position of each three-dimensional reference point in the coordinate system of the three-dimensional structural model is determined by design dimensions or high-precision measurement.

[0115] The control unit matches the fused coordinates of each region's feature points with the corresponding structural unit markers in the region attribute map according to the region identifier. Then, it reads the corresponding 3D reference points from the 3D structural model and pairs the fused coordinates of the region's feature points with these 3D reference points, forming a one-to-one correspondence set of feature points. When establishing this correspondence, the fused coordinates of the region's feature points for the outer contour region mainly correspond to the 3D reference points at the outer contour corners and reference edges; the fused coordinates of the region's feature points for the transparent window region correspond to the 3D reference points at the window boundary and window center; the fused coordinates of the region's feature points for the metal contact region correspond to the 3D reference point at the contact center; and the fused coordinates of the region's feature points for the deep hole region correspond to the 3D reference point at the hole center, ensuring that the geometric features of different types of structural units are fully utilized during the registration process.

[0116] The steps for registering the fused coordinates of regional feature points with the 3D reference points in the equipment's mechanical coordinate system are as follows:

[0117] The control unit uses the coarse attitude of the optical communication module in the image coordinate system as the initial pose estimate, and converts the coarse attitude into a set of initial rigid body transformation parameters. The coarse attitude includes the translational position of the module in the image plane and the rotation angles around the axes of the image coordinate system. The control unit combines this coarse attitude with the aforementioned unified transformation relationship to obtain the initial rigid body transformation parameters from the 3D structural model coordinate system to the equipment mechanical coordinate system. The rigid body transformation parameters include three translational components and three rotational components. The three translational components represent the positional offsets in the three coordinate axes of the equipment mechanical coordinate system, and the three rotational components represent the rotation angles around the three coordinate axes.

[0118] Based on the initial rigid body transformation parameters, the control unit maps the three-dimensional reference points in the three-dimensional structural model to the equipment mechanical coordinate system through the current rigid body transformation parameters, thereby obtaining the mapped position of each three-dimensional reference point in the equipment mechanical coordinate system.

[0119] Meanwhile, the fused coordinates of regional feature points are already represented in the equipment's mechanical coordinate system, so they can be directly compared with the mapped 3D reference point positions. The control unit calculates the spatial distance between each pair of fused coordinates of regional feature points and the corresponding mapped 3D reference point positions, and sums all distance differences by squared differences to obtain a total distance error metric. This total distance error metric is used to evaluate the overall effect of the current rigid body transformation parameters on the registration of all regional feature point fused coordinates with the 3D reference points. Subsequently, with the goal of reducing the total distance error metric, the control unit updates the rigid body transformation parameters by gradually adjusting the values ​​of translation and rotation components. After each adjustment, the spatial distance of all corresponding point pairs is recalculated and the total distance error metric is updated. When the change in the total distance error metric is less than the preset convergence threshold in multiple consecutive iterations, the rigid body transformation parameters are considered to be solved. The rigid body transformation parameters at this point are the rigid body transformation parameters used for registration. The convergence threshold is determined during the system debugging phase by measuring multiple sets of samples, comprehensively considering the maximum allowable registration error and measurement accuracy requirements.

[0120] After obtaining the rigid body transformation parameters for registration, the control unit decomposes the rigid body transformation parameters into translational and rotational components to represent the six-degree-of-freedom pose of the optical communication module relative to the device's mechanical coordinate system. The three translational components correspond to the coordinate values ​​of the module's reference coordinate origin in the three directions of the device's mechanical coordinate system, and the three rotational components correspond to the rotation angles of the module's reference coordinate system around the three coordinate axes of the device's mechanical coordinate system.

[0121] The control unit performs secondary optimization based on the aforementioned rigid body transformation parameters using an iterative optimization method to further improve the accuracy of the overall position and attitude. In each iteration, the current translation and rotation components are used to map all three-dimensional reference points in the three-dimensional structural model to the mechanical coordinate system of the equipment. The spatial distance between the mapped position and the fused coordinates of the corresponding regional feature points is calculated, and these spatial distances are recorded as residual distances. The control unit performs statistics on all residual distances and compares the residual distances with the residual thresholds determined based on the sample experimental data during the system debugging phase. Point pairs with residual distances significantly greater than the residual thresholds are marked as abnormal feature points. These abnormal feature points may come from areas with local occlusion, insufficient multi-source observation quality, or unmet regional geometric constraints.

[0122] For point pairs marked as anomalous feature points in the current iteration, the control unit temporarily removes them in subsequent iterations, uses only the remaining feature point pairs to construct a new total distance error metric, and makes small step adjustments to the translation and rotation components to gradually reduce the total distance error metric. Through multiple iterations, when the change in the total distance error metric is less than the set convergence threshold and the residual distance distribution meets the preset accuracy requirements, the translation and rotation components at this time are solidified as the overall position and attitude of the optical communication module in the equipment's mechanical coordinate system, thus completing the pose solving process.

[0123] After solving for the overall position and attitude, the control unit performs coordinate transformation on the reference plane point set used to construct the reference plane in the three-dimensional structural model of the optical communication module, based on the obtained overall position and attitude. The reference plane point set is a set of representative points located on the module reference plane selected in the three-dimensional structural model during the design phase. The spatial coordinates of these points in the three-dimensional structural model coordinate system are obtained through the module structure design.

[0124] The control unit transforms each point in the reference surface point set using the translation and rotation components corresponding to the overall position and attitude, obtaining the spatial coordinates of that point in the equipment's mechanical coordinate system. Then, based on the spatial positions of all reference surface points in the equipment's mechanical coordinate system, a plane fitting method is used to determine a plane that best represents the distribution of these points. This plane is represented by a plane normal direction and the coordinates of a point on the plane. The control unit records the geometric parameters of this plane in the equipment's mechanical coordinate system as the reference surface coordinate output. Based on this, the same coordinate transformation is performed on the reference points of the structural units corresponding to the alignment holes and optical windows in the 3D structural model, transforming these reference points from the 3D structural model coordinate system to the equipment's mechanical coordinate system. The spatial positions of the transformed reference points in the equipment's mechanical coordinate system are used to calculate the center coordinates of the alignment holes and the center coordinates of the optical windows. The center coordinates of the alignment holes are determined by comprehensively considering the spatial positions of the reference points of the hole structure units, and the center coordinates of the optical windows are determined by comprehensively considering the spatial positions of the representative points of the window boundary.

[0125] For the metal contact array, the control unit extracts the structural unit reference point corresponding to each metal contact according to the contact index order defined in the three-dimensional structural model, and then transforms these reference points into the equipment mechanical coordinate system through the translation and rotation components corresponding to the overall position and attitude, to obtain the metal contact spatial coordinate sequence in array form. Then, according to the contact index order, these spatial coordinates are combined into a metal contact array coordinate set.

[0126] Finally, the control unit outputs the coordinates of the reference plane, the center coordinates of the alignment hole, the center coordinates of the optical window, and the coordinates of the metal contact array as the visual image matching and positioning result, which can be used in applications such as assembly positioning, insertion and removal path planning, or size detection.

[0127] This invention constructs a region-adaptive visual measurement process based on multiple imaging units to achieve real-time image matching and precise positioning of the outer contour of the optical communication module, transparent window, metal contact point and deep hole in a unified equipment mechanical coordinate system. Using a three-dimensional structural model as a priori, the global image is divided into regions of interest and a region attribute map is generated. The material type, geometric shape and the features to be measured are mapped to the imaging mode and acquisition parameters. The global vision camera, structured light contour imaging unit, depth imaging unit and confocal reflection imaging unit are called as needed to effectively cover the measurement blind spots of transparent high reflective surfaces, metal strong reflective surfaces and deep cavity obstruction parts.

[0128] By employing a multi-source image quality evaluation and re-acquisition mechanism, and combining linear, circular hole, and planar geometric constraints to fuse and solve feature points, the stability and anti-interference capability of local features are improved. Then, by utilizing the registration calculation module of the fused feature points and three-dimensional reference points, the overall pose is calculated, and the spatial coordinates of the reference plane, alignment hole, optical window, and metal contact array are uniformly output. This provides high-confidence positioning results for automatic insertion and removal, precision assembly, and online dimensional inspection, enhancing the versatility and measurement reliability of the optical communication module detection line under complex surface and batch variation conditions.

[0129] The above embodiments can be implemented, in whole or in part, by software, hardware, firmware, or any other combination thereof. When implemented using software, the above embodiments can be implemented, in whole or in part, in the form of a computer program product.

[0130] Those skilled in the art will recognize that the modules and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0131] In addition, the functional modules in the various embodiments of this application can be integrated into one processing module, or each module can exist physically separately, or two or more modules can be integrated into one module.

[0132] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

[0133] In conclusion, the above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A real-time visual image matching and localization method for optical communication modules, characterized in that: The specific steps include: Configure a global vision camera and at least two types of auxiliary imaging units on an optical measurement platform, and establish a unified transformation relationship between the pixel coordinate system of each imaging unit and the mechanical coordinate system of the equipment through the calibration plate image; When the optical communication module enters the detection area, the global vision camera acquires a global image and performs preprocessing. It then performs template matching or feature matching with the reference image to obtain the coarse pose of the module in the image coordinate system. The region of interest is divided into the outline, transparent window, metal contact and deep hole according to the structural model. Based on the grayscale distribution and reflection characteristics of each region of interest, a region attribute map is generated by combining the structural model, and the region image acquisition task is selected from the imaging mode mapping table for each region, which contains different imaging unit combinations and acquisition parameters. The optical measurement platform is driven by the regional image acquisition task to adjust its attitude and viewing angle, acquire multi-source images of the same region under a unified coordinate system, perform feature extraction and quality evaluation on the multi-source images, and add supplementary images by calling unused imaging units when low-quality or missing observations occur. The effective observations are fused under regional geometric constraints to obtain the fused coordinates of regional feature points. The coordinates of all feature points in the region are fused and registered with the 3D structural model of the optical communication module. The pose solving algorithm is used to calculate the overall position and orientation of the optical communication module in the mechanical coordinate system of the equipment, and the coordinates of the reference plane, alignment hole, optical window and metal contact array are output as the visual image matching and positioning results. Based on the grayscale distribution and reflectance characteristics of each region of interest, and combined with the structural model, a region attribute map is generated. This process involves selecting regions from the imaging mode mapping table that contain different combinations of imaging units and acquisition parameters for image acquisition tasks. Specific steps include: For each region of interest, calculate the gray-level histogram, edge gradient distribution, and saturation pixel ratio to obtain the reflectance feature index; The reflection feature index is associated with the material type label and geometry type label in the structural model to form a region attribute record containing material attribute field, geometry field and the feature type field to be tested; The region attribute records are input into the imaging mode mapping table, the corresponding imaging unit combination and acquisition parameters are retrieved, the region image acquisition task is generated, and different imaging unit combinations are assigned to the transparent window region, the metal contact region and the deep hole region. The effective observations are fused under regional geometric constraints to obtain the fused coordinates of regional feature points. The specific steps include: Select a straight line model, circular arc model, circular hole model, or planar model as the region's geometric constraints based on the geometric shape field in the region attribute map. The observations marked as valid multi-source feature points are substituted into the corresponding geometric constraints, and the geometric model parameters are solved by minimizing the error. The spatial positions of the region's feature points in the equipment's mechanical coordinate system are calculated using the obtained geometric model parameters, and these positions are used as the fused coordinates of the region's feature points.

2. The real-time visual image matching and positioning method for an optical communication module according to claim 1, characterized in that: Configure a global vision camera and at least two types of auxiliary imaging units on an optical measurement platform. Establish a unified transformation relationship between the pixel coordinate system of each imaging unit and the mechanical coordinate system of the equipment using calibration board images. Specific steps include: Simultaneously acquire calibration images of the calibration board in the global vision camera and each auxiliary imaging unit under different postures; Extract the pixel coordinates of calibration feature points from each calibration image and calculate the pose of the calibration board relative to the global vision camera; Using the global vision camera as a reference, the pixel coordinates of the calibration feature points of each auxiliary imaging unit are registered with the pose of the calibration board. The spatial transformation parameters from each auxiliary imaging unit to the mechanical coordinate system of the equipment are obtained, and the spatial transformation parameters are recorded as a unified transformation relationship.

3. The real-time visual image matching and positioning method for an optical communication module according to claim 2, characterized in that: When the optical communication module enters the detection area, the global vision camera acquires a global image and performs preprocessing. It then performs template matching or feature matching with the reference image to obtain the module's coarse pose in the image coordinate system. Based on the structural model, it divides the region of interest into areas such as the outline, transparent window, metal contact, and deep hole. Specific steps include: Based on the 3D structural model of the optical communication module, a reference image from a reference perspective is pre-generated, and structural unit markers are established for the outline, transparent window, metal contact and deep hole. The acquired global image is subjected to grayscale normalization, noise suppression, and edge enhancement processing. By combining template matching and feature point matching, the global image and the reference image are registered to obtain the coarse orientation of the optical communication module relative to the image coordinate system. By projecting the structural unit labels from the reference image onto the current global image, the corresponding outlines, transparent windows, metal contacts, and deep hole regions of interest are obtained.

4. The real-time visual image matching and positioning method for an optical communication module according to claim 3, characterized in that: The optical measurement platform is driven to adjust its attitude and viewing angle according to the regional image acquisition task, and multi-source images of the same region are acquired under a unified coordinate system. The specific steps include: The optical measurement platform is controlled to move to the target orientation based on the viewpoint and position parameters in the regional image acquisition task. The global vision camera and auxiliary imaging unit are triggered sequentially to acquire local images, contour images, or depth images of the target area; The pixel coordinates of feature points in various images are transformed to the equipment mechanical coordinate system using a unified transformation relationship, and edges, corners, contours or depth extrema are extracted as multi-source feature point observations on each type of image.

5. A real-time visual image matching and positioning method for an optical communication module according to claim 4, characterized in that: Feature extraction and quality assessment are performed on multi-source images. When low-quality or missing observations are found, unused imaging units are used to acquire supplementary images. The specific steps include: For each region, the contrast, contour continuity, depth jump count, saturation pixel ratio, and repeated observation position deviation are calculated from the observations of multi-source feature points to form quality indicators. Compare the quality indicators with a preset threshold, and mark feature points whose quality is below the threshold or whose observation is missing. For the marked area, select unused imaging unit combinations according to the imaging mode mapping table to re-acquire local images, and add the supplemented feature point observations to the multi-source observation set of the marked area.

6. The real-time visual image matching and positioning method for an optical communication module according to claim 5, characterized in that: The coordinates of all feature points in the region are fused and registered with the 3D structural model of the optical communication module. The specific steps include: Collect the fused coordinates of feature points in each region, and associate each feature point with the corresponding structural unit in the 3D structural model based on the region attribute map; Using the coarse pose of the optical communication module in the image coordinate system as the initial pose, a one-to-one correspondence is established between the fused coordinates of the feature points and the three-dimensional reference points of the corresponding structural units. By minimizing the spatial distance error between the feature point fusion coordinates and the 3D reference point in the equipment's mechanical coordinate system, the rigid body transformation parameters used for registration are obtained.

7. A real-time visual image matching and positioning method for an optical communication module according to claim 6, characterized in that: The pose calculation algorithm is used to calculate the overall position and orientation of the optical communication module in the equipment's mechanical coordinate system. The specific steps include: The rigid body transformation parameters are decomposed into translational and rotational components to represent the six-degree-of-freedom pose of the optical communication module relative to the device's mechanical coordinate system. Based on the initial rigid body transformation parameters, the translation and rotation components are updated using an iterative optimization method to gradually reduce the residual distance of all feature points in the equipment's mechanical coordinate system. During the iteration process, abnormal feature points with residuals exceeding a preset threshold are removed, and the remaining feature points are used to recalculate the overall position and orientation until convergence.

8. The real-time visual image matching and positioning method for an optical communication module according to claim 7, characterized in that: The coordinates of the reference plane, alignment hole, optical window, and metal contact array are output as the visual image matching and positioning results. Specific steps include: Based on the obtained overall position and orientation, the reference surface point set in the three-dimensional structural model is transformed into the equipment mechanical coordinate system. The geometric parameters of the reference surface are obtained through plane fitting, and the reference surface coordinates are output. The coordinate transformation of the structural unit reference points corresponding to the alignment hole and optical window is performed according to the overall position and orientation, and the coordinates of the center of the alignment hole and the center of the optical window are output. The corresponding reference points of the metal contact array are extracted according to the contact index order. After coordinate transformation, the spatial coordinates of each contact in the mechanical coordinate system of the equipment are obtained, and the coordinate set of the metal contact array is formed according to the index order.

Images