Multi-directional laser positioning detection method and system based on machine vision

Abstract

The application provides a multi-directional laser positioning detection method and system based on machine vision, and belongs to the technical field of industrial automation processing. The method comprises the following steps: determining the pose of a workpiece and an optical characteristic distribution map based on initial image information; generating an initial projection scheme according to the pose of the workpiece, a preset target point position and the optical characteristic distribution map; controlling a laser emitting device to project a positioning laser beam onto the surface of the workpiece according to the initial projection scheme, and monitoring the dynamic change of the positioning light spot in the projection process by using a second vision sensor; based on the dynamic change, performing a coupling regulation process of a first closed-loop regulation and a second closed-loop regulation in parallel to obtain a target projection angle and target laser projection parameters; verifying the light spot by using a first vision sensor or a third vision sensor, and storing the target projection angle, the target laser projection parameters and the identification information of the workpiece in a database after verification.

Description

Technical Field

[0001] This application relates to the field of industrial automation processing technology, and in particular to a multi-directional laser positioning and detection method and system based on machine vision. Background Technology

[0002] With the rapid development of the high-end intelligent manufacturing industry, the laser positioning technology commonly used in current industrial mass production scenarios relies solely on a single vision device to acquire images of the workpiece and the laser spot. The laser projection angle is adjusted based on the spot's positional deviation, and the entire process uses fixed laser parameters. However, under ultra-high precision positioning requirements, individual differences and regional inhomogeneities in the optical properties of the workpiece surface can cause abnormal spot conditions, reducing coordinate recognition accuracy and resulting in continuous fluctuations in positioning accuracy and unstable batch yields. This makes it difficult to meet the stringent requirements of high-end precision manufacturing.

[0003] Therefore, there is an urgent need for a multi-directional laser positioning and detection method and system based on machine vision. Summary of the Invention

[0004] To address the aforementioned technical problems, this application provides a multi-directional laser positioning and detection method and system based on machine vision.

[0005] A first aspect of this application provides a multi-directional laser positioning and detection method based on machine vision, comprising: Based on the initial image information, the pose of the workpiece in the current equipment coordinate system and the optical characteristic distribution map of the workpiece surface are determined. The initial image information is obtained by acquiring the workpiece based on the first vision sensor. An initial projection scheme is generated based on the workpiece's pose, the preset target point, and the optical characteristic distribution map; the initial projection scheme includes the projection angle of the laser emitting device pointing towards the workpiece surface and the initial laser projection parameters; The laser emitting device is controlled to project a positioning laser beam onto the workpiece surface according to the initial projection scheme, and the dynamic changes of the positioning spot during the projection process are monitored by a second vision sensor. Based on the dynamic changes, a coupled control process of first closed-loop control and second closed-loop control is executed in parallel; wherein the first closed-loop control corrects the projection angle of the laser emitting device pointing to the workpiece surface according to the deviation between the actual position of the light spot and the theoretical position; the second closed-loop control adjusts the initial laser projection parameters according to the comparison between the quality index of the light spot and the preset quality threshold. Repeat the first and second closed-loop control as described above until the position deviation and quality index of the light spot both meet the preset requirements, and obtain the target projection angle and target laser projection parameters. The light spot that meets the preset requirements is verified using the first vision sensor or the third vision sensor. When the verification is qualified, the target projection angle, target laser projection parameters, environmental features and workpiece identification information are associated and stored in the database.

[0006] A second aspect of this application provides a multi-directional laser positioning and detection system based on machine vision, comprising: The image analysis module is used to determine the pose of the workpiece in the current equipment coordinate system and the optical characteristic distribution map of the workpiece surface based on the initial image information, wherein the initial image information is obtained by acquiring the workpiece based on the first vision sensor. The scheme generation module is used to generate an initial projection scheme based on the workpiece's pose, preset target points, and the optical characteristic distribution map; the initial projection scheme includes the projection angle of the laser emitting device pointing to the workpiece surface and initial laser projection parameters. The first control module is used to control the laser emitting device to project a positioning laser beam onto the workpiece surface according to the initial projection scheme, and to use the second vision sensor to monitor the dynamic changes of the positioning spot during the projection process. The second control module is used to perform a coupled control process of first closed-loop control and second closed-loop control in parallel based on the dynamic changes; wherein the first closed-loop control corrects the projection angle of the laser emitting device pointing to the workpiece surface according to the deviation between the actual position of the light spot and the theoretical position; the second closed-loop control adjusts the initial laser projection parameters according to the comparison between the quality index of the light spot and the preset quality threshold. Repeat the first and second closed-loop control as described above until the position deviation and quality index of the light spot both meet the preset requirements, and obtain the target projection angle and target laser projection parameters. A spot verification module is used to verify the spot that meets the preset requirements using the first vision sensor or the third vision sensor. The data storage module is used to associate and store the target projection angle, target laser projection parameters, environmental characteristics, and workpiece identification information in the database after the verification is qualified.

[0007] A third aspect of this application provides an electronic device, including a memory, a processor, and a computer program stored in the memory and running on the processor, wherein the processor executes the computer program to implement the steps of the above-described machine vision-based multi-directional laser positioning and detection method.

[0008] A fourth aspect of this application provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the steps of the above-described multi-directional laser positioning and detection method based on machine vision.

[0009] The beneficial effects of the multi-directional laser positioning and detection method and system based on machine vision provided in this application are as follows: Firstly, by employing a dual closed-loop coupled control of projection angle correction and laser parameter adjustment, this application ensures the accuracy of the laser spot position while adapting in real-time to the differences and dynamic changes in the optical properties of the workpiece surface. This fundamentally solves the problem of coordinate recognition accuracy deviation caused by abnormal laser spot states, effectively suppressing continuous fluctuations in positioning results and improving the accuracy of stable positioning. Secondly, by using multiple vision sensors to perform the tasks of workpiece pose and optical characteristic acquisition, laser spot dynamic monitoring, and positioning result verification, signal interference from single-device acquisition is avoided, significantly improving the detection reliability of the entire positioning process. Thirdly, by associating and storing the optimal positioning parameters, environmental features, and workpiece identification to form a reusable database, this application improves the efficiency of workpiece operations in the same batch, ensures the consistency of positioning accuracy across the entire batch, and meets the stringent production requirements of high-end precision manufacturing. Attached Figure Description

[0010] Figure 1 A flowchart illustrating a machine vision-based multi-directional laser positioning and detection method provided in an embodiment of this application; Figure 2 This is a structural block diagram of a machine vision-based multi-directional laser positioning and detection system provided in an embodiment of this application; Figure 3 This is a schematic block diagram of an electronic device provided in an embodiment of this application. Detailed Implementation

[0011] In the following description, specific details such as particular system architectures and techniques are set forth for illustrative purposes and not for limitation, in order to provide a thorough understanding of the embodiments of this application. However, those skilled in the art will understand that this application may also be implemented in other embodiments without these specific details. In other instances, detailed descriptions of well-known systems, apparatuses, circuits, and methods have been omitted so as not to obscure the description of this application with unnecessary detail.

[0012] To make the purpose, technical solution, and advantages of this application clearer, the following will be described in conjunction with the appendix. Figure 1-3 The following is an explanation using specific examples.

[0013] Please refer to Figure 1 , Figure 1 This is a flowchart illustrating a machine vision-based multi-directional laser positioning and detection method according to an embodiment of this application. The method includes: S101: Based on the initial image information, determine the pose of the workpiece in the current equipment coordinate system and the optical characteristic distribution map of the workpiece surface. The initial image information is obtained by acquiring the workpiece based on the first vision sensor.

[0014] In this embodiment, the initial image information refers to the raw image data acquired by a vision sensor to describe the surface features and spatial position of the workpiece. It can include various forms such as grayscale images, color images, or depth images. The workpiece pose refers to the spatial position and orientation of the workpiece in a specific coordinate system (e.g., the equipment coordinate system), described by six degrees of freedom parameters (three translations and three rotations). The optical property distribution map refers to two-dimensional or three-dimensional mapping data characterizing the optical response characteristics of the workpiece surface in different regions to laser reflection, absorption, and scattering. It can include information such as specular reflection index, diffuse reflection index, and absorptivity index.

[0015] The first vision sensor is used to acquire initial image information of the workpiece, and can be an industrial camera, structured light sensor, or laser scanner, etc. The first vision sensor acquires a two-dimensional image of the workpiece, such as a frontal image of the workpiece taken by a monocular camera. The workpiece pose is determined by manually measuring the positions of several feature points of the workpiece in the equipment coordinate system, and then obtaining the pose through simple geometric calculations or table lookups. Alternatively, the workpiece is placed in a pre-set fixture or positioning device, thereby directly acquiring its pose information. The determination of the optical property distribution map of the workpiece surface is based on the inspection of the workpiece surface, marking the optical property distribution areas according to the optical properties of different regions, for example, dividing the surface into high reflectivity areas, diffuse reflectivity areas, and absorption areas.

[0016] S102: Generate an initial projection scheme based on the workpiece's pose, preset target points, and optical characteristic distribution map; the initial projection scheme includes the expected projection angle of the laser emitting device pointing to the workpiece surface and the initial laser projection parameters.

[0017] In this embodiment, the initial projection scheme refers to the set of laser projection parameters initially determined at the start of laser positioning and detection based on the workpiece pose, target point position, and optical characteristic distribution map, including the projection angle of the laser emitting device pointing to the workpiece surface and the initial laser projection parameters.

[0018] Specifically, based on the workpiece's pose and the preset target point, the target projection point on the workpiece surface to be projected by the laser is calculated through simple coordinate transformation. The projection angle in the initial projection scheme can be determined through geometric calculation based on the position of the target projection point. The initial laser projection parameters are selected from a preset set of default values ​​based on the workpiece's material type and optical property distribution diagram.

[0019] S103: Control the laser emitting device to project a positioning laser beam onto the workpiece surface according to the initial projection scheme, and use a second vision sensor to monitor the dynamic changes of the positioning spot during the projection process.

[0020] In this embodiment, the laser emitting device is used to generate and project a positioning laser beam, and may include a laser, a beam shaping module, a scanning galvanometer, or a mechanical adjustment mechanism. The second vision sensor is a vision device used to monitor the dynamic changes of the positioning spot on the workpiece surface in real time, and may be a high-speed camera or a photodetector array. The positioning spot refers to the visible light point or spot image formed after the laser beam is projected onto the workpiece surface, and its position, shape, brightness, and energy distribution are all important detection indicators.

[0021] Specifically, the laser emitting device can be configured to emit laser light onto the workpiece surface according to the angle and parameters determined in the initial projection scheme. For example, the projection angle can be adjusted by driving a scanning galvanometer with a stepper motor, and the laser power can be adjusted by a power controller. The second vision sensor can continuously acquire image sequences of the light spot on the workpiece surface at a fixed frame rate. These image sequences are used for subsequent analysis of changes in the position, size, or brightness of the light spot.

[0022] S104: Based on dynamic changes, the coupled control process of the first closed-loop control and the second closed-loop control is executed in parallel; wherein the first closed-loop control corrects the projection angle of the laser emitting device pointing to the workpiece surface according to the deviation between the actual position of the light spot and the theoretical position; the second closed-loop control adjusts the initial laser projection parameters according to the comparison between the quality index of the light spot and the preset quality threshold; the above first closed-loop control and second closed-loop control are repeated until the position deviation of the light spot and the quality index of the light spot meet the preset requirements, and the target projection angle and the target laser projection parameters are obtained.

[0023] In this embodiment, the first closed-loop control refers to the control process of real-time correction of the projection angle of the laser emitting device based on the deviation between the actual position and the theoretical position of the positioning spot. The second closed-loop control refers to the control process of real-time adjustment of laser projection parameters (such as laser power, pulse width, spot size, etc.) based on the comparison result of the quality index of the positioning spot with the preset quality threshold. The coupled control process refers to the control process in which the first and second closed-loop controls influence each other and need to be optimized in parallel through coordination or decoupling mechanisms.

[0024] Spot quality indicators are parameters used to quantify and locate the quality of a light spot. These can include the spot's circularity, symmetry, energy concentration, signal-to-noise ratio, or peak intensity. Preset quality thresholds refer to the acceptable range or standard values ​​set for the spot quality indicators, used to determine whether the spot quality meets process requirements.

[0025] The target projection angle refers to the final laser projection angle after closed-loop control, ensuring that the position and quality of the positioning spot meet preset requirements. The target laser projection parameters refer to the final laser projection parameters after closed-loop control, ensuring that the position and quality of the positioning spot meet preset requirements.

[0026] Specifically, within each control cycle, the first and second closed-loop controls can be performed independently. In the first closed-loop control, the second visual sensor acquires the actual position of the light spot, compares it with the theoretical position, and calculates the position deviation. Based on this deviation, the angle correction is calculated, and the laser emitting device is driven to adjust the projection angle. Simultaneously, in the second closed-loop control, the second visual sensor acquires the quality index of the light spot and compares it with a preset quality threshold. If the quality is not up to standard, the laser projection parameters are adjusted using a preset lookup table or a simple linear function, such as increasing or decreasing the laser power. These two control processes can operate independently or simultaneously.

[0027] After each cycle, the deviation between the actual position and the theoretical position of the light spot is checked to see if it is less than the preset position tolerance, and whether the quality index of the light spot reaches the preset quality threshold. When both conditions are met, the control process terminates, and the projection angle and laser projection parameters at this time are determined as the target projection angle and target laser projection parameters.

[0028] S105: Verify the light spot that meets the preset requirements using a first vision sensor or a third vision sensor.

[0029] In this embodiment, the third visual sensor is a visual device used to independently verify the final located light spot. It may be the same as or different from the first visual sensor to provide additional verification capabilities. After the adjustment process is completed, the light spot image is acquired again using either the first or third visual sensor. The light spot's qualification is determined by calculating the distance between the center of the light spot and the target point, or by judging whether the light spot is clearly visible.

[0030] S106: Once the verification is successful, the target projection angle, target laser projection parameters, environmental characteristics, and workpiece identification information are associated and stored in the database.

[0031] In this embodiment, environmental characteristics refer to environmental factors that affect the laser positioning and detection process, including ambient light intensity, ambient temperature, ambient humidity, or dust concentration in the air. Workpiece identification information is a unique code or label used to identify the workpiece, including serial number, batch number, or model number. A database is an information system used to store and manage laser positioning and detection related data, which may include historical projection parameters, environmental characteristics, workpiece identification information, or process templates.

[0032] Specifically, once the light spot verification is successful, the currently determined target projection angle, target laser projection parameters, current environmental characteristics (e.g., temperature values ​​obtained through manual input or simple sensors), and workpiece identification information (e.g., obtained by scanning barcodes) are packaged and stored in a local file or a simple database.

[0033] As can be seen from the above, this application first acquires an initial image of the workpiece using a first vision sensor, calculates the workpiece pose and surface optical property distribution map, and lays the spatial reference and optical adaptation foundation for positioning; then, it generates an initial projection scheme including projection angle and laser parameters by combining the target point position, monitors the dynamic changes of the light spot in real time through a second vision sensor, executes dual closed-loop coupled control in parallel, and synchronously corrects the projection angle and laser parameters to solve the mutual interference problem of single-loop independent control, and quickly achieves dual compliance of light spot position and quality; finally, it locks the positioning accuracy through the verification process, associates and stores core parameters to achieve data reuse, and comprehensively improves the accuracy, convergence efficiency and scene robustness of laser positioning.

[0034] This application achieves parallel coupled control of the position and quality of the positioning spot by introducing the determination of the optical property distribution map of the workpiece surface into laser positioning and detection, combined with collaborative monitoring by multiple vision sensors. This effectively overcomes the problem of abnormal spot caused by differences in the optical properties of the workpiece surface and regional inhomogeneity, improves coordinate recognition accuracy, stabilizes the positioning process, and thus increases the yield of batch operations, meeting the stringent requirements of high-end precision manufacturing for ultra-high precision positioning.

[0035] In one embodiment of this application, determining the workpiece's pose in the current device coordinate system and the optical property distribution map of the workpiece surface based on initial image information includes: The initial image information is processed by multi-view 3D reconstruction to obtain the 3D point cloud data of the workpiece; The three-dimensional point cloud data is iteratively registered with the preset three-dimensional model of the workpiece to calculate the six-degree-of-freedom pose parameters of the workpiece in the current equipment coordinate system. Based on the initial image information and 3D point cloud data, optical property analysis is performed on the workpiece surface to generate an optical property distribution map. Based on the geometric features of optical property distribution maps and 3D point cloud data, optical interference areas are identified and calibrated. The identification information of the optical interference area is fused into the optical characteristic distribution map to obtain the final optical characteristic distribution map.

[0036] In this embodiment, multi-view 3D reconstruction processing is performed on the initial image information to recover the accurate 3D geometric information of the workpiece from 2D images acquired from different perspectives, thereby obtaining the 3D point cloud data of the workpiece. This can be achieved in various ways, for example, by using an algorithm based on motion recovery structure combined with multi-view stereo vision to estimate the camera pose and sparse 3D points from multiple images, and then using multi-view stereo vision to generate dense point cloud data.

[0037] Subsequently, the 3D point cloud data is iteratively registered with the pre-set 3D workpiece model to calculate the six-degree-of-freedom pose parameters of the workpiece in the current equipment coordinate system. The purpose of this step is to align the actually acquired 3D point cloud data of the workpiece with the CAD model from the design phase, thereby determining the position and orientation of the workpiece in the equipment space. Specifically, an iterative nearest-point algorithm is used. This algorithm iteratively optimizes the transformation matrix by repeatedly finding the nearest point correspondence between the point cloud and the model and minimizing the distance between them until convergence.

[0038] Based on this, optical property analysis is performed on the workpiece surface using initial image information and 3D point cloud data to generate an optical property distribution map. This step aims to quantify the optical response characteristics of each point on the workpiece surface, providing a basis for subsequent laser projection. Specifically, by combining pixel brightness information from the initial image and surface normal information provided by the 3D point cloud, the bidirectional reflection distribution function parameters of each point on the workpiece surface, such as specular reflection coefficient, diffuse reflection coefficient, and absorptivity, are estimated by analyzing the intensity and direction of reflected light under known illumination conditions. These parameters are then mapped onto the 3D surface of the workpiece to form an optical property distribution map.

[0039] Furthermore, based on the geometric features of the optical property distribution map and 3D point cloud data, optical interference regions are identified and calibrated. This step aims to discover areas on the workpiece surface that cause abnormal laser spot or inaccurate positioning. For example, it can analyze whether there are areas with high specular reflection, low absorptivity, or abrupt changes in optical properties in the optical property distribution map, while combining this with geometric features extracted from the 3D point cloud data, such as areas with excessive local curvature, abnormal surface roughness, or minor defects. By comprehensively judging these optical and geometric features, potential optical interference regions can be identified. Another approach is to use a pre-trained machine learning model, taking optical property parameters and geometric features as input, to predict the probability that each surface point belongs to an optical interference region.

[0040] Finally, the identification information of the optical interference areas is integrated into the optical characteristic distribution map to obtain the final optical characteristic distribution map. The purpose of this step is to clearly mark the identified interference areas on the optical characteristic distribution map so that the subsequent projection scheme generation process can fully consider the influence of these areas. This is achieved by adding specific identifiers, confidence values, or color codes to points or surface elements within the interference areas, thereby visually displaying in the final optical characteristic distribution map which areas are optical interference areas and their interference type or degree.

[0041] Through the aforementioned technical solution, this application ensures the accuracy of workpiece pose parameters via multi-view 3D reconstruction and iterative registration, providing an accurate spatial reference for subsequent laser projection. Simultaneously, by combining initial image information and 3D point cloud data for optical property analysis, the individual differences and regional inhomogeneities of the workpiece surface optical properties can be meticulously characterized. More importantly, by comprehensively utilizing the geometric features of the optical property distribution map and 3D point cloud data to identify optical interference areas and integrating their identification information into the final optical property distribution map, these potential interference factors can be avoided or compensated for in advance when generating the initial projection scheme. This improves the accuracy and robustness of the initial projection scheme and effectively enhances the precision and stability of laser positioning.

[0042] In one embodiment of this application, optical interference regions are identified and calibrated based on the geometric features of optical property distribution maps and three-dimensional point cloud data, including: Extract the specular reflection index, diffuse reflection index, and absorptivity index from the optical property distribution map; Based on 3D point cloud data, the local curvature and surface roughness of each sampling point are calculated, and a surface feature vector is constructed. The specular reflection index, diffuse reflection index, absorptivity index and surface feature vector are fused and input into a pre-trained optical interference recognition model to obtain the confidence probability value of each sampling point belonging to the optical interference region. The confidence probability value is determined based on the preset confidence threshold, the initial interference point set is selected, and the discrete initial interference points are merged and the edges are smoothed using a region growing algorithm to generate an optical interference region.

[0043] In this embodiment, when calculating the local curvature and surface roughness of each sampling point based on 3D point cloud data and constructing a surface feature vector, the calculation of local curvature can be achieved by fitting a local quadratic surface to the point cloud data and extracting the principal curvature, Gaussian curvature, or average curvature from the fitting parameters, or by estimating it by analyzing the rate of change of the normal vectors of neighboring points. Surface roughness can be quantified by calculating the root mean square deviation of the local point cloud on the fitting plane or surface, or characterized by statistically analyzing the height changes of neighboring points. The calculated local curvature values ​​(such as maximum principal curvature and minimum principal curvature) and surface roughness values ​​are combined into a multidimensional vector, namely the surface feature vector.

[0044] In this embodiment, the optical index and surface feature vector are reduced in dimensionality using principal component analysis and then concatenated. This concatenation is then input into a pre-trained optical interference recognition model. The pre-trained model employs a gradient boosting tree machine learning classifier, trained on a large amount of labeled data to learn the complex mapping relationship between optical properties and geometric features and optical interference regions. The confidence probability value output by the model represents the probability of belonging to a certain category. For example, for a binary classification problem (whether it is an interference region), the model will output a value between 0 and 1, indicating whether the point belongs to the interference region.

[0045] Specifically, the optical interference recognition model is constructed using a deep neural network, including: an input layer (6-dimensional, representing specular reflection index, diffuse reflection index, absorptivity index, local curvature, surface roughness, and rate of change of curvature); two hidden layers (128 neurons in the first layer and 64 neurons in the second layer); and an output layer (1 neuron, sigmoid), outputting a confidence probability value (0-1). The training dataset for the optical interference recognition model includes sampling points of workpieces made of different materials (metal, plastic, composite materials). Each sample consists of input features (spectral reflection index, diffuse reflection index, absorptivity index, local curvature, and surface roughness) and manually labeled information (whether it is an interference region). The training dataset covers the entire range from highly polished mirror surfaces to rough diffuse surfaces. The optical interference recognition model is trained using an early stopping method. The dataset is divided into training, validation, and test sets at 70%, 15%, and 15% respectively. The validation set loss is monitored using a binary cross-entropy loss function, the Adam optimizer, a learning rate of 0.001, a batch size of 32, and 100 training epochs. Training stops if the loss does not decrease for 10 consecutive epochs to prevent overfitting. On the test set, the model's accuracy should reach above 95%, and the recall should be no less than 90%. The optimal classification threshold (0.6-0.8) is determined using ROC curves, and hyperparameters are adjusted using grid search. The input is the fused feature vector, and the output is the confidence probability of belonging to the interference region.

[0046] When determining the confidence probability value based on a preset confidence threshold to select the initial set of interference points, and then using a region growing algorithm to merge connected components and smooth edges of the discrete initial interference points to generate optical interference regions, the confidence threshold determination involves comparing the model's output confidence probability value with a pre-set threshold. The confidence threshold is the optimal classification threshold determined through ROC curve analysis based on historical labeled data and model validation results, set between 0.6 and 0.8. The specific value is dynamically adjusted based on the tolerance for false positives and false negatives in the actual process. For example, if the probability value is greater than 0.7, the point is considered to belong to the optical interference region.

[0047] Region growing algorithms start with one or more seed points and progressively add points within their neighborhoods that meet specific similarity criteria (e.g., a confidence probability value higher than a confidence threshold, or similar optical / geometric features to the seed point) to the region until no more points meet the criteria. Connected component merging and edge smoothing are performed on multiple discrete regions obtained by the region growing algorithm, merging them based on their distance or degree of overlap. Edge smoothing can be achieved through morphological operations (such as dilation, erosion, opening, and closing operations) to eliminate jagged or irregular portions on region boundaries, making the generated interference regions more regular.

[0048] Through the above technical solution, the multi-dimensional feature fusion and intelligent recognition strategy of this application can effectively avoid abnormal light spot state caused by individual differences and regional non-uniformity of optical properties on the workpiece surface, thereby improving coordinate recognition accuracy and ensuring positioning accuracy stability and batch operation yield.

[0049] In one embodiment of this application, an initial projection scheme is generated based on the workpiece pose, preset target points, and optical characteristic distribution map, including: Determine the target projection point on the workpiece surface based on the workpiece pose and the preset target point. Based on the optical characteristic distribution map, determine whether the target projection point is located within the optical interference area; If it is located within the optical interference area, the corresponding feedforward compensation parameter is matched from the preset control strategy library according to the type of optical interference, and used as a component of the initial laser projection parameter. If the preset process requirements still cannot be met after compensation by the feedforward compensation parameters, then determine whether the target projection point is an unchangeable reference positioning point according to the preset process document. If the reference positioning point is unchangeable, maintain the feedforward compensation parameters and record the abnormal markers of the target projection point; If the reference positioning point is not immutable, then the sampling point with the lowest interference and curvature change less than the preset curvature threshold is searched within the preset neighborhood as the candidate point. If a candidate point is found, it is used as the new target projection point, and the projection angle and initial laser projection parameters are recalculated based on the new target projection point. If no qualified candidate point is found, the target projection point is maintained, and its feedforward compensation parameters are determined according to the unchangeable point strategy. An initial projection scheme is generated based on the target projection point and the initial laser projection parameters.

[0050] In this embodiment, a target projection point on the workpiece surface is determined based on the workpiece pose and a preset target point. The workpiece pose is described by six degrees of freedom parameters (e.g., three translations and three rotations) to indicate the workpiece's position and orientation in the equipment coordinate system. The preset target point refers to a specific location determined during the workpiece design or process planning stage that requires laser positioning; it is given in three-dimensional coordinates within the workpiece's own coordinate system. Determining the target projection point can be achieved by transforming the preset target point to the current equipment coordinate system using a workpiece pose transformation matrix and projecting it onto the workpiece's surface model. Alternatively, the preset target point can be directly located on the digitized workpiece model using the workpiece's CAD model and the currently calculated workpiece pose, and its precise three-dimensional coordinates in the equipment coordinate system can be obtained as the actual target point for laser projection.

[0051] Secondly, based on the optical property distribution map, it is determined whether the target projection point is located within the optical interference region. The optical property distribution map provides optical attribute information for various regions of the workpiece surface, including identified and calibrated optical interference regions. The determination process can be completed by comparing the spatial coordinates of the target projection point with the geometric boundaries of the marked optical interference regions on the optical property distribution map. If the coordinates of the target projection point fall within any of the calibrated optical interference regions, it is determined that it is located within the optical interference region.

[0052] If the target projection point is determined to be within an optical interference region, corresponding feedforward compensation parameters are matched from a pre-defined control strategy library based on the type of optical interference, serving as components of the initial laser projection parameters. The control strategy library is a pre-established knowledge base storing optimized combinations of laser projection parameters for different types of optical interference (e.g., high-reflection regions, strong absorption regions, scattering regions, etc.), such as laser power, pulse width, frequency, spot shape, or scanning speed. Matching is performed based on the specific type of optical interference identified (e.g., if identified as a specular reflection region, a strategy of reducing power and adjusting pulse width is matched) to pre-compensate for potential spot anomalies before laser projection.

[0053] The feedforward compensation parameter matching in the aforementioned control strategy library is implemented using a combination of case-based reasoning and deep neural networks. Specifically, the feature extraction network employs a three-layer convolutional neural network to extract features from the optical interference region. The input is a local image patch (32×32 pixels) of the optical interference region. After passing through two convolutional layers (3×3 kernels, stride 1, and 16 and 32 channels respectively) and a pooling layer, a 128-dimensional feature vector is obtained. The classification and regression network concatenates the 128-dimensional feature vector extracted by the convolutional neural network with the one-hot encoding of the optical interference type (specular reflection, diffuse reflection, absorption, etc.). This concatenation is then input into a three-layer fully connected network (hidden layer nodes: 256, 128, 64). The output layer has two branches: the classification branch uses the Softmax activation function and outputs the best-matching compensation strategy number; the regression branch uses a linear activation function and outputs continuous compensation parameter adjustments (such as power adjustment, pulse width adjustment, etc.).

[0054] The training method includes collecting images of optical interference regions encountered in historical positioning tasks, interference type labels, and optimal compensation parameters determined by expert systems or manual adjustments to form training samples. Each sample includes: an image of the interference region, an interference type label, and the optimal compensation parameters actually used (e.g., power +10%, pulse width -5%). A multi-task loss function is used; the optimizer is Adam, with a learning rate of 0.0005, a batch size of 64, and 150 training epochs. An early stopping mechanism is implemented: training stops if the validation set loss does not decrease for 15 consecutive epochs. After training, the model's compensation parameter prediction error (e.g., power adjustment) on the validation set must be less than ±2% of the nominal value, and the classification accuracy must be no less than 90%. The model's input is a local image of the optical interference region and interference type labels; the output is the optimal compensation strategy number and the specific compensation parameter adjustment amount. By learning from historical successful cases, the model establishes a mapping relationship between the visual features of interference and the optimal compensation parameters, enabling it to quickly provide a pre-compensation solution when encountering similar interference.

[0055] If the preset process requirements still cannot be met after feedforward compensation, the target projection point is determined to be an unchangeable reference positioning point based on the preset process document. Process requirements include strict limitations on indicators such as spot position accuracy and spot quality (e.g., circularity, energy concentration). If the simulation or actual test results after compensation still do not meet the standards, further judgment is required. The process document is a document or database that includes product design specifications and manufacturing process constraints, clearly indicating which positioning points are critical reference points for product function or assembly, and these points are not allowed to be adjusted in position. The attribute information of the target projection point is obtained by querying this process document to determine whether its position can be adjusted.

[0056] If the target projection point is determined to be an immutable reference point, the feedforward compensation parameters are maintained, and an anomaly marker for the target projection point is recorded. This means that even if the compensation does not fully meet the process requirements, the point's position cannot be changed due to its importance, and therefore the current feedforward compensation scheme must be accepted. The anomaly marker can be a system log entry recording information such as optical interference at the reference point or unsatisfactory compensation results, for subsequent quality traceability, manual intervention, or process improvement. For example, it can record the anomaly type, occurrence time, target point coordinates, and currently applied compensation parameters.

[0057] If the target projection point is determined not to be an immutable reference positioning point, then within a preset neighborhood, the sampling point with the lowest interference level and curvature change less than a preset curvature threshold is searched as a candidate point. The preset neighborhood is a defined range surrounding the original target projection point, such as a circular or square area, the size of which can be set according to the workpiece size, positioning accuracy requirements, and process flexibility. The search process traverses all sampling points within this neighborhood and evaluates each sampling point. Evaluation criteria include its optical interference level (e.g., reflectivity, absorptivity, etc. based on optical property distribution maps) and local surface curvature change (e.g., the difference between the new point and the average curvature of the original target point or the neighborhood). Selecting the point with the lowest interference level aims to avoid optically anomalous areas, while curvature change less than the preset curvature threshold ensures that the new point is geometrically similar to the original point, avoiding unstable laser projection effects due to drastic curvature changes. The search algorithm employs a heuristic search method to efficiently find the best candidate point that meets the conditions.

[0058] The preset curvature threshold is determined based on the workpiece surface machining tolerances. Through theoretical calculations or optical simulations, it is determined at what value the surface curvature change will cause the spot size to exceed the allowable range or the energy density distribution to become distorted. For example, for a laser beam with a focal length of f and a depth of focus of DOF, the preset curvature threshold is set as a function related to DOF, ensuring that the height change caused by curvature at candidate points is less than 1 / 3 of the DOF.

[0059] If a candidate point is found, it is used as the new target projection point, and the projection angle and initial laser projection parameters are recalculated based on the new target projection point. Once a better candidate point is found, it is established as the new laser projection target point. Subsequently, based on the position of this new target point on the workpiece's 3D model, combined with the geometric model and kinematic constraints of the laser emitter, the projection angle of the laser emitter pointing towards the workpiece surface is recalculated. At the same time, based on the local optical properties and surface geometric features of the new target point, a new set of initial laser projection parameters is re-evaluated and determined to ensure that the best laser projection effect can be obtained at the new position.

[0060] If no suitable candidate point is found, the target projection point is maintained, and its feedforward compensation parameters are determined according to the unchangeable point strategy. This means that an ideal alternative point that simultaneously satisfies low optical interference and small curvature change cannot be found within the preset neighborhood. In this case, the strategy used for handling unchangeable reference points will be reverted, i.e., the original target projection point is retained, and the previously matched or calculated feedforward compensation parameters are applied to minimize the impact of optical interference. A warning will be issued or relevant information will be recorded, indicating that positioning in this area is challenging. A suitable candidate point is defined as a sampling point with the smallest comprehensive interference index, a curvature change less than a preset curvature threshold, and a spatial distance not exceeding the maximum allowable offset threshold. These thresholds are determined experimentally based on workpiece machining accuracy and laser positioning requirements.

[0061] Finally, based on the target projection point and initial laser projection parameters, an initial projection scheme is generated. This step integrates the results of all previous decisions and calculations to form a complete and executable set of laser projection instructions. The initial projection scheme includes the projection angle of the laser emitter pointing towards the workpiece surface (whether it is the original target point or the adjusted candidate point) and a set of initial laser projection parameters (such as power, pulse width, frequency, etc.) that have been pre-compensated or optimized for optical interference. This scheme will serve as the starting setting for the laser positioning and detection process, guiding the laser emitter to perform its first projection.

[0062] Through the above technical solution, this application effectively solves the problem that optical interference areas on the workpiece surface cause the target projection point to be unsuitable for laser projection, leading to abnormal spot state and decreased positioning accuracy. This application first determines the target projection point based on the workpiece pose and the preset target point location, ensuring basic positioning accuracy. Then, based on the optical characteristic distribution map, it determines whether the target projection point is located within the optical interference area. This proactive identification mechanism can pre-detect potential problem areas, preventing the laser from directly projecting onto areas causing abnormal spots. If interference is detected, the corresponding feedforward compensation parameters are matched from the preset control strategy library according to the type of optical interference, directly pre-compensating for the interference type, thereby mitigating its impact on spot quality before laser projection. Furthermore, this application introduces a judgment on whether the target projection point is an unchangeable reference positioning point, enabling the system to distinguish between critical positioning points and adjustable positioning points. For unchangeable reference points, even if there are still deficiencies after compensation, their position is maintained and the anomaly is recorded, ensuring compliance with core process requirements and process traceability. For non-reference points, the system searches for sampling points with the lowest interference and curvature changes less than a preset curvature threshold within a preset neighborhood as candidate points. This strategy aims to find a better projection position locally to further optimize the laser projection effect. If a qualified candidate point is found, it is used as the new target projection point, and the projection angle and initial laser projection parameters are recalculated. This achieves dynamic adjustment and optimization of the initial projection scheme, improving its adaptability and robustness. Even if an ideal candidate point is not found, the system can revert to maintaining the original target point and performing maximum compensation, ensuring the feasibility of the scheme. Finally, by integrating these decisions, an initial projection scheme including optimized projection angles and initial laser projection parameters is generated. This mechanism for dynamically handling optical interference areas makes the generation of the initial projection scheme more intelligent and robust, effectively avoiding problems such as abnormal spot size and decreased positioning accuracy caused by uneven optical properties of the workpiece surface or the presence of interference areas. Compared with traditional methods that rely solely on fixed parameters or simple position adjustments, this improves the initial setup quality of laser positioning, further enhancing the accuracy, stability, and yield of the overall positioning and detection method.

[0063] In one embodiment of this application, the sampling point with the lowest interference level and curvature change less than a preset curvature threshold within a preset neighborhood is searched as a candidate point, including: The radius of the search neighborhood is determined based on the local surface curvature and optical property distribution at the target projection point, where the radius is inversely proportional to the rate of change of local curvature and directly proportional to the confidence level of the optical interference region. Within the search neighborhood, the comprehensive interference index of each sampling point is calculated. The comprehensive interference index is a weighted sum of the specular reflection index, diffuse reflection index, and absorptivity index. The sampling points with the smallest comprehensive interference index and curvature change less than the preset curvature threshold are selected as candidate points.

[0064] In this embodiment, the local surface curvature is calculated by fitting the three-dimensional point cloud data of the workpiece to the local surface (e.g., fitting a quadratic surface using the least squares method). This curvature value characterizes the geometric flatness or curvature of the surface near the target projection point. The optical property distribution is obtained from the aforementioned optical property distribution map. Based on the optical properties, the radius of the search neighborhood can be determined. For example, when the rate of change of local curvature at the target projection point is large, it indicates that the surface geometry of that region is complex. To avoid the laser being projected into an unstable region, the search radius should be reduced accordingly. Conversely, when the rate of change of curvature is small, the surface is relatively flat, and the search radius can be increased to find a better solution. At the same time, when the confidence level of the optical interference region is high, it means that there is significant optical interference in that region. To effectively avoid this interference, the search radius should be increased accordingly. Conversely, when the confidence level is low, the search radius can be reduced. This dynamic adjustment mechanism ensures the rationality of the search range and avoids blind searching.

[0065] Specifically, the radius of the search neighborhood is determined by both the rate of change of local curvature and the confidence level of optical interference, and the calculation formula is as follows: Among them, The reference radius for searching the neighborhood is preset based on the effective working range of the laser emitting device and the workpiece size; The interference confidence level influence coefficient. The coefficient representing the influence of the rate of change of curvature is denoted as , and ; The interference confidence level is output by the optical interference identification model; The rate of change of curvature is calculated from 3D point cloud data, and . and Through experimental design, typical workpieces with different optical properties and geometric features were selected, and a grid search was performed within a preset value range. The search success rate and positioning accuracy were used as evaluation indicators to select the workpiece with the optimal overall performance. and The value is set as a fixed parameter; or it is adjusted online using an adaptive algorithm based on feedback data from actual applications. and This is to adapt to the changing characteristics of different batches of workpieces. Among them, This is used to adjust the amplification effect of the interference confidence level on the search radius. The larger the value, the larger the search range of the region with high interference confidence level. This is used to adjust the inhibitory effect of the rate of curvature change on the search radius. The larger the value, the more drastically the search range shrinks in areas with large curvature changes.

[0066] Within the search neighborhood, the comprehensive interference index for each sampling point needs to be calculated. A sampling point refers to a discrete point extracted from the workpiece's 3D point cloud data within the search neighborhood. The comprehensive interference index aims to quantify the degree of optical interference at each sampling point; it is a weighted sum of the specular reflection index, diffuse reflection index, and absorptivity index. These indices can be directly obtained from the aforementioned optical characteristic distribution map. They characterize the surface's specular reflection, diffuse reflection, and absorption properties to laser light, respectively. By weighting and summing these three indices, the optical interference at the sampling point can be evaluated. For example, the weights of each index can be determined empirically or experimentally, such as assigning a higher weight to the specular reflection index because it has a greater impact on laser positioning accuracy.

[0067] Finally, from all sampling points within the search neighborhood, the sampling point with the smallest comprehensive interference index and a curvature change less than a preset curvature threshold is selected as the candidate point. The condition of the smallest comprehensive interference index ensures that the selected candidate point has the lowest optical interference, which is beneficial for the stable formation and accurate detection of the laser spot. The condition of a curvature change less than the preset curvature threshold ensures that the selected candidate point is located in a relatively flat area or an area with small curvature changes, avoiding laser projection onto geometrically unstable surfaces, which is crucial for maintaining spot quality and positioning accuracy.

[0068] Through the above technical solution, this application can dynamically determine the radius of the search neighborhood based on the local surface geometric features and optical property distribution at the target projection point, thereby avoiding blind searching and improving search efficiency. Simultaneously, by introducing a comprehensive interference index, defined as a weighted sum of specular reflection, diffuse reflection, and absorptivity indices, this application can comprehensively and accurately assess the degree of optical interference at each sampling point, overcoming the limitations of single-index evaluation. Based on this, candidate points are selected by combining the conditions of minimum comprehensive interference index and curvature change less than a preset curvature threshold, ensuring that the selected candidate points not only have minimal optical interference but also stable surface geometry, thus effectively avoiding optical interference areas and guaranteeing the surface stability of laser projection. This allows for the selection of better projection points when generating the initial projection scheme, thereby improving the robustness and accuracy of the entire laser positioning and detection method, especially when dealing with workpieces with complex surface optical properties and geometries, increasing the success rate of positioning and the yield of operations.

[0069] In one embodiment of this application, after selecting the sampling point with the smallest comprehensive interference index and a curvature change less than a preset curvature threshold as the candidate point, the method further includes: Calculate the spatial distance between the candidate point and the target projection point; If the spatial distance is greater than the preset distance threshold, a penalty factor is applied to the comprehensive interference index based on the spatial distance to obtain the penalized comprehensive interference index. The priority of the candidate points is re-evaluated, and the point with the smallest overall interference index after penalty is selected as the final candidate point. If the spatial distance of all candidate points is greater than the preset maximum allowable offset threshold, the search is abandoned and an alarm signal is issued or a global replanning process is initiated. If the spatial distance between all candidate points is greater than the preset distance threshold but less than the maximum allowable offset threshold, the search neighborhood radius is expanded and the search is repeated until a candidate point that satisfies the distance constraint and has the smallest comprehensive interference index is found.

[0070] In this embodiment, the spatial distance between the candidate point and the target projection point is calculated to quantify the degree of positional offset between the candidate point and the original target projection point. This spatial distance is calculated using the three-dimensional point cloud data of the workpiece to determine the Euclidean distance between the candidate point and the target projection point in three-dimensional space.

[0071] If the spatial distance exceeds a preset distance threshold, a penalty factor is applied to the comprehensive interference index based on the spatial distance, resulting in a penalized comprehensive interference index. This step introduces a penalty mechanism to reduce the priority of candidate points that, while having low optical interference, are too far from the original target point, thus balancing optical characteristic optimization with positional accuracy requirements. The penalty factor can be designed as a function positively correlated with the spatial distance; it takes effect when the spatial distance exceeds the preset distance threshold and increases with increasing distance.

[0072] After obtaining the penalized composite interference index, the priority of candidate points needs to be re-evaluated, and the point with the smallest penalized composite interference index is selected as the final candidate point. This process ensures that all candidate points can be reordered after considering the spatial distance penalty, thereby selecting the point that achieves the best balance between optical interference and positional offset. This is achieved by sorting all candidate points in ascending order of their penalized composite interference indices and selecting the point ranked first.

[0073] If the spatial distance between all candidate points exceeds the preset maximum allowable offset threshold, the search is abandoned and an alarm signal is issued, or a global replanning process is initiated. This mechanism serves as a safety safeguard, ensuring that the current invalid search is stopped promptly when a qualified candidate point cannot be found within an acceptable positional offset range, preventing the system from entering an infinite loop or generating unacceptable positioning errors. Alarm signals can be issued to the operator via a human-machine interface, audible and visual indicators, or network communication modules. Initiating a global replanning process triggers higher-level decision-making modules, such as reassessing the positioning strategy for the entire workpiece, adjusting the equipment attitude, or requesting manual intervention.

[0074] If the spatial distance between all candidate points is greater than a preset distance threshold but less than the maximum allowable offset threshold, the search neighborhood radius is expanded and the search is repeated until a candidate point that satisfies the distance constraint and has the minimum comprehensive interference index is found. An adaptive search strategy is provided, which, within the allowable offset range, gradually expands the search range to find better candidate points, thereby improving the success rate and robustness of the search. Expanding the search neighborhood radius can be done with a fixed step size or by increasing it proportionally until a candidate point that meets the conditions is found or the preset maximum search radius is reached. Simultaneously, the radius increment can be dynamically adjusted based on the current search results (e.g., the number of candidate points within the distance threshold, the distribution of the interference index, etc.) to further improve search efficiency. The maximum allowable offset threshold is determined based on the workpiece structural characteristics and process tolerances.

[0075] Through the above technical solution, this application, when selecting candidate laser projection points, not only considers the optical characteristics and curvature changes of the workpiece surface, but also introduces spatial distance constraints. By calculating the spatial distance between the candidate point and the target projection point and applying a penalty factor to candidate points that are too far away, the problem of candidate points deviating too far from the original target projection point due to simply pursuing the minimization of optical interference is effectively avoided. When all candidate points exceed the maximum allowable offset threshold, the system can promptly abandon the search and issue an alarm or initiate replanning to prevent invalid operations. Within the allowable offset range, by adaptively expanding the search neighborhood radius and re-searching, it ensures that the optimal projection point with the least optical interference can still be found while maintaining positional accuracy. This improves the robustness and practicality of the laser positioning and detection method, enabling laser positioning to simultaneously meet the dual requirements of high accuracy and low interference in complex workpiece surface environments. This effectively solves the problem of candidate points deviating from the original target projection point due to excessive spatial distance, leading to decreased actual positioning accuracy, impractical operation, or low search efficiency.

[0076] In one embodiment of this application, a penalty factor is applied to the comprehensive interference index based on spatial distance to obtain a penalized comprehensive interference index, including: Calculate the ratio of the spatial distance between the candidate point and the target projection point to a preset distance threshold, and obtain the angle between the local surface normal at the candidate point and the expected projection direction; The penalty factor is calculated based on the ratio and the included angle using a preset penalty function; Multiply the penalty factor by the comprehensive interference index to obtain the penalized comprehensive interference index.

[0077] In this embodiment, the Euclidean distance between the candidate point and the target projection point in the three-dimensional coordinate system is calculated, and then compared with a preset distance threshold to obtain the ratio. The preset distance threshold is determined based on the positioning accuracy requirements of the target point. For example, if the target point is the center of an aperture with a position tolerance of ±0.05mm, the preset distance threshold is set to 0.02mm (to activate the penalty mechanism), and the maximum allowable offset threshold is set to 0.05mm (as a hard boundary for the search). This step allows the system to consider not only the optical characteristics of the candidate point but also its positional relationship with the original target projection point when evaluating the candidate point, avoiding the selection of candidate points that are too far away and would lead to actual positioning deviations.

[0078] Secondly, the angle between the local surface normal at the candidate point and the expected projection direction is obtained. This step is used to evaluate the impact of the surface geometry at the candidate point on the laser projection effect. The local surface normal can be calculated by performing local plane fitting or principal component analysis on the 3D point cloud data around the candidate point. The expected projection direction refers to the ideal direction in which the laser emitting device points to the candidate point. By calculating the angle between these two vectors, it can be determined whether severe specular reflection, scattering, or spot distortion will occur when the laser beam is incident on the point, thus affecting the quality of the spot and the accuracy of position recognition. For example, when the angle is too large, it can cause the spot energy to disperse or form an irregular shape.

[0079] Next, a penalty factor is calculated using a preset penalty function based on the ratio and angle. This step combines two key geometric factors—spatial distance and surface normal angle—to generate a dynamic adjustment coefficient. This penalty function can be a mathematical model, such as a piecewise linear function or an exponential function, whose inputs are the aforementioned ratio and angle, and whose output is a penalty factor greater than or equal to 1. The magnitude of the penalty factor is positively correlated with the spatial distance ratio and angle; that is, the greater the distance or the larger the angle, the larger the penalty factor. In this way, the comprehensive interference index of candidate points can be flexibly adjusted according to actual geometric conditions.

[0080] Finally, the penalty factor is multiplied by the comprehensive interference index to obtain the penalized comprehensive interference index. This step is crucial for ultimately refining the candidate point evaluation index. The comprehensive interference index primarily characterizes the optical properties of the candidate point, while the penalty factor introduces the influence of geometric position and surface angle. Through multiplication, the penalty factor can effectively amplify or reduce the original comprehensive interference index. This means that candidate points with low optical interference but poor geometric position or angle will have an increased penalized comprehensive interference index, thus lowering their selection priority. Conversely, candidate points with superior geometric conditions will maintain a low penalized comprehensive interference index.

[0081] Through the above technical solution, this application, when selecting candidate points, not only considers the optical characteristics of the workpiece surface, but also fully takes into account the spatial distance and surface normal angle between the candidate point and the original target projection point. Specifically, by calculating the spatial distance ratio, the positional offset of the candidate point is quantified, avoiding the selection of points with good optical characteristics but excessive deviation in actual position; by obtaining the angle between the local surface normal and the expected projection direction, the influence of surface geometry on laser projection quality is evaluated, effectively avoiding spot distortion or energy loss caused by surface tilt. The preset penalty function dynamically integrates these two geometric factors into a penalty factor and applies it to the comprehensive interference index, so that the final penalized comprehensive interference index can more comprehensively and accurately characterize the comprehensive applicability of the candidate point. This application can prioritize the selection of candidate points that not only have low optical interference, but are also more suitable for laser projection in terms of spatial position and surface angle, thereby improving the accuracy, stability, and spot quality of laser positioning, effectively solving the problems of spot quality degradation and positioning accuracy fluctuation caused by positional offset and surface angle deviation, and ensuring the yield of operations under positioning requirements.

[0082] In one embodiment of this application, expanding the search neighborhood radius and re-searching includes: Obtain the comprehensive interference index distribution of all candidate points in the previous search, calculate its mean and variance, and determine the radius increment of the current search based on the mean and variance. If the mean is greater than the first mean threshold and the variance is less than the first variance threshold, the radius increment is increased based on the first step size. If the mean is less than the first mean threshold and the variance is greater than the first variance threshold, the radius increment is decreased based on the second step size, and the search neighborhood radius is expanded with the adjusted radius increment for a re-search.

[0083] In this embodiment, the first mean threshold and the first variance threshold are dynamically set based on the statistical characteristics of historical search results. Specifically, the first mean threshold is taken as the median or 75th percentile of the comprehensive interference index, used to determine whether the overall optical interference level of candidate points in the current search area is too high; the first variance threshold is used to evaluate the stability of the search results, taking the mean of the standard deviations of the interference indices of all candidate points. If the variance is greater than the first variance threshold, it indicates that the interference index is dispersed and the search results are unstable.

[0084] The first step length and the second step length are determined based on a combination of process requirements and search efficiency. The first step length is used to appropriately expand the search range when the overall interference level is high but the distribution is relatively stable in order to find better candidate points. The second step length is used to appropriately narrow the search range when the interference level is low but the distribution is unstable in order to focus on more reliable areas.

[0085] If the mean is greater than the first mean threshold and the variance is greater than the first variance threshold, it indicates that the level of interference is high and the distribution is unstable. In this case, a smaller step size should be used to cautiously expand the search radius to avoid introducing greater interference. If the mean is less than the first mean threshold and the variance is less than the first variance threshold, it indicates that the interference level is low and the distribution is stable. In this case, the current radius can be maintained or a smaller step size can be used for fine-tuning to improve search efficiency.

[0086] The step size and threshold mentioned above can be dynamically optimized through offline experiments or online self-learning. For example, by collecting historical data from multiple searches, a mapping model between search performance and step size and threshold can be established, and Bayesian optimization methods can be used to automatically adjust the parameters, allowing the search strategy to gradually approach the optimal state.

[0087] Through the aforementioned adaptive radius adjustment mechanism, this application can dynamically optimize the search range based on the statistical characteristics of local search results, avoiding blindly expanding or shrinking the radius, thereby improving the success rate and efficiency of candidate point search and ensuring that a projection point that meets the requirements can be found quickly in complex workpiece surface environments.

[0088] In one embodiment of this application, if no qualified candidate point is found, the target projection point is maintained, and its feedforward compensation parameters are determined according to the immutable point strategy, and the method further includes: After generating the initial projection scheme, the first vision sensor is used to perform a local scan of the target projection point to obtain the surface morphology features of the target projection point; Based on the surface morphology characteristics, the determined feedforward compensation parameters are adjusted to optimize the laser projection parameters in the initial projection scheme.

[0089] In this embodiment, after generating the initial projection scheme, a first vision sensor is used to perform a local scan of the target projection point to acquire surface geometric information of the area where the target projection point is located, such as micro-roughness, local curvature changes, or minor defects. The local scanning and morphological feature acquisition can be achieved in two ways: One method is that after generating the initial projection scheme, the first vision sensor (e.g., a high-resolution industrial camera) is guided to the vicinity of the target projection point to acquire multiple images of the area at a higher sampling frequency or a closer working distance. Subsequently, using multi-view stereo vision or structured light projection techniques, local 3D reconstruction is performed on these images to generate high-density point cloud data or a surface mesh model of the target projection point area. From this 3D data, morphological features such as local curvature, surface roughness parameters, and micro-texture can be extracted. Another method is that the first vision sensor can be integrated with or used in conjunction with a laser triangulation sensor. After generating the initial projection scheme, a line scan or point scan of the target projection point is performed. By analyzing the deformation of the laser spot on the workpiece surface or the intensity changes of the reflected signal, micron-level or even nanometer-level surface height information of the target projection point area can be directly obtained in real time, thereby calculating accurate surface morphology features, such as local slope and micro-undulation.

[0090] Based on this, the determined feedforward compensation parameters are adjusted according to the surface morphology characteristics to optimize the laser projection parameters in the initial projection scheme. This step aims to ensure that the laser projection parameters can more accurately adapt to the actual surface conditions of the target projection point, thereby improving positioning accuracy and spot quality, and reducing spot drift or quality degradation caused by surface morphology differences. The parameter adjustment and optimization are achieved by pre-establishing a compensation model based on the relationship between surface morphology characteristics and laser projection parameters. This compensation model is trained using a large amount of experimental data; for example, samples with different surface roughness and local curvature are correlated with corresponding optimal laser power, pulse width, and frequency parameters. Once the surface morphology characteristics of the target projection point are obtained, they are input into the compensation model, which outputs a correction value to adjust the previously determined feedforward compensation parameters, thereby obtaining more optimized laser projection parameters.

[0091] Through the above technical solution, after maintaining the target point when no suitable candidate point is found, the system uses a first vision sensor to locally scan the target projection point to obtain its surface morphology features, compensating for the shortcomings of relying solely on optical property distribution maps in terms of geometric details. Subsequently, based on these real morphology features, the determined feedforward compensation parameters are dynamically adjusted, enabling the laser projection parameters to more accurately match the actual surface conditions of the target point and ensuring effective coupling between laser energy and the workpiece surface. This reduces the risk of spot drift and quality degradation, thereby improving the accuracy and stability of laser positioning, and ensuring positioning quality even when an ideal candidate point cannot be found through adjustments.

[0092] In one embodiment of this application, the coupled control process of executing the first closed-loop control and the second closed-loop control in parallel includes: Obtain the spot position deviation sequence and quality index sequence within the current control cycle; The position deviation sequence and quality index sequence are input into a preset coupled control model, and the first control quantity for angle adjustment and the second control quantity for laser parameter adjustment are decoupled and calculated respectively. The first control quantity includes a compensation term for compensating for spot drift caused by changes in laser parameters, and the second control quantity includes a compensation term for compensating for spot quality attenuation caused by changes in angle. The coupled control model is used to describe the coupled effects of changes in projection angle and laser parameters on the spot position and quality indicators. The angle adjustment mechanism is driven by the first control quantity, and the laser emitting device is driven by the second control quantity, so as to achieve synchronous coupling correction of the projection angle and laser parameters.

[0093] In this embodiment, the steps of acquiring the laser spot position deviation sequence and quality index sequence within the current control cycle aim to continuously monitor the real-time status of the laser spot. The laser spot position deviation sequence refers to the record of the spatial distance or coordinate difference between the actual center position of the laser spot and the preset target point changing over time within a continuous control time cycle. The quality index sequence refers to the record of the changes over time in quantitative indicators (e.g., ellipticity, peak intensity, size, energy concentration, etc.) describing the characteristics of the laser spot, such as shape, energy distribution, and intensity uniformity, within the same time cycle. By acquiring sequence data rather than single instantaneous values, the dynamic change trend and response characteristics of the laser spot are characterized, providing rich and time-dependent feedback information for subsequent coupled control. This application utilizes a second vision sensor to continuously acquire laser spot images on the workpiece surface and extracts the position deviation and various quality indices of the laser spot in real time using image processing algorithms (e.g., calculating the spot center based on the gray-scale centroid method, evaluating the spot shape and size based on the second-order moment method, or analyzing energy concentration based on pixel intensity distribution). These data are then stored as a sequence in chronological order.

[0094] The process of inputting the position deviation sequence and quality index sequence into a preset coupled control model, and decoupling and calculating a first control quantity for angle adjustment and a second control quantity for laser parameter adjustment, involves using a mathematical model that understands the internal coupling relationship to separate the mutually influencing control objectives (spot position and quality). The coupled control model receives real-time spot position deviation and quality index sequences as input and calculates two independent control signals using an internal decoupling algorithm: the first control quantity drives the angle adjustment mechanism, and the second control quantity drives the laser emitting device. These two control quantities are not simply calculated independently but include mutually compensating terms. For example, when laser parameters change, causing a slight drift in the spot position, the first control quantity includes a compensation term to offset this drift; conversely, when the projection angle changes, leading to a decrease in spot quality, the second control quantity includes a compensation term to restore or maintain the spot quality. This design ensures that when adjusting one variable, its negative impact on another variable can be anticipated and offset simultaneously. This application employs a model predictive control strategy, where the coupled control model is a multi-input multi-output model. By predicting behavior over a future period and combining it with a decoupling matrix or feedforward compensator, the optimal first and second control variables are calculated, minimizing mutual interference while satisfying position and quality objectives.

[0095] The coupled control model describes the coupled effects of changes in projection angle and laser parameters on the spot position and quality indicators. This model forms the theoretical basis of the entire coupled control process, quantifying the interaction relationships between the internal variables. It reveals that when the projection angle changes slightly, in addition to a change in spot position, its quality indicators (such as spot shape and energy density) are also affected; conversely, when laser parameters (such as power, pulse width, and focal length) are adjusted, in addition to changes in spot quality, its actual position on the workpiece surface also drifts. An accurate coupling model is crucial for achieving effective decoupling and synchronous correction.

[0096] Specifically, the coupled control model is a multi-input multi-output linear time-invariant state-space model. Its training dataset was obtained through system identification experiments on a real device: with the workpiece stationary, pseudo-random binary sequence excitation signals (angle change amplitude ±0.5°, power change amplitude ±10%) were applied to the laser emitting device and the angle adjustment mechanism. The spot position and quality response were simultaneously acquired, with a sampling period of 10ms, continuously collecting data for at least 10,000 cycles. The data was divided into identification and validation datasets, 70% and 30% respectively. Subspace identification was used to initially estimate the model parameters, and then the prediction error was used for iterative optimization, aiming to minimize the prediction error. The training stopped when the root mean square error of the predictions on the validation set was less than half of the system's preset error threshold. The final model was validated on the test set, demonstrating its ability to accurately describe the system's dynamic characteristics.

[0097] The model structure of the coupled regulation model is as follows: Equations of state: Output equation: in, The state vector has four dimensions and includes internal states that cannot be directly measured, such as the current position deviation of the light spot, the cumulative effect of historical deviations, and dynamic components of light spot quality indicators (such as the rate of change of energy concentration). For the input vector, Let be the adjustment amount of the projection angle of the laser emitting device during the k-th control cycle. This represents the adjustment amount of the laser parameters during the k-th control cycle; For the output vector, The deviation of the light spot position measured at the end of the k-th control cycle. The light spot quality index is measured at the end of the kth control cycle; The state transition matrix is ​​4×4 in dimension and describes the evolution of the internal state of the system over time. The input matrix is ​​4×2-dimensional and describes the effect of the input control quantity on the state. The output matrix is ​​2×4-dimensional and describes the mapping relationship from the state to the measurable output. The direct pass matrix is ​​2×2 in size and describes the direct effect of the input on the output (set as a 0 matrix). The noise is 4×1 dimensional and follows a zero-mean Gaussian distribution. For measuring noise, a 2×1 dimension is used, which follows a zero-mean Gaussian distribution.

[0098] The model construction and parameter identification process includes the following steps: Experimental Design: A series of pseudo-random binary sequence (PRBS) excitation signals are applied to the laser positioning system while the projection angle and laser parameters are changed independently. The response data of the spot position deviation and quality indicators are recorded at a fixed sampling period (e.g., 10 ms). The amplitude range of the PRBS signal is determined based on the system's linear operating range, with the angle change amplitude not exceeding ±0.5° and the laser parameter change amplitude not exceeding ±10% of the rated value.

[0099] The collected raw data is denoised (e.g., using a low-pass filter to remove high-frequency noise) and normalized so that all data fall within the [-1,1] interval. The dataset is then divided into an identification dataset and a validation dataset, with a ratio of 70% and 30% respectively.

[0100] Based on prior knowledge and the physical characteristics of the system, the dimension of the state vector is determined to be 4. A subspace identification method is used to process the identification dataset, and the system matrix is ​​initially estimated. , , and The initial value is determined. With the goal of minimizing the prediction error, the system matrix is ​​iteratively optimized using the prediction error method.

[0101] In the step of synchronously coupling and correcting the projection angle and laser parameters, the calculated control commands are converted into actual physical actions. The angle adjustment mechanism (e.g., a high-precision galvanometer scanning system, a multi-axis electric displacement stage, or a rotary platform) receives the first control command and precisely adjusts the projection direction of the laser beam according to the command, thereby changing the position of the laser spot on the workpiece surface. The laser emitter (e.g., the laser itself, a beam shaping module, or a focusing lens group) receives the second control command and adjusts the laser output parameters, such as laser power, pulse frequency, pulse width, or focal length, according to the command to optimize the quality of the laser spot. This synchronous coupling correction emphasizes that these two adjustment actions are not independent but highly coordinated in time, and each adjustment has considered and compensated for the impact on the other, thus ensuring that while correcting the position of the laser spot, its quality is maintained or optimized, and vice versa.

[0102] For example, the angle adjustment mechanism uses a high-precision two-dimensional scanning galvanometer, with a positioning accuracy better than ±0.01mrad, a repeatability better than ±0.005mrad, and a response time of less than 1ms.

[0103] The laser emitting device uses a modulated fiber laser, with an output power adjustment range of 10% to 100% of the rated power and an adjustment step of 0.1% of the rated maximum adjustment range (FS); the pulse width adjustment range is 10ns to 500ns with a step of 1ns. Laser parameters are controlled via a digital communication interface with an update rate of no less than 1kHz.

[0104] The main controller, through a high-speed digital signal processor, simultaneously calculates the first and second control variables at the beginning of a control cycle (T=5ms). Within the first ms of the cycle, it broadcasts the angle adjustment to the galvanometer driver and the laser parameters to the laser controller via the bus. From the second to the fifth ms of the cycle, the actuator operates, while the second vision sensor acquires data for the next cycle, ensuring synchronization between the start of the angle adjustment and the parameter changes.

[0105] Through the above technical solution, this application synchronously drives the angle adjustment mechanism and the laser emitting device based on these decoupled and compensated control quantities, thereby realizing the synchronous coupling correction of the projection angle and laser parameters, which improves the accuracy and stability of laser positioning and detection. Especially when facing uneven optical properties of workpiece surfaces or high-precision positioning requirements, it can more efficiently and robustly adjust the spot position and quality to the preset requirements at the same time, effectively improving the yield of batch operations and overall production efficiency.

[0106] In one embodiment of this application, after associating and storing the target projection angle, target laser projection parameters, environmental features, and workpiece identification information in a database, the method further includes: When a new positioning and detection task for the same type of workpiece is received, environmental features are acquired. Perform similarity matching between environmental features and historical environmental features stored in the database; If at least one historical record with a similarity greater than a preset similarity threshold is matched, the corresponding historical target projection angle and historical target laser projection parameters are extracted from the database as the initial projection scheme for the new task. If no historical record with a similarity greater than the preset similarity threshold is found, a new initial projection scheme is generated by interpolation based on the parameters corresponding to the top N historical records with the highest similarity, combined with the difference between the current real-time environmental features and the environmental features of the top N historical records, and marked as a scheme to be verified.

[0107] Environmental features include at least one of ambient light intensity, ambient temperature, ambient humidity, and dust concentration in the air; similarity matching is calculated using weighted Euclidean distance, where the weight of each environmental feature is dynamically calibrated according to its influence on laser positioning accuracy.

[0108] In this embodiment, when a new positioning and detection task for the same type of workpiece is received, environmental characteristics are first acquired. These environmental characteristics can be collected in real time by various sensors integrated into the device; for example, a temperature sensor is used to acquire ambient temperature, a humidity sensor to acquire ambient humidity, a light sensor to acquire ambient light intensity, and a dust sensor to acquire the concentration of dust in the air.

[0109] Subsequently, environmental features are matched with historical environmental features stored in the database. This matching process aims to assess the similarity between the current environment and historical successful case environments, enabling intelligent reuse of historical experience. Specifically, similarity matching can be calculated using a weighted Euclidean distance algorithm, where the weights of each environmental feature are dynamically calibrated based on its impact on laser positioning accuracy. Alternatively, machine learning models (neural networks) can be used to vectorize environmental features, and similarity can be determined by calculating the cosine similarity between feature vectors.

[0110] If at least one historical record with a similarity greater than a preset similarity threshold is found, the corresponding historical target projection angle and historical target laser projection parameters are extracted from the database as the initial projection scheme for the new task. The purpose of this is to directly reuse validated parameters when the current environment is highly similar to a successful historical case, thereby improving efficiency and reliability. Specifically, a query operation is performed in the database, and based on the similarity matching results, the historical record that best matches the characteristics of the current environment is retrieved, and the target projection angle and target laser projection parameters corresponding to that record are extracted.

[0111] If no historical record with a similarity greater than a preset similarity threshold is found, a new initial projection scheme is generated based on the parameters corresponding to the top N most similar historical records. This is combined with the differences between the current real-time environmental features and the environmental features of the top N historical records, and interpolation is used to generate a new initial projection scheme, which is then marked as a scheme to be verified. In the absence of a completely matching historical record, this step generates an initial scheme based on approximate historical experience through intelligent interpolation and adjustment, and marks it to ensure subsequent verification. Specifically, a linear interpolation method can be used. For example, for the target projection angle and laser parameters, a weighted average or multidimensional interpolation can be calculated based on the relative differences between the current environmental features and the environmental features of the top N historical records to generate a new initial projection scheme. Simultaneously, the generated new initial projection scheme is marked as a scheme to be verified in the database for focused attention and adjustment during actual projection and verification.

[0112] The preset similarity threshold is determined using cross-validation. For example, some successful cases are randomly selected from the historical database, and the positioning errors caused by reusing historical parameters at different similarity levels are simulated. The lowest similarity value that makes the positioning error meet the process requirements is selected as the preset similarity threshold.

[0113] The weights of each environmental feature are dynamically calibrated through a machine learning model. For example, a regression model is trained to analyze the contribution of each feature to the positioning accuracy, thereby achieving adaptive adjustment of the weights.

[0114] The regression model is a multilayer perceptron regression model. The input layer is 4-dimensional (light intensity, temperature, humidity, dust concentration); the hidden layers consist of a first layer with 64 neurons (ReLU) and a second layer with 32 neurons (ReLU); the output layer has one neuron (no activation function), outputting the predicted localization accuracy error. The training method includes: extracting environmental features and their corresponding final localization errors from a historical task database. Data preprocessing includes feature normalization and logarithmic transformation of the error. The loss function is mean squared error, the optimizer is Adam, the learning rate is 0.001, the batch size is 16, the training epochs are 200, and an early stopping mechanism is used. After training, the contribution of each input feature to the output error is calculated using integral gradient descent, and after normalization, it is used as a weight in the weighted Euclidean distance. The multilayer perceptron regression model takes environmental features as input and outputs the predicted localization error; sensitivity analysis transforms the contribution of features to the error into weights, thus reflecting the influence of different environmental factors on accuracy in similarity matching.

[0115] Through the above technical solution, this application can intelligently utilize successful case data stored in the historical database to quickly and adaptively generate an initial projection scheme when receiving a new positioning and detection task of the same type of workpiece. This mechanism effectively solves the problem of how to efficiently utilize historical data and avoid redundant calculations and optimization processes when facing new positioning and detection tasks.

[0116] The weights of each environmental feature are dynamically calibrated based on its impact on laser positioning accuracy. Dynamic calibration involves adjusting the weights of each environmental feature in similarity calculations in real-time or periodically, according to a pre-defined evaluation model. This dynamism ensures that the weights adapt to different workpiece types, process requirements, or environmental changes, thus more accurately representing their actual impact on positioning accuracy. Dynamic calibration is performed using machine learning methods. For example, a large amount of historical positioning data is collected, including environmental features, initial projection schemes, and final positioning accuracy. Then, a regression model (neural network) is trained, using environmental features as input and positioning accuracy as output. By analyzing the contribution or sensitivity of each feature in the model, its weight in the weighted Euclidean distance calculation is dynamically adjusted. Dynamic calibration is crucial for ensuring the accuracy and adaptability of similarity matching. It enables the system to adjust the emphasis on different environmental factors according to actual conditions, thereby generating an initial projection scheme that better suits the current working conditions and improving the robustness and accuracy of positioning.

[0117] In this application, the weights of each environmental feature are dynamically calibrated based on their impact on laser positioning accuracy. The weights are not fixed values ​​but are dynamically adjusted according to the degree of influence. This ensures more accurate similarity calculations because the impact of different environmental factors on accuracy varies depending on the workpiece type or real-time conditions. Dynamic calibration can adaptively optimize the weights, improving the reliability and robustness of the matching. These features work synergistically to make similarity matching more accurate, thereby generating a more reliable initial projection scheme, reducing positioning errors, and ultimately improving the accuracy and stability of the entire laser positioning and detection method.

[0118] In one embodiment of this application, the verification of a light spot that meets preset requirements is performed using a first visual sensor or a third visual sensor, including: Control the first or third visual sensor to acquire an image of the range including the light spot; The range image is processed to extract the geometric center coordinates and energy distribution features of the light spot; The coordinates of the geometric center are compared with the preset target point to calculate the absolute position error; The energy distribution characteristics are compared with the standard spot template to calculate the energy distribution similarity. The verification is considered successful when the absolute position error is less than the final position threshold and the energy distribution similarity is greater than the final mass threshold.

[0119] In this embodiment, a first visual sensor or a third visual sensor is used to verify the light spot that meets the preset requirements. The aim is to perform a final quality and position assessment of the light spot formed after the coupling of the first and second closed-loop control. The third visual sensor is a high-speed camera specifically designed for light spot monitoring, capable of accurately capturing light spot information even in complex environments.

[0120] Controlling the first or third vision sensor to acquire an image encompassing the area of ​​the light spot refers to sending instructions from the system control unit to the selected vision sensor, instructing it to capture an image within a specific time and area. These instructions may include parameters such as trigger signals, exposure time, gain, and region of interest (ROI). For example, based on the current projection angle of the laser emitter and the workpiece pose, the approximate position of the light spot within the sensor's field of view is predicted, and a rectangular area including this predicted position is defined as the ROI, thereby efficiently acquiring image data focused on the light spot.

[0121] Processing the area image to extract the geometric center coordinates and energy distribution features of the light spot is a crucial step in the verification process. Specifically, the area image is binarized to separate the light spot region from the background, and then the average coordinates of all pixels in the binarized light spot region are calculated, which is the geometric centroid of the light spot. For energy distribution feature extraction, a statistical histogram of the grayscale values ​​of all pixels within the light spot region can be calculated to characterize its energy distribution.

[0122] The absolute position error is calculated by comparing the geometric center coordinates with the preset target point, aiming to quantify the deviation between the actual and desired landing point of the laser spot. The preset target point is the ideal laser action point determined according to process requirements and workpiece design, and its coordinates are defined in the device coordinate system. During the comparison, the geometric center coordinates of the laser spot extracted from the range image are first accurately transformed to the device coordinate system using pre-calibrated camera intrinsic and extrinsic parameters and coordinate transformation matrix. Then, the Euclidean distance between the transformed actual center coordinates of the laser spot and the preset target point is calculated, which is the absolute position error.

[0123] The energy distribution characteristics are compared with a standard spot template to calculate the energy distribution similarity, aiming to assess whether the spot quality meets the process requirements. The standard spot template can be the energy distribution curve, grayscale image, or a set of feature parameters of an ideal spot obtained in advance through experiments, simulations, or theoretical calculations. During comparison, the extracted spot energy distribution characteristics can be matched with the standard spot template. For example, the correlation coefficient between the two can be calculated as the similarity; the higher the similarity, the closer the spot quality is to the ideal state.

[0124] The verification is considered successful when the absolute position error is less than the final position threshold and the energy distribution similarity is greater than the final quality threshold. This is a dual-judgment mechanism that ensures the spot meets preset requirements in both position and quality dimensions. The final position threshold and final quality threshold are preset based on specific process accuracy requirements and product quality standards. For example, the final position threshold is set to 0.02 mm, and the final quality threshold is set to 0.95 (representing 95% similarity). Laser positioning is considered successful only when the deviation between the actual position of the spot and the target point is within the allowable range, and the energy distribution quality of the spot meets or exceeds the preset standards.

[0125] Through the above technical solution, this application provides a comprehensive spot verification mechanism, effectively solving the problem that traditional verification methods are not comprehensive enough, relying only on a single indicator or simple comparison, and cannot simultaneously and accurately evaluate position error and energy distribution characteristics, leading to deviations or omissions in the verification results. By simultaneously evaluating the position accuracy and energy distribution quality of the spot, the reliability of the verification results is improved.

[0126] In one embodiment of this application, in verifying the light spot that meets the preset requirements using the first visual sensor or the third visual sensor, the verification process further includes verifying the consistency of light spot characteristics using an optical characteristic distribution map.

[0127] Specifically, based on the actual position of the current light spot on the workpiece surface, the expected optical response characteristics of that actual position are extracted from the optical characteristic distribution map, including but not limited to the expected specular reflection intensity and the expected diffuse reflection intensity. In addition to acquiring the geometric center and energy distribution of the light spot, the first or third visual sensor also acquires the actual reflected light intensity of the light spot area. The actual reflected light intensity is compared with the expected reflected light intensity predicted based on the optical characteristic distribution map; Based on the current projection angle and the sensor's observation angle, calculate the deviation of the reflected light intensity between the actual reflected light intensity and the expected reflected light intensity; The verification is deemed successful only if the absolute position error is less than the final position threshold, the energy distribution similarity is greater than the final quality threshold, and the reflected light intensity deviation is less than the preset optical deviation threshold.

[0128] In this embodiment, firstly, after the position and quality of the light spot meet preset requirements, a spatial query is performed on the generated optical characteristic distribution map based on the actual position coordinates of the current light spot on the workpiece surface to extract the expected optical response characteristics at those coordinates. The optical characteristic distribution map stores multi-dimensional optical properties of various points on the workpiece surface, including specular reflection index, diffuse reflection index, and absorptivity index. For a given actual position, the corresponding expected specular reflection intensity and expected diffuse reflection intensity are obtained from the optical characteristic distribution map using an interpolation algorithm. These expected values ​​are theoretical responses calibrated under standard illumination conditions (such as a specific incident angle).

[0129] Secondly, the actual reflected light intensity of the light spot area is acquired using a first or third visual sensor. Specifically, while acquiring the geometric center and energy distribution of the light spot, the pixel grayscale values ​​within the light spot area are converted into physical light intensity values ​​using the sensor's photoelectric response characteristics. The conversion process considers the sensor's exposure time, gain, spectral response curve, and the subtraction of ambient background light. For example, the sensor is pre-calibrated radiometrically to establish a mapping relationship between grayscale values ​​and absolute radiance, thereby obtaining the actual reflected light intensity.

[0130] Then, based on the current laser projection angle and the sensor's observation angle, the expected optical response characteristics are converted into the theoretical reflected light intensity under the current observation direction. Since the expected specular and diffuse reflection intensities stored in the optical characteristic distribution map correspond to specific reference angles (such as the normal direction), the theoretical reflection intensity under the current incident-observation geometry needs to be calculated using a two-way reflection distribution function model. For example, a simplified Phong model can be used, taking the expected specular and diffuse reflection intensities as input and combining them with the current angle to calculate the theoretical reflection intensity. If the optical characteristic distribution map directly stores the full-angle two-way reflection distribution function parameters, the theoretical reflection intensity can be directly retrieved.

[0131] Next, the optical deviation between the actual reflected light intensity and the theoretical reflected light intensity is calculated; this optical deviation quantifies the difference between the actual and expected optical characteristics of the light spot.

[0132] Finally, a preset optical deviation threshold is set, and its calibration method is as follows: Collect a batch of qualified workpieces that are free from pollution and of the same material, complete the positioning under normal process conditions and measure their optical deviation threshold, calculate the optical mean and optical standard deviation, and take the sum of the optical mean and 3 times the optical standard deviation as the upper limit of normal fluctuation of the optical deviation threshold.

[0133] If the optical deviation is less than the optical deviation threshold, it indicates that the actual optical characteristics of the spot are consistent with the expectation and the surface condition of the workpiece is normal. If the optical deviation is greater than or equal to the optical deviation threshold, it indicates that there are abnormalities such as workpiece surface contamination, batch material differences, or errors in the generation of the previous optical characteristic distribution map.

[0134] In the comprehensive judgment stage, the verification is finally deemed qualified only when the absolute position error is less than the final position threshold, the energy distribution similarity is greater than the final quality threshold, and the reflected light intensity deviation is less than the preset optical deviation threshold.

[0135] Through the above technical solution, the supplementary verification step of this application can effectively identify anomalies caused by workpiece surface contamination, batch material differences, or errors in the generation of optical property distribution maps, thereby further improving the reliability of positioning detection.

[0136] Corresponding to the machine vision-based multi-directional laser positioning and detection method in the above embodiments, Figure 2 This is a structural block diagram of a machine vision-based multi-directional laser positioning and detection system according to an embodiment of this application. For ease of explanation, only the parts relevant to the embodiment of this application are shown. References Figure 2 The machine vision-based multi-directional laser positioning and detection system 20 includes: an image analysis module 21, a scheme generation module 22, a first control module 23, a second control module 24, a spot verification module 25, and a data storage module 26.

[0137] The image analysis module 21 is used to determine the pose of the workpiece in the current equipment coordinate system and the optical characteristic distribution map of the workpiece surface based on the initial image information. The initial image information is obtained by acquiring the workpiece based on the first vision sensor. The scheme generation module 22 is used to generate an initial projection scheme based on the workpiece's pose, preset target points, and optical characteristic distribution map; the initial projection scheme includes the projection angle of the laser emitting device pointing to the workpiece surface and the initial laser projection parameters. The first control module 23 is used to control the laser emitting device to project a positioning laser beam onto the workpiece surface according to the initial projection scheme, and to use the second vision sensor to monitor the dynamic changes of the positioning spot during the projection process. The second control module 24 is used to perform a coupled control process of first closed-loop control and second closed-loop control in parallel based on dynamic changes; wherein the first closed-loop control corrects the projection angle of the laser emitting device pointing to the workpiece surface according to the deviation between the actual position of the light spot and the theoretical position; the second closed-loop control adjusts the initial laser projection parameters according to the comparison between the quality index of the light spot and the preset quality threshold. Repeat the first and second closed-loop control as described above until the position deviation and quality index of the light spot meet the preset requirements, and obtain the target projection angle and target laser projection parameters. The spot verification module 25 is used to verify the spot that meets the preset requirements using a first vision sensor or a third vision sensor. The data storage module 26 is used to associate and store the target projection angle, target laser projection parameters, environmental characteristics and workpiece identification information in the database after the verification is qualified.

[0138] See Figure 3 , Figure 3 This is a schematic block diagram of an electronic device provided according to an embodiment of this application. Figure 3 The electronic device 300 in this embodiment may include one or more processors 301, one or more input devices 302, one or more output devices 303, and one or more memories 304. The processors 301, input devices 302, output devices 303, and memories 304 communicate with each other via a communication bus 305. The memories 304 store computer programs, including program instructions. The processors 301 execute the program instructions stored in the memories 304. Specifically, the processors 301 are configured to invoke the program instructions to perform the functions of the modules in the aforementioned device embodiments, for example... Figure 2 The functions of modules 21, 22 and 23 are shown.

[0139] It should be understood that, in the embodiments of this application, the processor 301 may be a central processing unit (CPU), or it may be other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor may be a microprocessor or any conventional processor.

[0140] Input device 302 may include a touchpad, a fingerprint sensor (for collecting the user's fingerprint information and fingerprint orientation information), a microphone, etc., and output device 303 may include a display (LCD, etc.), a speaker, etc.

[0141] The memory 304 may include read-only memory and random access memory, and provides instructions and data to the processor 301. A portion of the memory 304 may also include non-volatile random access memory. For example, the memory 304 may also store device type information.

[0142] In specific implementations, the processor 301, input device 302, and output device 303 described in the embodiments of this application can execute the implementation methods described in any embodiment of the multi-directional laser positioning and detection method based on machine vision provided in the embodiments of this application, or they can execute the implementation methods of the electronic devices described in the embodiments of this application, which will not be repeated here.

[0143] In another embodiment of this application, a computer-readable storage medium is provided. This computer-readable storage medium stores a computer program, which includes program instructions. When executed by a processor, the program instructions implement all or part of the processes in the methods described above. Alternatively, the computer program can instruct related hardware to complete the process. The computer program can be stored in a computer-readable storage medium, and when executed by a processor, it can implement the steps of the various method embodiments described above. The computer program includes computer program code, which can be in the form of source code, object code, executable files, or certain intermediate forms. The computer-readable medium can include any entity or device capable of carrying computer program code, a recording medium, a USB flash drive, a portable hard drive, a magnetic disk, an optical disk, a computer memory, a read-only memory (ROM), a random access memory (RAM), an electrical carrier signal, a telecommunication signal, and a software distribution medium, etc.

[0144] The computer-readable storage medium can be an internal storage unit of the electronic device in any of the foregoing embodiments, such as a hard disk or memory of the electronic device. The computer-readable storage medium can also be an external storage device of the electronic device, such as a plug-in hard disk, smart media card (SMC), secure digital card (SD), flash card, etc., equipped on the electronic device. Furthermore, the computer-readable storage medium can include both internal and external storage units of the electronic device. The computer-readable storage medium is used to store computer programs and other programs and data required by the electronic device. The computer-readable storage medium can also be used to temporarily store data that has been output or will be output.

[0145] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of both. To clearly illustrate the interchangeability of hardware and software, the components and steps of the various examples have been generally described in terms of functionality in the foregoing description. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementations should not be considered beyond the scope of this application.

[0146] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working process of the electronic devices and units described above can be referred to the corresponding process in the foregoing method embodiments, and will not be repeated here.

[0147] In the several embodiments provided in this application, it should be understood that the disclosed electronic devices and methods can be implemented in other ways. For example, the device embodiments described above are merely illustrative; for instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. In addition, the mutual coupling or direct coupling or communication connection shown or discussed may be indirect coupling or communication connection through some interfaces or units, or it may be an electrical, mechanical, or other form of connection.

[0148] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of the embodiments of this application, depending on actual needs.

[0149] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0150] The above are merely specific embodiments of this application, but the scope of protection of this application is not limited thereto. Any person skilled in the art can easily conceive of various equivalent modifications or substitutions within the technical scope disclosed in this application, and these modifications or substitutions should all be covered within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A laser positioning and detection method based on machine vision, characterized in that, include: Based on the initial image information, the pose of the workpiece in the current equipment coordinate system and the optical characteristic distribution map of the workpiece surface are determined. The initial image information is obtained by acquiring the workpiece based on the first vision sensor. An initial projection scheme is generated based on the workpiece's pose, the preset target point, and the optical characteristic distribution map; the initial projection scheme includes the projection angle of the laser emitting device pointing towards the workpiece surface and the initial laser projection parameters; The laser emitting device is controlled to project a positioning laser beam onto the workpiece surface according to the initial projection scheme, and the dynamic changes of the positioning spot during the projection process are monitored by a second vision sensor. Based on the dynamic changes, a coupled control process of first closed-loop control and second closed-loop control is executed in parallel; wherein the first closed-loop control corrects the projection angle of the laser emitting device pointing to the workpiece surface according to the deviation between the actual position of the light spot and the theoretical position; the second closed-loop control adjusts the initial laser projection parameters according to the comparison between the quality index of the light spot and the preset quality threshold. Repeat the first and second closed-loop control as described above until the position deviation and quality index of the light spot both meet the preset requirements, and obtain the target projection angle and target laser projection parameters. The light spot that meets the preset requirements is verified using the first vision sensor or the third vision sensor. When the verification is qualified, the target projection angle, target laser projection parameters, environmental features and workpiece identification information are associated and stored in the database.

2. The laser positioning and detection method based on machine vision according to claim 1, characterized in that, The step of determining the workpiece's pose in the current equipment coordinate system and the optical property distribution map of the workpiece surface based on the initial image information includes: The initial image information is subjected to multi-view 3D reconstruction processing to obtain the 3D point cloud data of the workpiece; The three-dimensional point cloud data is iteratively registered with the preset three-dimensional workpiece model to calculate the six-degree-of-freedom pose parameters of the workpiece in the current equipment coordinate system. Based on the initial image information and the three-dimensional point cloud data, optical property analysis is performed on the workpiece surface to generate an optical property distribution map. Based on the optical characteristic distribution map and the geometric features of the three-dimensional point cloud data, the optical interference region is identified and calibrated. The identification information of the optical interference region is fused into the optical characteristic distribution map to obtain the final optical characteristic distribution map.

3. The laser positioning and detection method based on machine vision according to claim 2, characterized in that, The identification and labeling of optical interference regions based on the geometric features of the optical characteristic distribution map and 3D point cloud data includes: Extract the specular reflection index, diffuse reflection index, and absorptivity index from the optical property distribution map; Based on 3D point cloud data, the local curvature and surface roughness of each sampling point are calculated, and a surface feature vector is constructed. The specular reflection index, diffuse reflection index, absorptivity index and surface feature vector are fused together and input into a pre-trained optical interference recognition model to obtain the confidence probability value of each sampling point belonging to the optical interference region. The confidence probability value is determined according to a preset confidence threshold, an initial set of interference points is selected, and a region growing algorithm is used to merge connected components and smooth edges of the discrete initial interference points to generate the optical interference region.

4. The laser positioning and detection method based on machine vision according to claim 1, characterized in that, The step of generating an initial projection scheme based on the workpiece pose, the preset target point, and the optical characteristic distribution map includes: Based on the workpiece pose and the preset target point, determine the target projection point on the workpiece surface; Based on the optical characteristic distribution map, determine whether the target projection point is located within the optical interference area; If it is located within the optical interference area, the corresponding feedforward compensation parameter is matched from the preset control strategy library according to the type of optical interference, and used as a component of the initial laser projection parameter. If the preset process requirements still cannot be met after compensation by the feedforward compensation parameters, then it is determined whether the target projection point is an unchangeable reference positioning point according to the preset process document. If it is the unchangeable reference positioning point, then maintain the feedforward compensation parameters and record the abnormal marker of the target projection point; If it is not the immutable reference positioning point, then the sampling point with the lowest interference and curvature change less than the preset curvature threshold is searched in the preset neighborhood as the candidate point. If the candidate point is found, it is used as the new target projection point, and the projection angle and initial laser projection parameters are recalculated based on the new target projection point. If no qualified candidate point is found, the target projection point is maintained, and its feedforward compensation parameters are determined according to the unchangeable point strategy. The initial projection scheme is generated based on the target projection point and the initial laser projection parameters.

5. The laser positioning and detection method based on machine vision according to claim 4, characterized in that, The process of searching for sampling points with the lowest interference level and curvature change less than a preset curvature threshold within a preset neighborhood as candidate points includes: The radius of the search neighborhood is determined based on the local surface curvature and optical property distribution at the target projection point, wherein the radius is inversely proportional to the rate of change of local curvature and directly proportional to the confidence level of the optical interference region; Within the search neighborhood, the comprehensive interference index of each sampling point is calculated. The comprehensive interference index is a weighted sum of the specular reflection index, diffuse reflection index, and absorptivity index. The sampling points with the smallest comprehensive interference index and curvature change less than the preset curvature threshold are selected as candidate points.

6. The laser positioning and detection method based on machine vision according to claim 5, characterized in that, After selecting the sampling points with the smallest comprehensive interference index and curvature change less than a preset curvature threshold as candidate points, the method further includes: Calculate the spatial distance between the candidate point and the target projection point; If the spatial distance is greater than a preset distance threshold, a penalty factor is applied to the comprehensive interference index based on the spatial distance to obtain the penalized comprehensive interference index. The priority of the candidate points is re-evaluated, and the point with the smallest overall interference index after penalty is selected as the final candidate point. If the spatial distance of all candidate points is greater than the preset maximum allowable offset threshold, the search is abandoned and an alarm signal is issued or a global replanning process is initiated. If the spatial distance between all candidate points is greater than the preset distance threshold but less than the maximum allowable offset threshold, the search neighborhood radius is expanded and the search is repeated until a candidate point that satisfies the distance constraint and has the smallest comprehensive interference index is found.

7. The laser positioning and detection method based on machine vision according to claim 1, characterized in that, The parallel execution of the coupled control process of the first closed-loop control and the second closed-loop control includes: Obtain the spot position deviation sequence and quality index sequence within the current control cycle; The position deviation sequence and quality index sequence are input into a preset coupled control model, and a first control quantity for angle adjustment and a second control quantity for laser parameter adjustment are decoupled and calculated respectively; wherein the first control quantity includes a compensation term for compensating for spot drift caused by changes in laser parameters, and the second control quantity includes a compensation term for compensating for spot quality attenuation caused by changes in angle. The coupled control model is used to describe the coupled influence of changes in projection angle and laser parameters on the spot position and quality indicators. The angle adjustment mechanism is driven by the first control quantity, and the laser emitting device is driven by the second control quantity to achieve synchronous coupling correction of the projection angle and laser parameters.

8. A multi-directional laser positioning and detection system based on machine vision, characterized in that, include: The image analysis module is used to determine the pose of the workpiece in the current equipment coordinate system and the optical characteristic distribution map of the workpiece surface based on the initial image information, wherein the initial image information is obtained by acquiring the workpiece based on the first vision sensor. The scheme generation module is used to generate an initial projection scheme based on the workpiece's pose, preset target points, and the optical characteristic distribution map; the initial projection scheme includes the projection angle of the laser emitting device pointing to the workpiece surface and initial laser projection parameters. The first control module is used to control the laser emitting device to project a positioning laser beam onto the workpiece surface according to the initial projection scheme, and to use the second vision sensor to monitor the dynamic changes of the positioning spot during the projection process. The second control module is used to perform a coupled control process of first closed-loop control and second closed-loop control in parallel based on the dynamic changes; wherein the first closed-loop control corrects the projection angle of the laser emitting device pointing to the workpiece surface according to the deviation between the actual position of the light spot and the theoretical position; the second closed-loop control adjusts the initial laser projection parameters according to the comparison between the quality index of the light spot and the preset quality threshold. Repeat the first and second closed-loop control as described above until the position deviation and quality index of the light spot both meet the preset requirements, and obtain the target projection angle and target laser projection parameters. A spot verification module is used to verify the spot that meets the preset requirements using the first vision sensor or the third vision sensor. The data storage module is used to associate and store the target projection angle, target laser projection parameters, environmental characteristics, and workpiece identification information in the database after the verification is qualified.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and running on the processor, characterized in that, When the processor executes the computer program, it implements the steps of the method as described in any one of claims 1 to 7.

10. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by a processor, it implements the steps of the method as described in any one of claims 1 to 7.

Images