Crystal oscillator wafer visual identification and mechanical arm cooperative mounting method

Abstract

The present application relates to the technical field of automatic mounting of crystal vibration wafers, and discloses a crystal vibration wafer visual recognition and mechanical arm cooperative mounting method, which comprises the following steps: collecting two-dimensional and three-dimensional information of the crystal vibration wafer by using a multispectral visual sensing fusion system, outputting three-dimensional posture estimation data containing a center point three-dimensional coordinate and a direction vector through a deep convolutional neural network; generating a time-optimal trajectory based on the data and the current position of the mechanical arm, and performing feedforward compensation by using an extended Kalman filtering algorithm to eliminate time lag errors; moving the crystal vibration wafer towards the target mounting plane after grabbing the crystal vibration wafer, monitoring the contact force, and switching to a zero force mode to realize flexible contact when the threshold is reached; and in the zero force mode, iteratively adjusting the pose according to the multi-sensor contact force distribution variance until the standard is met, and the mounting is completed. The present application realizes high-precision and non-damage compliant assembly of micro-nano crystal vibration wafers.

Description

Technical Field

[0001] This invention relates to the field of automated packaging and micro / nano manufacturing assembly equipment technology, specifically to a method for co-loading crystal oscillator wafers using visual recognition and robotic arms. Background Technology

[0002] The electronics and information industry's demand for high-frequency, high-stability signal generating units in communication terminals and automotive electronics continues to rise, driving the evolution of crystal oscillators towards miniaturization and ultra-thinness. As the core frequency control unit, the crystal oscillator chip is typically made of quartz crystal, which is highly susceptible to stress fracture, and its surface is coated with a high-precision conductive film. The crystal oscillator assembly process requires precisely mounting the unpackaged, tiny crystal oscillator chip onto a target surface pre-coated with conductive adhesive or solder. The degree of physical overlap directly determines the parasitic capacitance value, and the physical pressure distribution on the assembly surface directly determines the internal resonant resistance and aging lifespan of the device.

[0003] Current crystal oscillator production lines generally use surface mount equipment based on isolated vision modules. Existing mounting systems rely on vision sensors to take static planar photographs of the loading carrier tape, calculate the center coordinate deviation, and generate a mechanical path for unidirectional open-loop pressing from the current coordinates to the target coordinates. The micro-nano level assembly environment is subject to various non-ideal physical factors. From the exposure and acquisition of images by industrial cameras, the processing of pixel matrices by vision algorithms, to the issuance of driver commands, there is an unavoidable physical communication delay. When a robotic arm performs a pressing action based on outdated coordinates generated by this delay during high-speed movement, it inevitably produces micron-level pose misalignment. The cutting thickness difference of the crystal oscillator wafer and the fluid viscosity damping of the base adhesive have high uncertainty. Existing actuators based on position-based open-loop pressing strategies or one-dimensional torque alarm shutdowns cannot dissipate impact energy within the microsecond-level time window of rigid contact. Rigid pressing not only easily leads to stress concentration at the edges of the crystal oscillator wafer, causing microscopic defects, but also causes uneven force extrusion of the base adhesive, resulting in microscopic tilting, significantly reducing the yield and consistency of high-end frequency control components. Summary of the Invention

[0004] The purpose of this invention is to provide a method for co-loading crystal oscillator wafers with visual recognition and robotic arms, which solves the technical problems of spatial alignment misalignment caused by communication delay and microscopic damage to brittle crystal oscillator wafers caused by rigid contact in existing wafer loading equipment.

[0005] To solve the above-mentioned technical problems, the technical solution adopted by the present invention is as follows: A method for coordinating crystal oscillator wafer visual recognition and robotic arm assembly includes the following steps: The crystal oscillator wafers are transported to the testing station via a feeding mechanism. The two-dimensional image information and three-dimensional spatial coordinate information of the crystal oscillator wafer are acquired using a multispectral visual sensing fusion system; The two-dimensional image information and the three-dimensional spatial coordinate information are processed by a deep convolutional neural network to obtain feature vectors. Based on the feature vectors, the three-dimensional attitude estimation data of the crystal oscillator wafer is output. The three-dimensional attitude estimation data includes the three-dimensional coordinates of the center point, as well as the length direction vector, the width direction vector, and the thickness direction vector. A spatial path curve is constructed based on the three-dimensional attitude estimation data and the current position of the robotic arm end effector. The spatial path curve is then smoothed using cubic spline interpolation to generate the time-optimal trajectory of the robotic arm end effector. The end effector of the robotic arm is controlled to move along the time-optimal trajectory. The current position of the end effector is predicted using the extended Kalman filter algorithm to obtain the predicted position. The actual position of the end effector is compared with the predicted position to obtain the deviation value. The deviation value is the difference between the predicted position and the actual position. The deviation value is used as a feedforward correction amount and directly superimposed on the time-optimal trajectory to obtain the compensated trajectory. The end effector of the robotic arm is controlled to move along the compensated trajectory to the target grasping position, and the crystal oscillator is grasped by the adaptive vacuum gripping mechanism. The end effector of the robotic arm is controlled to carry the crystal oscillator towards the target placement plane. During the movement towards the target placement plane, the contact force output by multiple contact force sensors installed on the end effector of the robotic arm is monitored. When the contact force reaches the first contact force threshold, the end effector of the robotic arm is controlled to reduce the descent speed; when the contact force reaches the second contact force threshold, the end effector of the robotic arm is controlled to switch to zero force mode so that the crystal oscillator wafer forms a flexible contact with the target placement plane. In the zero-force mode, the current contact force distribution is constructed based on the contact forces output by multiple contact force sensors. The position deviation and attitude deviation of the end effector of the robotic arm are calculated based on the current contact force distribution. The position and attitude deviation of the end effector of the robotic arm are adjusted according to the position deviation and attitude deviation until the variance corresponding to the current contact force distribution is less than the set variance value, and the crystal oscillator wafer is assembled.

[0006] As a preferred embodiment of the present invention, a multispectral visual sensing fusion system is used to acquire two-dimensional image information and three-dimensional spatial coordinate information of the crystal oscillator wafer, specifically including: The ring light source, coaxial light source and near-infrared light source integrated in the multispectral visual sensing fusion system are used to perform independent multispectral frequency band lighting to remove highlights, enhance contour edges and map metal coating layers on the crystal oscillator wafer. Using a first industrial camera and a second industrial camera arranged in a spatially separate manner, after receiving a global shutter synchronization trigger pulse, they simultaneously capture the multispectral pixel matrix and discrete point cloud dataset of the crystal oscillator wafer from different perspectives within the same sampling time window, which are respectively used as the two-dimensional image information and the three-dimensional spatial coordinate information.

[0007] As a preferred embodiment of the present invention, the two-dimensional image information and the three-dimensional spatial coordinate information are processed based on a deep convolutional neural network to obtain a feature vector, and the three-dimensional attitude estimation data of the crystal oscillator wafer is output based on the feature vector, specifically including: The two-dimensional image information is input into the backbone network of the deep convolutional neural network to perform multi-layer two-dimensional convolutional filtering operations, extracting local pixel gradient change features of the image to output an initial feature vector; the initial feature vector is then processed by a cross-stage local network module to perform channel segmentation and non-linear activation dimensionality reduction mapping, retaining strong semantic response channels and removing weak response channels to obtain a low-dimensional shallow feature tensor. The shallow feature tensor and the three-dimensional spatial coordinate information are synchronously transmitted to the path aggregation network. The two-dimensional texture features in the shallow feature tensor and the depth gradient features in the three-dimensional spatial coordinate information are aligned in the coordinate system and spliced ​​across layers in the manifold space through the top-down data interaction bus, and the aggregated feature tensor that fuses multimodal optical features is output. The aggregated feature tensor is input into the attitude estimation branch network, and a nonlinear regression operation is performed on the aggregated feature tensor through a multilayer perceptron fully connected model to reduce the dimensionality of the high-dimensional tensor space and project it onto the Euclidean physical space, thereby solving for the multidimensional continuous variable matrix that serves as the three-dimensional attitude estimation data.

[0008] As a preferred embodiment of the present invention, a spatial path curve is constructed based on the three-dimensional attitude estimation data and the current position of the robotic arm end effector. The spatial path curve is then smoothed using cubic spline interpolation to generate the time-optimal trajectory of the robotic arm end effector. Specifically, this includes: At the moment the shutters of the first and second industrial cameras are triggered, a time stamp is embedded in the two-dimensional image information and the three-dimensional spatial coordinate information by a hardware timestamp generation module, and the time stamp is synchronized to the motion controller using a precise time protocol. The motion controller reads the three-dimensional attitude estimation data and the current position, constructs a spatial path curve in a spatial rectangular coordinate system, and smooths each discrete control node of the spatial path curve through a cubic spline interpolation algorithm, so that the first and second derivatives of the velocity curve remain continuous. The motion controller uses the system's rated power and the maximum angular velocity of the joint as constraints, and minimizes the total motion time as the optimization objective function. It solves a sequential quadratic programming subproblem in the time domain to generate a set of discrete interpolation points for the position coordinates of the time-optimal trajectory.

[0009] As a preferred embodiment of the present invention, the extended Kalman filter algorithm is used to construct a state prediction update model for performing feedforward position prediction as follows:

[0010] in, This is the predicted position vector corresponding to the end effector of the robotic arm at the current sampling time; The state transition matrix characterizes the dynamic decay of the system; The state vector of the end effector of the robotic arm at the previous discrete sampling time; For servo control input transformation matrix; This is the control command vector input to the joint drive motor; A transformation operator that maps the three-dimensional pose parameters of the crystal wafer contained in a multidimensional continuous variable matrix to an observation correction vector of the same dimension as the state vector of the robotic arm; The multidimensional continuous variable matrix is ​​the independent output of the attitude estimation branch network.

[0011] As a preferred embodiment of the present invention, the measured position of the end effector of the robotic arm is compared with the predicted position to obtain a deviation value, and the deviation value is used as a feedforward correction amount to compensate the time-optimal trajectory to obtain the compensated trajectory, specifically including: The motion controller internally constructs a subtraction comparator to perform a matrix difference comparison operation between the measured position and the predicted position, calculates the difference between the predicted position and the measured position, and extracts the absolute deviation value in the spatial rectangular coordinate system. The absolute deviation value is used as a feedforward correction amount and weighted over each discrete interpolation coordinate matrix corresponding to the time-optimal trajectory to regenerate the compensated trajectory, thereby offsetting the millisecond-level communication delay error caused by the visual algorithm calculation within the time mark transmission period.

[0012] As a preferred embodiment of the present invention, the crystal oscillator wafer is gripped by an adaptive vacuum holding mechanism, specifically including: The adaptive vacuum holding mechanism calculates the required vacuum negative pressure airflow cross-sectional area and holding pressure parameters based on the length direction vector and width direction vector extracted from the multidimensional continuous variable matrix. The adaptive vacuum holding mechanism controls the pneumatic proportional servo valve to open, adjusting the pipeline air pressure to the set negative pressure state corresponding to the holding pressure parameter, thereby completing the non-destructive adsorption and gripping of the crystal oscillator wafer surface.

[0013] Furthermore, controlling the segmented deceleration and zero-force mode switching of the robotic arm's end effector specifically includes: The plurality of contact force sensors continuously and dynamically monitor the contact force generated when the bottom surface of the crystal oscillator chip touches the target placement plane at a set sampling frequency, and feed the contact force as a continuous analog voltage signal back to the motion controller; When the motion controller detects that the contact force has reached the first contact force threshold, it blocks the speed feedforward loop and controls the end effector of the robotic arm to reduce the descent speed to a constant detection speed according to the exponential decay curve. When the motion controller detects that the contact force reaches the second contact force threshold, it triggers the highest priority hard interrupt protection program, controls the end effector of the robotic arm to be forced into zero force mode, cuts off the target position holding current of all spatial joint motors, and makes the end effector of the robotic arm passively follow and give way to the external physical force field.

[0014] In the zero-force mode, the pose of the robotic arm's end effector is adjusted based on the position and attitude deviations of the current contact force distribution. Specifically, this is achieved using a gradient descent pose compensation model based on the fusion of the three-dimensional pose estimation data, the predicted position, and the current contact force distribution using multi-dimensional tensors. The control equations of the gradient descent pose compensation model are constructed as follows:

[0015] in, For the robotic arm end effector in the first The pose correction vector generated in the next iteration cycle; In the first The pose correction vector of the memory-resident record in the next iteration cycle; An adaptive learning rate is used to control the micro-motion offset step size; The gradient guiding variable is calculated by expanding the partial derivative of the variance with respect to the pose correction vector. The visual anchor is assigned a deviation prevention penalty weight coefficient; This is a mapping operator that projects the Cartesian physical position state transformation onto the Euler angle attitude matrix; For the first The predicted position vector corresponding to the predicted position during the next pose iteration loop.

[0016] As a preferred embodiment of the present invention, the adaptive learning rate in the gradient descent pose compensation model is dynamically adjusted nonlinearly by integrating an exponential decay model, and the operational control equation of the adaptive learning rate update model is constructed as follows:

[0017] in, To enter the Dynamic learning rate during sub-physical deflection calculation iteration; This is the initial, fixed learning rate constant for the system. This is the time decay constant parameter; Count the number of physical optimization iterations currently invoked; It is a logical step function operator with hysteresis; Set a value for the conduction threshold variance. Set the turn-off threshold variance value, and ; For the first The variance of the output derived from the next physical iteration operation; the system is also configured to, only when the The dynamic learning rate will only be adjusted when the number of sampling periods during which the output remains zero exceeds a preset lockout delay threshold. Force the value to zero and permanently lock it.

[0018] As a preferred embodiment of the present invention, after the crystal oscillator wafer is assembled, the following quality inspection and sorting steps are further included: The contact-type micro-force measurement probe is activated, and a pre-tightening verification force in the vertical direction is slowly applied to the upper surface of the crystal oscillator wafer at a set thrust slope through the internally integrated voice coil motor. The vertical micro-displacement of the crystal oscillator wafer surface is dynamically acquired using an interferometric fiber displacement sensor, and the uniform and void-free spreading of the conductive adhesive on the bottom surface is determined based on the mapping curve between the pre-tightening verification force and the vertical micro-displacement. If the vertical micro-displacement exceeds the safety deformation threshold before the pre-tightening verification force reaches the set peak value, the crystal oscillator wafer will be identified as a defective product with bonding defects and corresponding defective product marking data will be generated. The central computing and processing unit converts the defective product marking data into a diversion control electrical signal and sends it to the high-frequency electromagnetic reversing valve of the downstream node. This controls the pneumatic push rod mechanism to perform a lateral pushing action, pushing the defective crystal oscillator wafer as a whole into the side scrap recycling container, while the verified qualified devices flow into the good product container.

[0019] In addition, the present invention also discloses a crystal oscillator wafer visual recognition and robotic arm collaborative mounting system for performing a crystal oscillator wafer visual recognition and robotic arm collaborative mounting method as described above, the collaborative mounting system comprising: The feeding mechanism is used to transport crystal oscillator wafers to the testing station; A multispectral visual sensing fusion system is used to acquire two-dimensional image information and three-dimensional spatial coordinate information of the crystal oscillator wafer; The central computing and processing unit has a built-in deep convolutional neural network, which is used to process the two-dimensional image information and the three-dimensional spatial coordinate information to output three-dimensional pose estimation data and perform contact force distribution variance extraction calculation. The motion controller is bidirectionally electrically connected to the central computing and processing unit, and has a built-in extended Kalman filter prediction module for solving the time-optimal trajectory and the compensated trajectory, and performing feedforward position prediction calculations. The robotic arm has an end effector cascaded onto its end flange. The end effector is composed of an adaptive vacuum holding mechanism, multiple contact force sensors arranged in a ring array, and a contact micro-force measurement probe. It is used to perform precise, stress-free, compliant, and cooperative bonding under the drive of the motion controller.

[0020] As a preferred embodiment of the present invention, the contact-type micro-force measurement probe is internally encapsulated with a miniature voice coil motor capable of outputting linearly increasing electromagnetic thrust, and an interferometric fiber optic displacement sensor that forms an optical path difference variation between the reference end face and the reflective surface of the wafer through multiple coherent beams. The signal output end of the interferometric fiber optic displacement sensor is cascaded with the deformation verification data sampling interface of the central computing processing unit to form an absolute data transmission feedback closed loop for physical bonding strength.

[0021] Compared with the prior art, the present invention has the following beneficial effects: This invention combines a multispectral visual sensing fusion system with a deep convolutional neural network to extract the two-dimensional texture, three-dimensional contour, and internal coating coordinates of the crystal oscillator wafer from the optical source, calculating attitude estimation data that includes three-dimensional spatial position and full attitude direction vectors. Based on this, hardware timestamps and an extended Kalman filter algorithm are used to feedforward compensate for communication delays, transforming outdated visual observations into real-time correction trajectories for the robotic arm's end effector. This eliminates the accumulated spatial misalignment errors caused by data transmission and algorithm computation delays, ensuring precise synchronization between the grasping action and the moving crystal oscillator wafer during high-speed motion.

[0022] In the contact assembly stage, this invention abandons the open-loop control strategy of rigid position pressing and designs a force control closed-loop logic based on a multi-quadrant contact force sensing array and a zero-force compliance mode. By switching between segmented speed reduction and impedance control modes, the crystal oscillator chip can passively and adaptively conform to the micro-geometric tilt angle of the base surface the instant it contacts the conductive adhesive fluid, thus physically eliminating stress concentration and micro-chipping at the edges of the brittle quartz material.

[0023] Furthermore, in the pose fine-tuning optimization stage of the zero-force mode, this invention introduces a gradient descent pose compensation model based on visual absolute anchoring, and uses a step function with hysteresis logic to nonlinearly decay and eventually lock the adaptive learning rate. While the force distribution variance converges, the servo micro-motion loop is shut down, eliminating the high-frequency mechanical vibration near the equilibrium point in traditional force control algorithms, ensuring uniform spreading and stress-free curing of the adhesive layer on the bottom surface of the crystal oscillator wafer. In addition, in the post-assembly process, micro-force verification and secondary visual re-inspection using interferometric fiber displacement sensing are integrated. This achieves online full inspection and automatic sorting of latent defects such as conductive adhesive voids and poor soldering in a non-destructive manner, constructing a reliable production chain from optical capture, dynamic gripping, compliant mounting to physical quality closed loop. Attached Figure Description

[0024] To more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings used in the embodiments will be briefly introduced below. It should be understood that the following drawings only show some embodiments of the present invention and should not be regarded as a limitation of the scope. For those skilled in the art, other related drawings can be obtained from these drawings without creative effort.

[0025] Figure 1 This is an overall architecture diagram of the system described in this invention.

[0026] Figure 2 This is an overall flowchart of the method described in this invention.

[0027] Figure 3 This is a flowchart of the quality inspection and sorting process described in this invention. Detailed Implementation

[0028] In the following description, only certain exemplary embodiments are briefly described. As those skilled in the art will recognize, the described embodiments can be modified in various ways without departing from the spirit or scope of the embodiments of the invention. Therefore, the drawings and description are considered to be exemplary in nature and not restrictive.

[0029] The following is in conjunction with the appendix Figures 1-3 The embodiments of the present invention will be described in detail below.

[0030] Example 1: This example provides a crystal oscillator wafer visual recognition and robotic arm collaborative wafer mounting system, including: The feeding mechanism is used to transport crystal oscillator wafers to the testing station; A multispectral visual sensing fusion system is used to acquire two-dimensional image information and three-dimensional spatial coordinate information of the crystal oscillator wafer; The central computing and processing unit has a built-in deep convolutional neural network, which is used to process the two-dimensional image information and the three-dimensional spatial coordinate information to output three-dimensional pose estimation data and perform contact force distribution variance extraction calculation. The motion controller is bidirectionally electrically connected to the central computing and processing unit, and has a built-in extended Kalman filter prediction module for solving the time-optimal trajectory and the compensated trajectory, and performing feedforward position prediction calculations. The robotic arm has an end effector cascaded onto its end flange. The end effector is composed of an adaptive vacuum holding mechanism, multiple contact force sensors arranged in a ring array, and a contact micro-force measurement probe. It is used to perform precise, stress-free, compliant, and cooperative bonding under the drive of the motion controller.

[0031] This embodiment is applied to a large-scale automated assembly line for miniaturized surface-mount crystal oscillators. The physical entity involved in the mounting process is a crystal wafer with dimensions at the sub-millimeter level and an internal high-precision conductive metal film layer.

[0032] The feeding mechanism performs the task of oriented array queuing of crystal oscillator wafers. It breaks the static friction lock of the crystal oscillator wafers through a high-frequency micro-vibration chassis and uses physical guide channels of a set width to restrict the lateral displacement freedom of the wafers, completing the interval-based fixed-distance transport. The moving crystal oscillator wafers are transported to the inspection station, forming the physical input basis for optical recognition. Due to the extremely high light transmittance and edge refractive index of quartz crystal, conventional monochromatic light sources cannot accurately obtain the three-dimensional outer contour and the relative position of the internal coating layer.

[0033] The multispectral visual sensing fusion system performs omnidirectional optical feature stripping on crystal oscillator wafers transported to the inspection station. The system integrates an array of light sources across different frequency bands. It controls a ring light source to excite uniform broadband diffuse reflection, neutralizing localized high-brightness areas on the quartz crystal surface and restoring scratches and contamination features. Simultaneously, it activates a coaxial parallel probe light wave; this parallel light wave, illuminating the wafer edge, generates a sharp pixel contrast gradient, solidifying the external physical contours. Finally, it activates a near-infrared light source, utilizing the physical penetration effect of infrared light on quartz material to map the absolute coordinates of the metal coating layer encased within the quartz.

[0034] The first and second industrial cameras are installed from different spatial perspectives. The multispectral vision sensing fusion system synchronously sends global shutter pulse signals to the first and second industrial cameras via a hardware trigger harness. The first and second industrial cameras complete exposure within the same microsecond time window, acquiring two-dimensional image information and three-dimensional spatial coordinate information, including the wafer's external contour, surface texture, and the relative position of the internal coating. The two-dimensional image information consists of a three-channel pixel matrix of multispectral channels; the three-dimensional spatial coordinate information consists of a point cloud dataset generated by epipolar constraint matching based on the principle of binocular stereo vision. As a digital snapshot of the physical world, the two-dimensional image information and the three-dimensional spatial coordinate information need to be normalized to the [0,1] interval before being input into the deep convolutional neural network. The point cloud data in the three-dimensional spatial coordinate information needs to be registered to a unified camera coordinate system and provide the underlying data source to the subsequent control system composed of the central computing processing unit and the motion controller.

[0035] After acquiring the underlying optical data source, the central computing unit needs to transform the discrete pixel array and spatial point cloud into physical boundary parameters that can directly drive the joint motors of the robotic arm. This dimensional transformation process is performed based on a deep convolutional neural network. The deep convolutional neural network has made an architectural replacement for traditional visual recognition algorithms, abandoning the open-loop paradigm that relies on manually setting grayscale thresholds and edge extraction operators.

[0036] Two-dimensional image information and three-dimensional spatial coordinate information are synchronously input to the low-level data interface of the deep convolutional neural network. The deep convolutional neural network calls its built-in backbone network to perform two-dimensional convolutional filtering on the two-dimensional image information, extracting local pixel gradient change features and outputting an initial feature vector. Since the initial feature vector has too high a dimension and contains a lot of redundant background information, the deep convolutional neural network uses a cross-stage local network module to perform channel splitting and nonlinear activation dimensionality reduction mapping on the initial feature vector, retaining strong semantic response channels and eliminating weak response channels, thus obtaining a low-dimensional shallow feature tensor.

[0037] Shallow feature tensors and 3D spatial coordinate information are fed into a path aggregation network. The path aggregation network establishes a top-down data interaction bus, aligning the depth gradient features from the 3D spatial coordinate information with the 2D texture features from the shallow feature tensors within the manifold space and performing cross-layer concatenation operations. The output is an aggregated feature tensor that fuses multimodal optical information. Each data element in the aggregated feature tensor possesses dual attributes: 2D pixel semantics and 3D depth information.

[0038] The aggregated feature tensor is passed to the attitude estimation branch network. This branch network uses a multilayer perceptron to perform a fully connected nonlinear regression operation on the aggregated feature tensor, reducing the high-dimensional tensor space to a specific Euclidean physical space and calculating a multidimensional continuous variable matrix. This multidimensional continuous variable matrix constitutes the core data carrier of this invention: the three-dimensional attitude estimation data. The three-dimensional attitude estimation data includes not only the three-dimensional coordinates of the crystal oscillator's center point but also the length, width, and thickness vectors describing the crystal's spatial deflection state. This three-dimensional attitude estimation data forms the absolute physical anchor point for the subsequent spatial interception strategy formulated by the robotic arm actuator.

[0039] In the mechanical motion generation stage, the process from exposure by the first industrial camera to the output of 3D pose estimation data by the pose estimation branch network inevitably consumes system computation time. During this time period, the crystal oscillator wafer on the conveyor belt has already undergone physical slippage displacement. If drive commands are issued directly based on outdated 3D pose estimation data, the robotic arm will either grab air or crush the wafer edges.

[0040] To eliminate temporal coordinate misalignment, the hardware timestamp generation module adds a low-level time stamp to the 2D image information and 3D spatial coordinate information at the instant the shutters of the first and second industrial cameras are triggered. The hardware timestamp generation module uses a precise time protocol to synchronize the data packet containing the time stamp to the motion controller. The motion controller reads the 3D attitude estimation data and the current position coordinates fed back by the encoder inside the robotic arm's end effector, constructing a spatial path curve in a Cartesian coordinate system connecting the current position with the 3D coordinates of the center point in the 3D attitude estimation data.

[0041] To avoid high-frequency current surges and mechanical vibrations in the motors of the robotic arm's joints during startup and shutdown, the motion controller uses a cubic spline interpolation algorithm to smooth the discrete control nodes of the spatial path curve, ensuring the continuity of the first derivative (acceleration) and second derivative (jerk) of the velocity curve. The motion controller uses the robotic arm system's rated power and maximum angular velocity as constraints, and minimizes the total motion time as the optimization objective function. It solves a sequential quadratic programming subproblem in the time domain to generate the time-optimal trajectory for driving the robotic arm's end effector to perform the grasping action. The time-optimal trajectory is stored in the motion controller's register as a set of position coordinates with millisecond-level interpolation periods.

[0042] The motion controller controls the robotic arm's end effector to move along a time-optimal trajectory. Within the execution cycle of the physical motion, to eliminate millisecond-level communication delay errors caused by time stamp transmission, the motion controller internally calls an extended Kalman filter algorithm in real time.

[0043] The Extended Kalman Filter (EKF) algorithm uses the internal encoder state parameters of the robotic arm's end effector as the basis for prediction, and simultaneously integrates the 3D attitude estimation data output by the aforementioned attitude estimation branch network as a high-confidence observation constraint boundary to perform high-frequency feedforward prediction of the current position of the robotic arm's end effector. Since the conventional Kalman filter equations only rely on a linear dynamic system model, this embodiment creatively and deeply improves the state prediction partial differential equations by directly injecting the visual multidimensional continuous variable matrix as the state constraint boundary into the state estimation logic. The state prediction update model of the EKF algorithm is constructed as follows:

[0044] in, This is the predicted position vector of the robotic arm's end effector at the current moment; The state transition matrix characterizes the dynamic decay of the system; This is the state vector of the end effector of the robotic arm at the previous discrete sampling time. For servo control input transformation matrix; This is the control command vector input to the joint drive motor; A transformation operator that maps the three-dimensional pose parameters of the crystal wafer contained in a multidimensional continuous variable matrix to an observation correction vector of the same dimension as the state vector of the robotic arm; This is a multidimensional continuous variable matrix that is independently output by the attitude estimation branch network.

[0045] By calling the state prediction update model, the extended Kalman filter algorithm compensates for the physical space coordinate drift caused by the visual computation cycle through feedforward compensation, and outputs the predicted position representing the true correction coordinates.

[0046] The motion controller internally calls a subtraction comparator. This comparator performs a subtraction comparison between the measured position of the robotic arm's end effector and the predicted position output by the extended Kalman filter algorithm, calculating and extracting the absolute deviation value. This deviation value is the difference between the predicted and measured positions. The motion controller uses this absolute deviation value as a feedforward correction compensation amount, directly weighting and superimposing it onto each discrete interpolated coordinate point in the time-optimal trajectory to regenerate the compensated trajectory.

[0047] The motion controller precisely controls the robotic arm's end effector to move along the compensated trajectory, accurately reaching the dynamically corrected target grasping position. Upon reaching the target grasping position, the adaptive vacuum gripping mechanism calculates the required vacuum negative pressure airflow cross-sectional area and gripping pressure parameters based on the length and width vectors extracted from the multidimensional continuous variable matrix. The adaptive vacuum gripping mechanism then controls the pneumatic proportional servo valve to open the corresponding gripping pressure parameter, grasping the crystal oscillator wafer.

[0048] The robotic arm's end effector, carrying a vacuum-adsorbed crystal oscillator, detaches from the detection station and performs a large-stroke spatial traversal motion towards the target placement plane. When the robotic arm's end effector carrying the crystal oscillator approaches the micro-descent spatial threshold of the target placement plane, the underlying control loop of the motion controller must be forcibly disconnected from the rigid servo position closed-loop control and switch to a flexible impedance following control mechanism.

[0049] Between the nozzle mounting base and the end effector flange of the robotic arm's adaptive vacuum holding mechanism, multiple contact force sensors are mounted in a circular array. Each contact force sensor corresponds to a quadrant region of the nozzle, and the detection value of each sensor has a linear relationship with the magnitude of the contact force on the corresponding region of the crystal oscillator wafer's bottom surface. During the robotic arm's end effector's vertical micro-movement towards the target placement plane, the multiple contact force sensors continuously and dynamically monitor the contact force generated the instantaneously when the bottom surface of the crystal oscillator wafer touches the fluid layer of the target placement plane at a high-frequency sampling rate.

[0050] The contact force sensor feeds back the contact force collected in real time to the motion controller as a continuous analog voltage signal. When the motion controller detects that the contact force has reached the first contact force threshold, it immediately blocks the original velocity feedforward calculation loop and controls the end effector of the robotic arm to reduce its descent speed to a constant detection speed according to an exponential decay curve. This prevents the instantaneous shear force of the fluid on the contact surface from damaging the topological uniformity of the micron-sized silver powder particles inside the underlying conductive adhesive.

[0051] The robotic arm's end effector continuously descends at a probing speed. When the motion controller detects that the contact force has reached the second contact force threshold, the motion controller immediately triggers the highest priority hard interrupt protection program, controlling the robotic arm's end effector to forcibly switch to zero-force mode.

[0052] Zero-force mode is defined as cutting off the dwell current at the target position of all spatial joint motors of the robotic arm, forcing the servo drive to passively follow and yield to the external physical force field. In zero-force mode, the crystal oscillator chip, relying on its own mass and gravity and the residual inertia of the micro-system, forms a flexible contact with the target placement plane without internal tension.

[0053] In zero-force mode, due to the machining tolerances of the base metal pins or the geometric tilt angle of the base assembly slot, the four rectangular corners of the crystal oscillator wafer are prone to uneven force distribution, resulting in microscopic skew. To quantify the multidimensional physical skew into digital control parameters, the central computing unit constructs a current contact force distribution reflecting the microscopic pressure topology of the wafer based on the discrete contact force values ​​output by multiple contact force sensors in the same clock slice.

[0054] The central processing unit extracts the variance of the data corresponding to the current contact force distribution. The variance directly and physically maps the microscopic contact flatness and fit between the bottom surface of the crystal oscillator wafer and the conductive fluid layer. If the variance corresponding to the current contact force distribution is too large, the central processing unit reverse-analyzes the position and attitude deviations of the robotic arm's end effector in each axis of the three-dimensional Cartesian coordinate system based on the current contact force distribution.

[0055] The motion controller regenerates the micro-motion compensation current based on the position and attitude deviations, adjusting the spatial pose of the robotic arm's end effector to change the stress application topology of the crystal oscillator on the target placement plane. The pose adjustment action triggers the contact force sensor to output a new contact force array, and the central computing unit iteratively calculates the new variance.

[0056] The system repeatedly executes an iterative closed-loop instruction stream for physical pose fine-tuning and digital variance verification until the variance corresponding to the current contact force distribution converges and is less than the variance setting value pre-stored in memory. After the variance convergence condition is met, the adaptive vacuum holding mechanism cuts off the negative pressure air source flow channel, and the crystal oscillator wafer is detached from the robotic arm nozzle, completing the stress-free wafer mounting operation.

[0057] Example 2: This example is a further optimization based on Example 1. In this example, a deep mathematical control theory model was reconstructed for the specific iterative logic of adjusting the position and attitude deviation of the robotic arm end effector in zero-force mode, so as to eliminate the risk of high-frequency physical resonance or fatigue fracture that may be triggered by the brittle crystal oscillator wafer during the fine posture grinding process.

[0058] During the zero-force mode fine-tuning and compensation phase, the robotic arm's cantilever mechanical structure no longer forcibly anchors to absolutely fixed coordinates in external space. Instead, it completely conforms to the geometric tilt angle of the target placement plane, performing a floating physical envelope fit. The motion controller frequently polls the pressure digital values ​​uploaded by the contact force sensors in various dimensions, reconstructing the current contact force distribution based on the acquired multiple contact forces. The central computing and processing unit continuously extracts the variance parameter corresponding to the current contact force distribution.

[0059] If the central computing unit finds a significantly large discrepancy, it indicates that the crystal oscillator wafer is in a partially warped, suspended state or in a state of contact damage at an extremely high stress point with the target placement plane. The motion controller immediately resets the current limiting closed loop inside the servo driver, activates the high-frequency power fine-tuning subroutine, and forcibly applies physical pressure to the warped end face by precisely changing the three-dimensional yaw angle of the robotic arm's end effector.

[0060] To ensure efficient convergence of the fine-tuning optimization process of the robotic arm's end effector within a very short wafer curing time window without causing system control divergence, this embodiment creatively introduces a gradient descent pose compensation model based on visual absolute anchoring. The gradient descent pose compensation model fuses the 3D pose estimation data locked at the front end of the multispectral visual sensing fusion system, the predicted position output by the extended Kalman filter algorithm, and the current contact force distribution characterizing the uniformity of microscopic forces through multidimensional tensor product fusion. The specific gradient descent pose compensation model calculation control equations are derived and constructed as follows:

[0061] in, For the end effector of the robotic arm in the first The pose correction vector generated in each iteration operation cycle; For the end effector of the robotic arm in the first The pose correction vector recorded in memory during each iteration operation cycle; An adaptive learning rate is used to control the micro-motion offset step size; The gradient guiding variable is calculated by expanding the partial derivative of the variance with respect to the pose correction vector. The visual anchoring anti-deviation penalty weight coefficient has a value range of 0.1 to 0.5 and is pre-calibrated according to the assembly accuracy requirements; It is a multidimensional continuous variable matrix; This is a mapping operator that projects the Cartesian physical position state transformation onto the Euler angle attitude matrix; For the first The predicted position vector corresponding to the predicted position during the next pose iteration loop.

[0062] A deep analysis of the internal physical traction mechanism control mechanism of the gradient descent pose compensation model: The second term on the right side of the equation introduces the underlying variance negative feedback gradient descent mechanism, which forces the servo motor to drive the end effector of the robotic arm to continuously deflect in the three-dimensional direction with the least physical contact stress and the most uniform topological distribution; The third term on the right side of the equation is independently introduced into the closed-loop control loop as a spatial absolute orientation anchoring penalty term, which forcibly restrains and fine-tunes the search path, ensuring that any extremely small cumulative amount of optimization spatial deflection will never cause the macroscopic physical coordinates of the end effector of the robotic arm to deviate from the position of the absolute center assembly reference point pre-locked by the multispectral vision sensing fusion system.

[0063] Conventional constant step-size gradient descent algorithms inevitably generate continuous high-frequency oscillations near the physical equilibrium surface when the error cost function approaches the optimal region of minimum value. These mathematical oscillations manifest as highly destructive high-frequency jitter at the micro / nano-scale encapsulated mechanical execution level. To eliminate high-frequency physical jitter at its source, this embodiment incorporates an exponential decay control model at the algorithm layer to dynamically adjust the adaptive learning rate nonlinearly. The adaptive learning rate update model is constructed as follows:

[0064] in, To control the algorithm to enter the first Dynamic learning rate during sub-physical deflection calculation iteration; The initial base learning rate constant that is fixed during the power-on calibration of the wafer mounting system; The time decay constant parameter, which controls the exponential decay rate of the iterative adjustment step size, has a value range of 0.1 to 0.3 and is pre-calibrated according to the requirements of the iterative convergence rate. Counts the number of physical optimization iterations called in the current loop; It is a logical step function operator with hysteresis, whose inputs are two comparison variables; Set a value for the conduction threshold variance. Set the turn-off threshold variance value, and ; For the first iteration of the loop body The variance of the output is derived from the next physical iteration operation. From greater than Reduced to less than hour, The output changes from 1 to 0; when From less than Rise to greater than hour, The output changes from 0 to 1. Also, only when... The dynamic learning rate is only finally adjusted when the number of sampling periods during which the output remains zero exceeds a preset deadlock delay threshold. Force the value to zero and permanently lock it.

[0065] The significance of the adaptive learning rate update model is that: In the very early stages of physical contact between the crystal oscillator wafer and the fluid adhesive layer on the target placement plane, the physical contact surface exhibits an extremely unbalanced point contact limit state. At this point, the judgment variance is much larger than the set variance value, the step function is in full conduction mode, and the robotic arm end effector is given the largest initial base learning rate adjustment step size, driving the mechanical structure to quickly cut into and flatten the severely warped physical gap. As the number of physical correction iterations increases linearly, the contact area of ​​the underlying cured adhesive fluid gradually spreads and thickens, and the variance value shows a parabolic decreasing trend, approaching the set variance value. The exponential decay term rapidly decreases by multiples, fine-tuning the compensation correction step size.

[0066] When the final physical contact variance remains below the set shut-off threshold variance and the delay lock-up condition is met, the dynamic learning rate is forcibly cleared and blocked by the underlying logic gate. This causes all fine-tuning deflection compensation drive command flows of the robotic arm's end effector to be hardware-level deadlocked off, thereby locking the entire three-dimensional mechanical pose to its theoretical optimum with absolute rigidity. This cuts off the basis for the high-frequency micro-vibration phenomenon of cantilever resonance in the mechanical end effector structure from the underlying design logic of the dynamic physical mechanism. After satisfying the parallel conditions of absolute variance convergence and physical pose rigidity locking, the motion controller sends an air cut-off command to cut off the negative pressure pipeline system of the adaptive vacuum holding mechanism, ending the physical release process of the crystal oscillator wafer.

[0067] Example 3: This example is based on the same principles as Example 1, except that in this example, in order to meet the requirements for mechanical dissipation integrity verification after the assembly of micro-nano level crystal devices and the dynamic physical closed-loop management of the entire manufacturing cycle, the wafer mounting system automatically starts the bottom surface micro-displacement interference verification mechanism after completing the flexible mounting of the crystal oscillator wafer, which supplements the physical quality detection and sorting action command flow after the crystal oscillator wafer is mounted.

[0068] Because the conductive adhesive applied to the bottom of the crystal oscillator chip has complex fluid rheological characteristics before curing, relying solely on force control feedback along each axis during the assembly pressing stage cannot fully verify the actual diffusion area and physical adhesion strength of the conductive adhesive on the bottom surface of the chip. If microscopic pores exist inside the conductive adhesive or local voids are formed due to uneven extrusion, the device will experience lattice slip due to stress concentration during subsequent high-frequency oscillation operation, leading to excessive frequency deviation.

[0069] In this embodiment, after the adaptive vacuum holding mechanism cuts off the negative pressure and releases the crystal oscillator wafer, the motion controller directs the end effector of the robotic arm to move laterally and withdraw. The contact-type micro-force measuring probe on the side of the detection station is then horizontally fed to directly above the crystal oscillator wafer under the drive of a pneumatic slide. The contact-type micro-force measuring probe integrates a miniature voice coil motor and an interferometric fiber optic displacement sensor. The measuring optical axis of the interferometric fiber optic displacement sensor is coaxially arranged with the central axis of the detection pin. The detection pin is made of sapphire transparent material, or has a micron-sized through-hole in the center for the measurement optical path to pass through.

[0070] After the testing program is started, the voice coil motor receives a linearly increasing current command from the central controller, driving the testing pin to slowly press down at a set thrust slope, applying a vertical pre-tightening verification force to the upper surface of the crystal oscillator wafer. Simultaneously with the linear increase in thrust, the interferometric fiber optic displacement sensor utilizes the change in the optical path difference between two coherent beams between the reference end face and the reflective surface of the crystal oscillator wafer to acquire the nanometer-scale vertical displacement of the crystal oscillator wafer surface in real time.

[0071] The central processing unit continuously performs multidimensional mapping between the real-time force output by the voice coil motor and the microscopic displacement fed back by the interferometric fiber optic displacement sensor to construct a displacement response curve. If the conductive adhesive on the bottom of the crystal oscillator chip achieves uniform, void-free spreading, and the displacement response curve maintains the linear characteristics of rigidity before the pre-tightening verification force reaches the set peak value, and the final microscopic displacement is suppressed within the preset safe deformation threshold, the central processing unit determines that the bottom surface adhesion of the crystal oscillator chip meets the standard.

[0072] If there are significant welding voids or incomplete welds inside the conductive adhesive layer, the local adhesive layer will undergo microscopic yield shear rheology when the crystal oscillator is subjected to vertical thrust. This will cause the displacement collected by the interferometric fiber displacement sensor to undergo a nonlinear abrupt jump as the thrust increases. If the absolute value of the abrupt jump exceeds the safe deformation threshold, the central computing and processing unit will immediately determine that the crystal oscillator has a bottom bonding defect and generate defective product marking data with the physical code of the device.

[0073] After completing the bottom surface micro-displacement interference verification, the contact-type micro-force measurement probe is reset under the drive of the pneumatic slide. The motion controller retrieves the aforementioned multispectral visual sensing fusion system to perform a second inspection exposure on the assembled crystal oscillator wafer, obtains the final spatial three-dimensional dwell coordinates, and calculates the difference between these coordinates and the theoretically calibrated target coordinates to determine the final spatial assembly center deviation.

[0074] The central processing unit comprehensively evaluates the deformation verification data, the bottom surface adhesion compliance status, and the final spatial assembly center deviation. If all physical indicators fall within the acceptable range, the central processing unit issues a good product passage instruction to the subsequent material conveyor line; if the deformation verification data is abnormal or the center deviation exceeds the tolerance boundary, the central processing unit converts the defective product marking data into a diversion control electrical signal and sends it to the high-frequency electromagnetic reversing valve located at the downstream node of the conveyor line.

[0075] The high-frequency electromagnetic reversing valve precisely opens the air supply passage according to the timing of the diversion control electrical signal, driving the pneumatic pusher mechanism to perform an instantaneous lateral push-out action. Within the extremely short physical time window of the device flowing through the diversion node, the pneumatic pusher mechanism pushes the defective crystal oscillator wafer as a whole into the side scrap recycling container, while the verified qualified devices flow smoothly into the good product container. This process does not occupy the core cycle time of the main assembly line, and while ensuring the continuous high-speed operation of the crystal oscillator production line, it constructs an absolute physical closed loop from multimodal visual capture, dynamic trajectory prediction, zero-force compliant grinding and bonding to the final physical and mechanical quality screening.

[0076] To facilitate a better understanding of the present invention by those skilled in the art, the present invention will be further described below with reference to specific implementation examples; In specific implementation, the physical object being processed is an ultra-small surface-mount quartz crystal resonator, and the external physical dimensions of the crystal wafer are: length 1.2 mm, width 1.0 mm, and thickness 0.2 mm.

[0077] Specifically as follows: Step 1: The loading mechanism transports the crystal oscillator wafer to the inspection station. The multispectral visual sensing fusion system is activated. The ring light source emits a uniform broadband white diffuse reflection beam, the coaxial light source emits a green parallel probe light with a wavelength of 520 nm, and the near-infrared light source emits an infrared probe light with a wavelength of 850 nm. The global shutters of the first and second industrial cameras open simultaneously, with the exposure time set to 50 μs, capturing multi-channel two-dimensional image information and discrete three-dimensional spatial coordinate point cloud datasets.

[0078] Step 2: Input the image information and point cloud dataset into a deep convolutional neural network. The backbone network performs feature mapping with 3×3 convolutional kernels, the cross-stage local network module reduces the feature dimension from 256 to 64, and the path aggregation network completes the manifold alignment of multimodal features. The pose estimation branch network finally outputs a multidimensional continuous variable matrix, from which the three-dimensional pose estimation data of the crystal oscillator wafer is extracted: the three-dimensional coordinates of the center point are (150.25 mm, 85.40 mm, 2.50 mm), the length direction vector is [0.998, 0.063, 0], the width direction vector is [-0.063, 0.998, 0], and the thickness direction vector is [0, 0, 1.000]. The hardware timestamp generation module embeds an absolute clock mark T0 = 10:00:00.004 into this set of data.

[0079] Step 3: The motion controller reads the clock marker via a precise time protocol and obtains the real-time feedback coordinates (120.00 mm, 50.00 mm, 30.00 mm) of the robotic arm's end effector. The system measures the cumulative physical communication delay from camera exposure to control command issuance as Δt = 4 ms. The motion controller uses a cubic spline interpolation algorithm to smooth discrete nodes and generate a time-optimal trajectory.

[0080] Step 4: As the robotic arm rapidly approaches the target point, the motion controller invokes the Extended Kalman Filter (EKF) algorithm at a high-frequency sampling rate of 1 kHz. At a certain discrete moment, the measured position fed back by the encoder inside the robotic arm is (150.10 mm, 85.35 mm, 10.00 mm), while the predicted position vector calculated by the EKF algorithm based on the previous state prediction update model is (150.23 mm, 85.39 mm, 10.00 mm). The subtraction comparator automatically calculates the deviation value (predicted position minus measured position) as (Δx = 0.13 mm, Δy = 0.04 mm, Δz = 0). The motion controller uses this deviation value as a real-time control feedforward compensation amount, directly superimposed on the corresponding interpolation point of the current time-optimal trajectory. The corrected target position is (150.23 mm, 85.39 mm, 10.00 mm), driving the joint servo motors to precisely correct the motion path.

[0081] Step 5: The end effector of the robotic arm arrives at the actual dynamic gripping point without misalignment. The adaptive vacuum holding mechanism controls the proportional valve group to adjust the negative pressure of the pipeline to -65 kPa, smoothly picks up the crystal oscillator wafer, and carries the device to press it vertically downwards towards the target placement plane.

[0082] Step Six: Within a micro-space 0.5 mm above the target plane, four contact force sensors arranged in a ring array maintain high-frequency force monitoring at 2 kHz. When pressure is applied to a certain coordinate, the real-time contact force value measured by the contact force sensors abruptly changes from 0 N to 0.052 N, exceeding the set first contact force threshold of 0.05 N. The motion controller immediately and forcibly cuts off the high-speed feedforward, controlling the vertical descent speed of the robotic arm's end effector to drop sharply from 50 mm / s to the detection speed of 2 mm / s.

[0083] Step 7: The robotic arm continues to descend at the detection speed. When the contact force continues to climb to 0.31 N, exceeding the second contact force threshold of 0.3 N, a hardware-level highest priority interrupt is triggered, and the system instantly switches to zero-force mode. At this time, the position-holding current of the robotic arm joint motors maintaining spatial dwell is cleared to zero. The output readings of the four contact force sensors in the current clock slice are: F1 = 0.35 N, F2 = 0.22 N, F3 = 0.38 N, and F4 = 0.29 N. The central computing unit calculates the variance of the current contact force distribution of this set of force point distribution data as Vforce = 0.00375 N. 2 This value is greater than the preset conduction threshold variance setting value Vth1 = 0.0005 N. 2 .

[0084] Step 8: The system initiates an iterative optimization subroutine based on the gradient descent pose compensation model. The initial base learning rate is set to η0 = 0.01, and the time decay constant is set to γ ​​= 0.15. The control program uses the gradient of the partial derivative of the variance with respect to the pose correction vector to guide the robotic arm actuator to make a micro-angle deflection in the direction of the lower F2 pressure, i.e., the suspended end. The adjustment step size decreases exponentially with the number of iterations. After four consecutive iterations of spatial micro-motion deflection, the force values ​​fed back by the four contact force sensors converge to: F1 = 0.31 N, F2 = 0.30 N, F3 = 0.31 N, F4 = 0.30 N. At this point, the recalculated output variance converges sharply to Vforce = 0.00003 N. 2 The value successfully fell within the turn-off threshold variance setpoint Vth2 = 0.0004 N. 2 The following steps are performed: The output of the logic step function with hysteresis changes from 1 to 0, and after a delay period in which the variance value remains stable, the dynamic learning rate is cleared and permanently locked, achieving absolute rigidity locking of the spatial mechanical pose. The adaptive vacuum holding mechanism cuts off the negative pressure air source, physically releasing the crystal oscillator wafer.

[0085] Step Nine: The robotic arm retracts, and the contact-type micro-force measurement probe is fed above the crystal oscillator. The voice coil motor applies a pre-tightening verification force that linearly increases from 0 N to 0.5 N. The interferometric fiber optic displacement sensor measures that the vertical micro-displacement of the crystal oscillator wafer surface during the entire pressurization process is only 0.11 μm, which does not exceed the safe deformation threshold of 0.4 μm, thus determining that the bonding strength of the conductive adhesive on the bottom surface is qualified. Multispectral visual re-inspection shows that the assembly center deviation of the spatial center point is 3 μm, which fully meets the process technical specifications of the high-frequency resonator. The finished product flows safely into the good product container along the conveyor line, and the entire closed-loop process ends.

[0086] Although preferred embodiments of the invention have been described, those skilled in the art, upon learning the basic inventive concept, can make other changes and modifications to these embodiments. Therefore, the appended claims are intended to be interpreted as including both the preferred embodiments and all changes and modifications falling within the scope of the invention.

[0087] The above description is merely a preferred embodiment of the present invention and is not intended to limit the present invention. It should be noted that any modifications, equivalent substitutions, and improvements made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A method for co-loading crystal oscillator wafers using visual recognition and a robotic arm, characterized in that, Includes the following steps: The crystal oscillator wafers are transported to the testing station via a feeding mechanism. The two-dimensional image information and three-dimensional spatial coordinate information of the crystal oscillator wafer are acquired using a multispectral visual sensing fusion system; The two-dimensional image information and the three-dimensional spatial coordinate information are processed by a deep convolutional neural network to obtain feature vectors. Based on the feature vectors, the three-dimensional attitude estimation data of the crystal oscillator wafer is output. The three-dimensional attitude estimation data includes the three-dimensional coordinates of the center point, as well as the length direction vector, the width direction vector, and the thickness direction vector. A spatial path curve is constructed based on the three-dimensional attitude estimation data and the current position of the robotic arm end effector. The spatial path curve is then smoothed using cubic spline interpolation to generate the time-optimal trajectory of the robotic arm end effector. The end effector of the robotic arm is controlled to move along the time-optimal trajectory. The current position of the end effector is predicted using the extended Kalman filter algorithm to obtain the predicted position. The actual position of the end effector is compared with the predicted position to obtain the deviation value. The deviation value is the difference between the predicted position and the actual position. The deviation value is used as a feedforward correction amount and directly superimposed on the time-optimal trajectory to obtain the compensated trajectory. The end effector of the robotic arm is controlled to move along the compensated trajectory to the target grasping position, and the crystal oscillator is grasped by the adaptive vacuum gripping mechanism. The end effector of the robotic arm is controlled to carry the crystal oscillator towards the target placement plane. During the movement towards the target placement plane, the contact force output by multiple contact force sensors installed on the end effector of the robotic arm is monitored. When the contact force reaches the first contact force threshold, the end effector of the robotic arm is controlled to reduce the descent speed; when the contact force reaches the second contact force threshold, the end effector of the robotic arm is controlled to switch to zero force mode so that the crystal oscillator wafer forms a flexible contact with the target placement plane. In the zero-force mode, the current contact force distribution is constructed based on the contact forces output by multiple contact force sensors. The position deviation and attitude deviation of the end effector of the robotic arm are calculated based on the current contact force distribution. The position and attitude deviation of the end effector of the robotic arm are adjusted according to the position deviation and attitude deviation until the variance corresponding to the current contact force distribution is less than the set variance value, and the crystal oscillator wafer is assembled.

2. The method for co-loading crystal oscillator wafers with visual recognition and robotic arm according to claim 1, characterized in that, The two-dimensional image information and three-dimensional spatial coordinate information of the crystal oscillator wafer are acquired using a multispectral visual sensing fusion system, specifically including: The ring light source, coaxial light source and near-infrared light source integrated in the multispectral visual sensing fusion system are used to perform independent multispectral frequency band lighting to remove highlights, enhance contour edges and map metal coating layers on the crystal oscillator wafer. Using a first industrial camera and a second industrial camera arranged in a spatially separate manner, after receiving a global shutter synchronization trigger pulse, they simultaneously capture the multispectral pixel matrix and discrete point cloud dataset of the crystal oscillator wafer from different perspectives within the same sampling time window, which are respectively used as the two-dimensional image information and the three-dimensional spatial coordinate information.

3. The method for co-loading crystal oscillator wafers with visual recognition and robotic arm according to claim 1, characterized in that, The two-dimensional image information and the three-dimensional spatial coordinate information are processed based on a deep convolutional neural network to obtain a feature vector. The three-dimensional pose estimation data of the crystal oscillator wafer is then output based on the feature vector, specifically including: The two-dimensional image information is input into the backbone network of the deep convolutional neural network to perform multi-layer two-dimensional convolutional filtering operations, extracting local pixel gradient change features of the image to output an initial feature vector; the initial feature vector is then processed by a cross-stage local network module to perform channel segmentation and non-linear activation dimensionality reduction mapping, retaining strong semantic response channels and removing weak response channels to obtain a low-dimensional shallow feature tensor. The shallow feature tensor and the three-dimensional spatial coordinate information are synchronously transmitted to the path aggregation network. The two-dimensional texture features in the shallow feature tensor and the depth gradient features in the three-dimensional spatial coordinate information are aligned in the coordinate system and spliced ​​across layers in the manifold space through the top-down data interaction bus, and the aggregated feature tensor that fuses multimodal optical features is output. The aggregated feature tensor is input into the attitude estimation branch network, and a nonlinear regression operation is performed on the aggregated feature tensor through a multilayer perceptron fully connected model to reduce the dimensionality of the high-dimensional tensor space and project it onto the Euclidean physical space, thereby solving for the multidimensional continuous variable matrix that serves as the three-dimensional attitude estimation data.

4. The method for co-loading crystal oscillator wafers with visual recognition and robotic arm according to claim 2, characterized in that, A spatial path curve is constructed based on the three-dimensional attitude estimation data and the current position of the robotic arm's end effector. This spatial path curve is then smoothed using cubic spline interpolation to generate the time-optimal trajectory of the robotic arm's end effector. Specifically, this includes: At the moment the shutters of the first and second industrial cameras are triggered, a time stamp is embedded in the two-dimensional image information and the three-dimensional spatial coordinate information by a hardware timestamp generation module, and the time stamp is synchronized to the motion controller using a precise time protocol. The motion controller reads the three-dimensional attitude estimation data and the current position, constructs a spatial path curve in a spatial rectangular coordinate system, and smooths each discrete control node of the spatial path curve through a cubic spline interpolation algorithm, so that the first and second derivatives of the velocity curve remain continuous. The motion controller uses the system's rated power and the maximum angular velocity of the joint as constraints, and minimizes the total motion time as the optimization objective function. It solves a sequential quadratic programming subproblem in the time domain to generate a set of discrete interpolation points for the position coordinates of the time-optimal trajectory.

5. The method for co-loading crystal oscillator wafers with visual recognition and robotic arm according to claim 3, characterized in that, The extended Kalman filter algorithm is used to construct a state prediction update model for performing feedforward position prediction as follows: in, This is the predicted position vector corresponding to the end effector of the robotic arm at the current sampling time; The state transition matrix characterizes the dynamic decay of the system; The state vector of the end effector of the robotic arm at the previous discrete sampling time; For servo control input transformation matrix; This is the control command vector input to the joint drive motor; A transformation operator that maps the three-dimensional pose parameters of the crystal wafer contained in a multidimensional continuous variable matrix to an observation correction vector of the same dimension as the state vector of the robotic arm; The multidimensional continuous variable matrix is ​​the independent output of the attitude estimation branch network.

6. The method for co-loading crystal oscillator wafers with visual recognition and robotic arm according to claim 1, characterized in that, The measured position of the robotic arm's end effector is compared with the predicted position to obtain a deviation value. This deviation value is then used as a feedforward correction to compensate for the time-optimal trajectory, resulting in the compensated trajectory. Specifically, this includes: The subtraction comparator built inside the motion controller is invoked to perform a matrix difference comparison operation between the measured position and the predicted position, and the difference between the predicted position and the measured position is calculated to extract the absolute deviation value in the spatial rectangular coordinate system. The absolute deviation value is used as a feedforward correction amount and weighted over each discrete interpolation coordinate matrix corresponding to the time-optimal trajectory to regenerate the compensated trajectory, thereby offsetting the millisecond-level communication delay error caused by the visual algorithm calculation within the time mark transmission period.

7. The method for co-loading crystal oscillator wafers with visual recognition and robotic arm according to claim 3, characterized in that, The crystal oscillator wafer is gripped by an adaptive vacuum holding mechanism, specifically including: The adaptive vacuum holding mechanism calculates the required vacuum negative pressure airflow cross-sectional area and holding pressure parameters based on the length direction vector and width direction vector extracted from the multidimensional continuous variable matrix. The adaptive vacuum holding mechanism controls the pneumatic proportional servo valve to open, adjusting the pipeline air pressure to the set negative pressure state corresponding to the holding pressure parameter, thereby completing the non-destructive adsorption and gripping of the crystal oscillator wafer surface.

8. The method for co-loading crystal oscillator wafers with visual recognition and robotic arm according to claim 1, characterized in that, Controlling the segmented deceleration and zero-force mode switching of the robotic arm's end effector specifically includes: The plurality of contact force sensors continuously and dynamically monitor the contact force generated when the bottom surface of the crystal oscillator chip touches the target placement plane at a set sampling frequency, and feed the contact force as a continuous analog voltage signal back to the motion controller; When the motion controller detects that the contact force has reached the first contact force threshold, it blocks the speed feedforward loop and controls the end effector of the robotic arm to reduce the descent speed to a constant detection speed according to the exponential decay curve. When the motion controller detects that the contact force reaches the second contact force threshold, it triggers the highest priority hard interrupt protection program, controls the end effector of the robotic arm to be forced into zero force mode, cuts off the target position holding current of all spatial joint motors, and makes the end effector of the robotic arm passively follow and give way to the external physical force field.

9. The method for co-loading crystal oscillator wafers with visual recognition and robotic arm according to claim 5, characterized in that, In the zero-force mode, the pose of the robotic arm's end effector is adjusted based on the position and attitude deviations of the current contact force distribution. Specifically, this is achieved using a gradient descent pose compensation model based on the fusion of the three-dimensional pose estimation data, the predicted position, and the current contact force distribution using multi-dimensional tensors. The control equations of the gradient descent pose compensation model are constructed as follows: in, For the robotic arm end effector in the first The pose correction vector generated in the next iteration cycle; In the first The pose correction vector of the memory-resident record in the next iteration cycle; An adaptive learning rate is used to control the micro-motion offset step size; The gradient guiding variable is calculated by expanding the partial derivative of the variance with respect to the pose correction vector. The weighting coefficient for visual anchoring to prevent deviation; This is a mapping operator that projects the Cartesian physical position state transformation onto the Euler angle attitude matrix; For the first The predicted position vector corresponding to the predicted position during the next pose iteration loop.

10. The method for co-loading crystal oscillator wafers with visual recognition and robotic arm according to claim 1, characterized in that, After the crystal oscillator wafers are assembled, the following quality inspection and sorting steps are also included: The contact-type micro-force measurement probe is activated, and a pre-tightening verification force in the vertical direction is slowly applied to the upper surface of the crystal oscillator wafer at a set thrust slope through the internally integrated voice coil motor. The vertical micro-displacement of the crystal oscillator wafer surface is dynamically acquired using an interferometric fiber displacement sensor, and the uniform and void-free spreading of the conductive adhesive on the bottom surface is determined based on the mapping curve between the pre-tightening verification force and the vertical micro-displacement. If the vertical micro-displacement exceeds the safety deformation threshold before the pre-tightening verification force reaches the set peak value, the crystal oscillator wafer will be identified as a defective product with bonding defects and corresponding defective product marking data will be generated. The central computing and processing unit converts the defective product marking data into a diversion control electrical signal and sends it to the high-frequency electromagnetic reversing valve of the downstream node. This controls the pneumatic push rod mechanism to perform a lateral pushing action, pushing the defective crystal oscillator wafer as a whole into the side scrap recycling container, while the verified qualified devices flow into the good product container.

Images