Multi-camera visual servoing-based flexible circuit board assembly method and system

Abstract

The present invention provides a multi-camera visual servoing-based flexible circuit board assembly method and system. The method comprises: positioning, by means of a first camera, a flexible circuit board, obtaining a first outer contour feature of a back plate, and gripping the back plate on the basis of the first outer contour feature; positioning, by means of a second camera, the flexible circuit board, obtaining a second outer contour feature of the back plate and a circuit feature of a connector, and on the basis of a correspondence between the first outer contour feature and the second outer contour feature and a positional relationship between the second outer contour feature and the circuit feature, mapping the position of the circuit feature to a viewing angle of the first camera to obtain a visual identification feature value; and filtering the visual identification feature value on the basis of particle filtering and Kalman filtering, and assembling the flexible circuit board on the basis of the filtered feature value. In the present invention, multi-camera visual servoing technology is employed to accurately identify the flexible circuit board, thereby solving the problem that a camera cannot directly acquire a circuit board feature due to occlusion in the field of view during assembly.

Description

A Flexible Circuit Board Assembly Method and System Based on Multi-Camera Field-of-View Visual Servo Technical Field

[0001] This invention relates to the field of industrial vision technology, and more specifically, to a flexible circuit board assembly method and system based on multi-camera field-of-view visual servoing. Background Technology

[0002] In industrial production, the assembly of flexible printed circuit boards (PCBs) has always been a challenge. PCBs are characterized by their small size, thinness, flexibility, high customization requirements, complex operating conditions, and dense internal wiring. Several issues hinder automated assembly: First, the front-side PCBs require high-precision interfaces for assembly; however, during assembly, the camera's field of view is often obstructed, only allowing visualization of the back panel and hindering the identification and positioning of PCB features. Second, gripping is difficult, and improper gripping can easily damage the PCBs. Therefore, high-precision servo control is needed to prevent damage during gripping. Consequently, in actual production, manual assembly is usually required. Therefore, a solution for automating the assembly of flexible PCBs is meaningful and meets market demands.

[0003] Visual servoing is a technology that uses computer vision information to achieve closed-loop control. It features high precision and flexible control. By acquiring image information and target images in real time and performing calculations, it quickly and accurately feeds back visual information to the robot, thereby achieving robot control.

[0004] Chinese patent document CN220776150U discloses a flexible circuit board that is easy to install, belonging to the field of flexible circuit board technology. It includes a fixing plate, with a circuit board body on top of the fixing plate. The upper surface of the circuit board body has two sets of slots, each containing a insert rod. The bottom surface of each insert rod is fixedly connected to the upper surface of the fixing plate. Slots are formed on the sides of the two sets of insert rods that are close to each other. Because the camera's field of view is obstructed during assembly, only the back panel is visible, and the circuit board features are not. Therefore, this flexible circuit board still requires manual assembly.

[0005] To address the challenges of automated assembly of flexible circuit boards, an automated assembly method for flexible circuit boards is needed. Summary of the Invention

[0006] To address the shortcomings of existing technologies, the purpose of this invention is to provide a flexible circuit board assembly method and system based on multi-camera field-of-view visual servoing.

[0007] According to the present invention, a flexible circuit board assembly method based on multi-camera field-of-view visual servoing is provided, wherein the flexible circuit board includes a backplane and an interface, and the assembly method includes:

[0008] Step S1: Position the flexible circuit board using the first camera to obtain the first outer contour feature of the back panel, and grasp the back panel based on the first outer contour feature;

[0009] Step S2: Position the flexible circuit board using the second camera to obtain the second outer contour features of the back panel and the circuit features of the interface. Based on the correspondence between the first and second outer contour features and the positional relationship between the second outer contour features and the circuit features, map the position of the circuit features to the viewpoint of the first camera to obtain visual recognition feature values.

[0010] Step S3: Filter the visual recognition feature values ​​using a filtering algorithm, and then assemble the flexible circuit board using the filtered feature values.

[0011] Preferably, step S1 includes the following sub-steps:

[0012] Step S1.1: Use the first camera to perform coarse positioning of the backplane of the flexible circuit board and obtain the coarse positioning result of the backplane;

[0013] Step S1.2: Use the edge contour detection algorithm to finely locate the edges in the coarse positioning result to obtain the first outer contour feature of the back plate;

[0014] Step S1.3: Input the first outer contour feature into the servo system to complete the clamping of the flexible circuit board backplate.

[0015] Preferably, the first outer contour feature is a feature line or a feature point; the feature line includes contour edge lines, and the feature point includes the intersection of contour edge lines, or key points identified through a key point model.

[0016] Preferably, step S2 includes the following sub-steps:

[0017] Step S2.1: Position the flexible circuit board using the second camera to obtain the second outer contour features of the backplate, the circuit features of the interface, and the positional relationship between the two sets of features from the second camera's perspective.

[0018] Step S2.2: Obtain the field of view of the first camera and the second camera, and calculate the ratio of their field of view sizes;

[0019] Step S2.3: Select the same reference feature from the first outer contour feature and the second outer contour feature, and obtain the field rotation ratio based on the angular relationship between the two reference features;

[0020] Step S2.4: Calculate the position of the circuit feature under the first camera's viewpoint, i.e., the visual recognition feature value, based on the positional relationship between the circuit feature and the second outer contour, the field of view size ratio, and the field of view rotation ratio.

[0021] Preferably, the filtering algorithm is a particle filtering and Kalman filtering algorithm, and the combination of particle filtering and Kalman filtering to filter visual recognition feature values ​​includes:

[0022] Step S3.1: Obtain visual recognition feature values, and initialize M particles with equal weights around the circuit feature location, with each particle representing a possible state;

[0023] Step S3.2: Predict the next motion state of the particles based on the motion model, calculate the similarity between each particle and the observed feature value at this moment, and update the weights so that the particles that match the observed value more closely get a larger weight.

[0024] Step S3: Calculate the state estimate using the partial particles with the larger weights, and predict the covariance matrix of the state estimate as the prior estimate for step S4.

[0025] Step S4: Calculate the Kalman gain using the state covariance matrix and the noise covariance matrix, and update the state estimate and covariance matrix using the gain;

[0026] Step S5: Iteratively update the particle weights and gain calculations to make the state estimate approximate the true state and converge.

[0027] A flexible circuit board assembly system based on multi-camera field-of-view visual servoing, according to the present invention, includes:

[0028] Module M1: Positions the flexible circuit board using the first camera to obtain the first outer contour features of the back panel, and grasps the back panel based on the first outer contour features;

[0029] Module M2: The flexible circuit board is positioned by the second camera to obtain the second outer contour features of the back panel and the circuit features of the interface. Based on the correspondence between the first outer contour features and the second outer contour features, as well as the positional relationship between the second outer contour features and the circuit features, the position of the circuit features is mapped to the view of the first camera to obtain visual recognition feature values.

[0030] Module M3: Combines filtering algorithms to filter visual recognition feature values, and uses the filtered feature values ​​to assemble flexible circuit boards.

[0031] Preferably, module M1 includes the following sub-modules:

[0032] Module M1.1: Uses the first camera to perform coarse positioning of the backplane of the flexible circuit board and obtains the coarse positioning result of the backplane;

[0033] Module M1.2: Uses an edge contour detection algorithm to finely locate the edges in the coarse positioning results, and obtains the first outer contour feature of the back plate;

[0034] Module M1.3: Inputs the first outer contour feature into the servo system to complete the clamping of the flexible circuit board backplate.

[0035] Preferably, the first outer contour feature is a feature line or a feature point; the feature line includes contour edge lines, and the feature point includes the intersection of contour edge lines, or key points identified through a key point model.

[0036] Preferably, module M2 includes the following sub-modules:

[0037] Module M2.1: The flexible circuit board is positioned using a second camera to obtain the second outer contour features of the backplate, the circuit features of the interface, and the positional relationship between the two sets of features from the second camera's perspective.

[0038] Module M2.2: Obtain the field of view of the first camera and the second camera, and calculate the ratio of their field of view sizes;

[0039] Module M2.3: Select the same reference feature from the first outer contour feature and the second outer contour feature, and obtain the field rotation ratio based on the angular relationship between the two reference features;

[0040] Module M2.4: Based on the positional relationship between the circuit features and the second outer contour, the field of view size ratio, and the field of view rotation ratio, calculate the position of the circuit features under the first camera's viewpoint, i.e., the visual recognition feature value.

[0041] According to the present invention, a flexible circuit board assembly device is provided, wherein the assembly device performs the flexible circuit board assembly method based on multi-camera field-of-view visual servoing during operation.

[0042] Compared with the prior art, the present invention has the following beneficial effects:

[0043] 1. This invention uses visual servoing technology under multi-camera field of view to accurately identify flexible circuit boards, solving the problem that the camera cannot directly obtain the circuit board features due to the obstruction of the field of view during the assembly process.

[0044] 2. This invention uses a multi-stage vision algorithm to track and locate materials. Compared with single-stage positioning, multi-stage positioning can filter irrelevant information, reduce the possibility of false detection, and achieve higher positioning accuracy. Multi-stage positioning first quickly and initially locates the region of interest, and then only needs to precisely locate and detect features within this region, thereby reducing the amount of computation and improving computational efficiency.

[0045] 3. This invention uses contour features or key point features as the output of the positioning algorithm, which can be compatible with more types of materials.

[0046] 4. This invention combines an improved visual servoing method, optimizes the control mode, solves the problem of low stability in traditional servo control, and completes the automatic assembly of flexible circuit boards more efficiently.

[0047] 5. This invention solves the problems of damage to flexible circuit boards caused by positioning errors during gripping and damage to flexible circuit boards caused by assembly errors by achieving high-precision positioning and assembly of the flexible circuit boards.

[0048] Attached Figure Description

[0049] Other features, objects, and advantages of the present invention will become more apparent from the following detailed description of non-limiting embodiments with reference to the accompanying drawings:

[0050] Figure 1 is a flowchart of the flexible circuit board assembly method based on multi-camera field-of-view visual servoing according to the present invention;

[0051] Figure 2 is a schematic diagram of the coarse positioning effect of the flexible circuit board in this invention;

[0052] Figure 3 is a schematic diagram of the fine positioning effect of the flexible circuit board in this invention;

[0053] Figure 4 is a schematic diagram of the flexible circuit board from the perspective of the second camera in this invention;

[0054] Figure 5 is a schematic diagram of the material positions during the assembly process of the flexible circuit board in this invention. Detailed Implementation

[0055] The present invention will now be described in detail with reference to specific embodiments. These embodiments will help those skilled in the art to further understand the present invention, but do not limit the invention in any way. It should be noted that those skilled in the art can make several changes and improvements without departing from the concept of the present invention. These all fall within the protection scope of the present invention.

[0056] This invention discloses a flexible circuit board assembly method based on multi-camera field-of-view visual servoing. The overall technical flow of this method is shown in Figure 1. The method includes three steps: backplane identification, circuit feature identification, and assembly. The flexible circuit board includes a backplane and an interface. For ease of description, the side of the flexible circuit board containing the backplane is defined as the back side, and the side containing the interface is defined as the front side. First, the position of the flexible circuit board needs to be identified. The flexible circuit board is clamped to the assembly position using a tool on the back side of the flexible circuit board. Second, the circuit features in the interface on the front side of the flexible circuit board are identified, mapped onto the back side of the flexible circuit board, and used as servo input. Finally, automated assembly is completed through servo control.

[0057] The following is a further description of the flexible circuit board assembly method based on multi-camera field-of-view visual servoing disclosed in this invention.

[0058] The backplane identification process is as follows:

[0059] First, the back image of the flexible circuit board is acquired from a first-view perspective using a first camera. Due to the small size of the material, a coarse positioning algorithm is first used to coarsely locate the edge of the target to obtain the coarse positioning result of the back plate.

[0060] In a preferred embodiment, as shown in Figure 2, for a rectangular flexible circuit board, an object detection algorithm is used to coarsely locate the four edges of the backplate to obtain the model's predicted result, i.e., the portion selected by the rectangular box in the figure. The object detection algorithm can be SSD, YOLO series, R-CNN series, etc. Of course, the object detection algorithm is not limited to the listed algorithms; any algorithm capable of coarsely locating materials falls within the protection scope of this invention.

[0061] Secondly, an edge contour detection algorithm based on an AI model is used to refine the edge localization in the coarse localization result, thereby obtaining the first outer contour feature f of the backplate. l .

[0062] In one specific implementation, taking the identification of the edge of the backplate in Figure 3 as an example, the fine localization algorithm can use Hough transform, LineNet, etc.

[0063] The coarse-fine two-stage localization algorithm has two advantages over direct fine localization: First, it offers higher accuracy. Coarse localization filters feature regions by learning and predicting the region of interest, eliminating a large amount of irrelevant background and focusing more on the target region, thus filtering out irrelevant information and reducing missed detections. However, the vast image contains a lot of irrelevant background information, which can interfere with accurate localization. Noise, specks, or other similar structures in the background can mislead the detection algorithm, leading to decreased accuracy. Therefore, direct fine localization would increase the difficulty of recognition. Second, it offers higher computational efficiency. In the fine localization stage, high-accuracy algorithms typically have high complexity. Using only fine localization requires traversing the entire image, resulting in low computational efficiency. The coarse-fine localization method can quickly and initially locate the region of interest, then only needs to finely locate and detect features within that region.

[0064] The first outer contour feature f output by the edge contour detection algorithm l This can be the edge lines of the backplate outline, the feature points of the backplate outline, or the key point information of the backplate; taking a rectangular backplate as an example, output the four outline lines of the backplate:

[0065] Line0=[Line0x1, Line0y1, Line0x2, Line0y2]

[0066] Line1=[Line1x1, Line1y1, Line1x2, Line1y2]

[0067] Line2=[Line2x1, Line2y1, Line2x2, Line2y2]

[0068] Line3=[Line3x1, Line3y1, Line3x2, Line3y2]

[0069] Line0, Line1, Line2, and Line3 are the output values ​​of the edge contour detection algorithm.

[0070] Output the feature points of the backplate profile:

[0071] In some cases, the output requirement is based on the visual features of key points. This can be achieved by calculating the intersection of four edge lines to obtain the coordinates of the four key points. Taking the intersection of Line0 and Line1 as an example, this involves solving a system of linear equations.

[0072] For Line0, let:

[0073] x1, y1 = (Line0x1, Line0y1)

[0074] x2, y2 = (Line0x2, Line0y2)

[0075] Similarly, for Line 1:

[0076] x3, y3 = (Line1x1, Line1y1)

[0077] x4, y4 = (Line1x2, Line1y2)

[0078] The following system of equations can be obtained:

[0079] (y2-y1)x-(x2-x1)y=x1y2-x2y1

[0080] (y4-y3)x-(x4-x3)y=x3y4-x4y3

[0081] Solving the system of equations yields the intersection point R = (x, y) of the two lines. The four intersection points are then calculated sequentially and ordered. This ordering can be achieved by calculating the center point of each intersection point and then, using a specified anchor vector, ordering the intersection points by clockwise or counterclockwise angles.

[0082] Key points of the output background: The position of the key points of the background can be directly predicted by the key point model and used as the visual output value.

[0083] For servos, the vision component returns feature values ​​in a fixed order, and the type of feature values ​​is not limited; for example, they can be contours or key points as mentioned above.

[0084] Finally, after the servo system obtains the returned contour feature value, it calculates the contour value of the corresponding back plate position on the fixture in real time during actual operation. By controlling the servo to make the current value approach the returned contour feature value, the target of recognizing and grasping the back plate can be realized.

[0085] The process for identifying circuit features is as follows:

[0086] The front of the flexible circuit board is shown in Figure 4. Since the inner circuit cannot be touched directly during actual assembly, the back plate can only be attached to the back of the flexible circuit board using a tool.

[0087] Figure 4 shows the positional relationship of materials during assembly. The upper part is the flexible circuit board, and the lower part is the interface. The circuit structure is located in the lower interface. Since the position of the circuit cannot be directly seen from the top, it is necessary to first identify the key features of the circuit board and map them to the position of the back plate during the assembly process.

[0088] Specifically, methods for mapping circuit feature locations to the backplane include:

[0089] Step S1: Install a second camera under the flexible circuit board. The second viewpoint of the second camera is shown in Figure 4. Following the method of backplane identification, obtain the interface circuit features F under the second viewpoint. e And the second outer contour feature F of the flexible circuit board backplane l It can also obtain the relative positional relationship between two sets of features in the image coordinate system from a second perspective. The circuit feature F of this interface... e It can be the outline features of the circuit structure at the interface, or the position features of the pins inside the interface.

[0090] Step S2: Calculate the ratio V of the field of view between the two cameras based on their aspect ratios. s .

[0091] Step S3: Select the same contour edge in both cameras to calculate the angular relationship, which yields the field-of-view rotation ratio V. r .

[0092] Step S4: Obtain the first outer contour feature f of the backplate from the first camera using model prediction. l .

[0093] Step S5: Based on the circuit characteristics and the second outer contour characteristic F l The positional relationship and the field of view ratio V s and field of view rotation ratio V r Calculate the circuit characteristic Fe Position in the first-person perspective is denoted as f. e .

[0094] Specifically, F is obtained in step S1 e =[(x1,y1),(x2,y2)], F l =[(a1,b1),(a2,b2)]. The positional relationship between the two sets of features can be represented by a combination of distance D and included angle R. For ease of calculation, the two cameras are usually mounted in parallel, i.e., the field-of-view rotation ratio V in step S3. r =1, at this point the positional relationship can be simplified and represented by a vector as F e F l =V r [[(x1-a1),(y1-b1),[(x2-a2),(y2-b2)]], from F e Starting from, move along the x-axis and y-axis by F respectively. e F l The calculated vector represents the relationship between the two sets of features. The outer edge of a flexible circuit board is usually rectangular. If the two cameras are not installed parallel, select the model from the two cameras to identify the long or short side of the outer contour and calculate the angular relationship.

[0095] In step S2, the camera field-of-view ratio is measured. Since the positional relationship was calculated using vectors in the x and y directions in the previous step, the field-of-view ratio is calculated here using the aspect ratio. Assuming the current field of view of the first camera is 180mm * 60mm and the current field of view of the second camera is 120mm * 40mm, and the aspect ratios of the two cameras are the same, V is obtained. s =A / B=1.5. If the aspect ratios of the two fields of view are inconsistent, it is necessary to calculate the two ratios separately by dividing the x-vector and y-vector by the corresponding ratios. Let f be calculated in step S4. l = [(a3,b3),(a4,b4)], then the result of the final step S5 is f. e =f l -(F e F l / V s ).

[0096] The assembly process is as follows:

[0097] As shown in Figure 5, the circuit characteristics f of the slot at the assembly target location are obtained through a coarse-fine positioning algorithm. s and the f obtained in the previous step e The feature value F of visual recognition output output =[f e ,f sThese two sets of feature values ​​are returned to the robot in real time as visual output information. Subsequently, servo control enables the robot to assemble the flexible circuit board under the view of multiple cameras; the assembly here refers to the assembly of interfaces, such as the assembly of the backplate plug and the slot at the assembly target position, or the assembly of the backplate slot and the plug at the assembly target position.

[0098] For traditional visual servoing, the control objective is to minimize the error R between the current image features and the desired target features:

[0099] R = S(P) f ,a)-P t

[0100] P f Here are the coordinate values ​​of the image features for visual output. S(P) f a) is a visual feature vector used to describe the position of the target in the current image, where a is the camera intrinsic parameter, P t The target desired value is determined by selecting a control scheme, designing a speed controller, and solving the Jacobian matrix to obtain the relationship between the eigenvectors and the velocity variables. Additionally, the difference between the input image eigenvectors and the target eigenvectors needs to be calculated to obtain the error signal. Based on the Jacobian matrix and the error signal, the posture and velocity of the robotic arm's end effector can be further calculated, enabling the robot to accurately track the target in the image.

[0101] Therefore, it can be seen that traditional servo methods rely heavily on visual information, but in complex scenarios, two problems often exist. First, visual output may experience frame loss, meaning that visual results are not received at a certain moment; second, when the difference between the visual output frequency and the control frequency is too large, it will also affect the robot's control.

[0102] To mitigate the impact of the aforementioned two problems on visual servoing, this invention employs filtering to post-process the visual output values. Specifically, assuming a visual return frequency of 20ms and a control frequency of 5ms, with a 15ms difference, the robot's movement is controlled using the filtered prediction results at a control frequency of 5ms / time until the visual result is received. This visual result is then used as the true value at that moment. This avoids robot stuttering or significant errors in movement direction caused by the two aforementioned problems.

[0103] Hybrid filtering mainly utilizes the ideas of particle filtering and Kalman filtering. The complete process is as follows:

[0104] Step A1: Initialize M particles with equal weights around the circuit feature location, each particle representing a possible state. The size of M depends on a combination of the reliability of the visual output feature values ​​and computation time.

[0105] Step A2: Predict the next motion state of the particles based on the motion model, calculate the similarity between each particle and the observed feature value at this moment, and update the weights so that the particles that match the observation value more closely get greater weights.

[0106] Step A3: A weighted average is applied to the particles with larger weights to calculate new state values, representing the target's current state characteristics such as position and velocity. Furthermore, the covariance matrix of the predicted state estimate is used as a priori estimate for step A4. The covariance matrix represents the confidence level of the state estimate and is estimated using particle weights and a motion model. The choice of motion model can be based on the specific circumstances; for example, a nonlinear model, a Markov model, or a Gaussian model can be selected.

[0107] Step A4: Calculate the Kalman gain using the state covariance matrix and the noise covariance matrix. The previously estimated covariance matrix is ​​denoted as p. k,k-1 Let k and k-1 represent the current and previous time steps, respectively. The noise covariance matrix is ​​denoted as R. k This is used to represent the current statistical characteristics of the noise. If the noise originates from vibration or friction, it can be obtained through physical experimental measurements; if the source is unknown, parameter values ​​can be input using statistical averaging. The state vector is transformed into a matrix H representing the measurement vector at this moment. k , That is its transpose matrix. The Kalman gain can be calculated as follows:

[0108] Step A5: Iteratively update the particle weights and gain calculations. This is achieved by comparing the similarity between the particle's predicted state and the observed feature values. After the weight update, the larger the particle's weight, the better it matches the observed value, and therefore the higher its priority. During the iteration, the state estimate and covariance matrix are continuously updated using Kalman gain, making the current state approximate the true state and thus converge, improving the accuracy and reliability of the visual feature values ​​for servo control.

[0109] The invention also provides a flexible circuit board assembly system based on multi-camera field-of-view visual servoing. The flexible circuit board assembly system based on multi-camera field-of-view visual servoing can be implemented by executing the process steps of the flexible circuit board assembly method based on multi-camera field-of-view visual servoing. That is, those skilled in the art can understand the flexible circuit board assembly method based on multi-camera field-of-view visual servoing as a preferred embodiment of the flexible circuit board assembly system based on multi-camera field-of-view visual servoing.

[0110] This invention provides a flexible circuit board assembly system based on multi-camera field-of-view visual servoing, comprising:

[0111] Module M1: Positions the flexible circuit board using the first camera, obtains the first outer contour features of the back panel, and grasps the back panel based on the first outer contour features.

[0112] The module M1 includes the following sub-modules:

[0113] Module M1.1: Uses the first camera to perform coarse positioning of the backplane of the flexible circuit board and obtains the coarse positioning result of the backplane;

[0114] Module M1.2: Uses an edge contour detection algorithm to finely locate the edges in the coarse positioning results, and obtains the first outer contour feature of the back plate;

[0115] Module M1.3: Inputs the first outer contour feature into the servo system to complete the clamping of the flexible circuit board backplate.

[0116] The first outer contour feature is a feature line or a feature point; the feature line includes contour edge lines, and the feature point includes the intersection of contour edge lines, or key points identified through a key point model.

[0117] Module M2: The flexible circuit board is positioned by the second camera to obtain the second outer contour features of the back panel and the circuit features of the interface. Based on the correspondence between the first outer contour features and the second outer contour features, as well as the positional relationship between the second outer contour features and the circuit features, the position of the circuit features is mapped to the view of the first camera to obtain visual recognition feature values.

[0118] Module M2 includes the following sub-modules:

[0119] Module M2.1: The flexible circuit board is positioned using a second camera to obtain the second outer contour features of the backplate, the circuit features of the interface, and the positional relationship between the two sets of features from the second camera's perspective.

[0120] Module M2.2: Obtain the field of view of the first camera and the second camera, and calculate the ratio of their field of view sizes;

[0121] Module M2.3: Select the same reference feature from the first outer contour feature and the second outer contour feature, and obtain the field rotation ratio based on the angular relationship between the two reference features;

[0122] Module M2.4: Based on the positional relationship between the circuit features and the second outer contour, the field of view size ratio, and the field of view rotation ratio, calculate the position of the circuit features under the first camera's viewpoint, i.e., the visual recognition feature value.

[0123] Module M3: Combines particle filtering and Kalman filtering to filter visual recognition feature values, and then uses the filtered feature values ​​for flexible circuit board assembly. The filtering process includes:

[0124] Module M3.1: Acquire visual recognition feature values ​​and initialize M particles with equal weights around the circuit feature location, each particle representing a possible state;

[0125] Module M3.2: Predicts the next motion state of the particles based on the motion model, calculates the similarity between each particle and the observed feature value at this moment, and updates the weights so that the particles that match the observation value more closely get greater weights.

[0126] Module M3: Calculates the state estimate using the fractional particles with the largest weights, predicts the covariance matrix of the state estimate, and uses it as the prior estimate for step S4.

[0127] Module M4: Calculates the Kalman gain using the state covariance matrix and the noise covariance matrix, and updates the state estimate and covariance matrix using the gain;

[0128] Module M5: Iteratively updates the particle weights and gains, making the state estimates approximate the true state and converge.

[0129] The present invention also discloses a flexible circuit board assembly device, which executes the above-described flexible circuit board assembly method based on multi-camera field-of-view visual servoing during operation.

[0130] Those skilled in the art will understand that, besides implementing the system and its various devices, modules, and units provided by this invention in the form of purely computer-readable program code, the same functions can be achieved entirely through logical programming of the method steps, making the system and its various devices, modules, and units of this invention function in the form of logic gates, switches, application-specific integrated circuits, programmable logic controllers, and embedded microcontrollers. Therefore, the system and its various devices, modules, and units provided by this invention can be considered as a hardware component, and the devices, modules, and units included therein for implementing various functions can also be considered as structures within the hardware component; alternatively, the devices, modules, and units for implementing various functions can be considered as both software modules implementing the method and structures within the hardware component.

[0131] In the description of this application, it should be understood that the terms "upper", "lower", "front", "back", "left", "right", "vertical", "horizontal", "top", "bottom", "inner", "outer", etc., indicate the orientation or positional relationship based on the orientation or positional relationship shown in the accompanying drawings. They are only for the convenience of describing this application and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation. Therefore, they should not be construed as limitations on this application.

[0132] Specific embodiments of the present invention have been described above. It should be understood that the present invention is not limited to the specific embodiments described above, and those skilled in the art can make various changes or modifications within the scope of the claims, which do not affect the essence of the present invention. Unless otherwise specified, the embodiments and features described in this application can be arbitrarily combined with each other.

Claims

1. A flexible circuit board assembly method based on multi-camera field vision visual servoing, characterized by, The flexible circuit board includes a backplane and an interface, and the assembly method includes: Step S1: Position the flexible circuit board using the first camera to obtain the first outer contour feature of the back panel, and grasp the back panel based on the first outer contour feature; Step S2: Position the flexible circuit board using the second camera to obtain the second outer contour features of the back panel and the circuit features of the interface. Based on the correspondence between the first and second outer contour features and the positional relationship between the second outer contour features and the circuit features, map the position of the circuit features to the viewpoint of the first camera to obtain visual recognition feature values. Step S3: Filter the visual recognition feature values ​​using a filtering algorithm, and then assemble the flexible circuit board using the filtered feature values.

2. The multi-camera field-of-view vision servo based flexible circuit board assembly method of claim 1, wherein, Step S1 includes the following sub-steps: Step S1.1: Use the first camera to perform coarse positioning of the backplane of the flexible circuit board and obtain the coarse positioning result of the backplane; Step S1.2: Use the edge contour detection algorithm to finely locate the edges in the coarse positioning result to obtain the first outer contour feature of the back plate; Step S1.3: Input the first outer contour feature into the servo system to complete the clamping of the flexible circuit board backplate.

3. The multi-camera field-of-view vision servo based flexible circuit board assembly method of claim 1, wherein, The first outer contour feature is a feature line or a feature point; the feature line includes contour edge lines, and the feature point includes the intersection of contour edge lines, or key points identified through a key point model.

4. The multi-camera field-of-view vision servo based flexible circuit board assembly method of claim 1, wherein, Step S2 includes the following sub-steps: Step S2.1: Position the flexible circuit board using the second camera to obtain the second outer contour features of the backplate, the circuit features of the interface, and the positional relationship between the two sets of features from the second camera's perspective. Step S2.2: Obtain the field of view of the first camera and the second camera, and calculate the ratio of their field of view sizes; Step S2.3: Select the same reference feature from the first outer contour feature and the second outer contour feature, and obtain the field rotation ratio based on the angular relationship between the two reference features; Step S2.4: Calculate the position of the circuit feature under the first camera's viewpoint, i.e., the visual recognition feature value, based on the positional relationship between the circuit feature and the second outer contour, the field of view size ratio, and the field of view rotation ratio.

5. The multi-camera field-of-view vision servo based flexible circuit board assembly method of claim 1, wherein, The filtering algorithm is a combination of particle filtering and Kalman filtering. The filtering of visual recognition feature values ​​using a combination of particle filtering and Kalman filtering includes: Step S3.1: Obtain visual recognition feature values, and initialize M particles with equal weights around the circuit feature location, with each particle representing a possible state; Step S3.2: Predict the next motion state of the particles based on the motion model, calculate the similarity between each particle and the observed feature value at this moment, and update the weights so that the particles that match the observed value more closely get a larger weight. Step S3: Calculate the state estimate using the partial particles with the larger weights, and predict the covariance matrix of the state estimate as the prior estimate for step S4. Step S4: Calculate the Kalman gain using the state covariance matrix and the noise covariance matrix, and update the state estimate and covariance matrix using the gain; Step S5: Iteratively update the particle weights and gain calculations to make the state estimate approximate the true state and converge.

6. A flexible circuit board assembly system based on multi-camera field vision visual servoing, characterized by, include: Module M1: Positions the flexible circuit board using the first camera to obtain the first outer contour features of the back panel, and grasps the back panel based on the first outer contour features; Module M2: The flexible circuit board is positioned by the second camera to obtain the second outer contour features of the back panel and the circuit features of the interface. Based on the correspondence between the first outer contour features and the second outer contour features, as well as the positional relationship between the second outer contour features and the circuit features, the position of the circuit features is mapped to the view of the first camera to obtain visual recognition feature values. Module M3: Combines filtering algorithms to filter visual recognition feature values, and uses the filtered feature values ​​to assemble flexible circuit boards.

7. The multi-camera field-of-view vision servo based flexible circuit board assembly system of claim 6, wherein, The module M1 Includes the following sub-modules: Module M1.1: Uses the first camera to perform coarse positioning of the backplane of the flexible circuit board and obtains the coarse positioning result of the backplane; Module M1.2: Uses an edge contour detection algorithm to finely locate the edges in the coarse positioning results, and obtains the first outer contour feature of the back plate; Module M1.3: Inputs the first outer contour feature into the servo system to complete the clamping of the flexible circuit board backplate.

8. The multi-camera field-of-view vision servo based flexible circuit board assembly system of claim 6, wherein, The first outer contour feature is a feature line or a feature point; the feature line includes contour edge lines, and the feature point includes the intersection of contour edge lines, or key points identified through a key point model.

9. The multi-camera field of view vision servo based flexible circuit board assembly system of claim 6, wherein, The module M2 Includes the following sub-modules: Module M2.1: The flexible circuit board is positioned using a second camera to obtain the second outer contour features of the backplate, the circuit features of the interface, and the positional relationship between the two sets of features from the second camera's perspective. Module M2.2: Obtain the field of view of the first camera and the second camera, and calculate the ratio of their field of view sizes; Module M2.3: Select the same reference feature from the first outer contour feature and the second outer contour feature, and obtain the field rotation ratio based on the angular relationship between the two reference features; Module M2.4: Based on the positional relationship between the circuit features and the second outer contour, the field of view size ratio, and the field of view rotation ratio, calculate the position of the circuit features under the first camera's viewpoint, i.e., the visual recognition feature value.

10. A flexible circuit board assembly apparatus, characterized by, The assembly equipment performs the flexible circuit board assembly method based on multi-camera field-of-view visual servoing as described in any one of claims 1 to 5 when it is in operation.

Images