Flexible circuit board assembling method and system based on multi-camera visual field visual servo

Abstract

The invention provides a flexible circuit board assembly method and system based on multi-camera visual field visual servo, and the method comprises the steps: positioning a flexible circuit board through a first camera, obtaining a first outer contour feature of a backboard, and grabbing the backboard according to the first outer contour feature; and positioning the flexible circuit board through the second camera to obtain a second outer contour feature of the backboard and a circuit feature of the interface, and mapping the position of the circuit feature to a visual angle of the first camera according to a corresponding relation between the first outer contour feature and the second outer contour feature and a position relation between the second outer contour feature and the circuit feature. A visual identification characteristic value is obtained; and filtering the visual identification characteristic value by combining particle filtering and Kalman filtering, and assembling the flexible circuit board by combining the filtered characteristic value. According to the invention, the visual servo technology under the multi-camera visual field is used to accurately identify the flexible circuit board, and the problem that the camera cannot directly obtain the characteristics of the circuit board due to the fact that the visual field is shielded in the assembly process is solved.

Description

Technical Field

[0001] The present invention relates to the field of industrial vision technology, and in particular to a flexible circuit board assembly method and system based on multi-camera field of view visual servoing. Background Art

[0002] In the field of industrial production, the assembly of flexible circuit boards has always been a difficult point. Flexible circuit boards have the characteristics of small size, thin thickness, easy bending, high customization, complex working conditions, and dense circuits on the board. There are several problems in realizing automated assembly. First, the front circuit board requires a high-precision interface for assembly. However, during the assembly process, the camera's field of view is blocked, and only the backplane can be seen, not the circuit board features, so identification and positioning are difficult. Second, it is difficult to grasp, and improper grasping can easily damage the circuit board. Therefore, high-precision servo control is required to prevent the circuit board from being damaged during the grasping process. Therefore, in actual production, manual assembly is usually required. The solution to realize automated assembly of flexible circuit boards is meaningful and meets market demand.

[0003] Visual servoing is a technology that uses computer vision information to achieve closed-loop control. It has the characteristics of high precision and flexible control. It obtains image information and target images in real time for calculation, and quickly and accurately feeds back visual information to the robot, thereby achieving control of the robot.

[0004] In the Chinese patent document with the announcement number CN220776150U, a flexible circuit board that is easy to install is disclosed, which belongs to the technical field of flexible circuit boards and includes a fixing plate, a circuit board body is arranged above the fixing plate, and two groups of sockets are provided on the upper surface of the circuit board body, each of which is provided with a plug rod inside, and the bottom surface of each plug rod is fixedly connected to the upper surface of the fixing plate, and a slot is provided on the side of the two groups of plug rods that are close to each other. Since the camera field of view is blocked during the assembly process, only the back plate can be seen but the circuit board features cannot be seen, so the flexible circuit board still needs to be assembled manually.

[0005] In order to solve the problem of difficulty in automatic assembly of flexible circuit boards, an automatic assembly method of flexible circuit boards is needed. Summary of the invention

[0006] In view of the defects in the prior art, an object of the present invention is to provide a flexible circuit board assembly method and system based on multi-camera field of view visual servoing.

[0007] According to a flexible circuit board assembly method based on multi-camera field of view visual servoing provided by the present invention, the flexible circuit board includes a backplane and an interface, and the assembly method includes:

[0008] Step S1: positioning the flexible printed circuit board by using a first camera to obtain a first outer contour feature of the backplane, and grabbing the backplane according to the first outer contour feature;

[0009] Step S2: positioning the flexible printed circuit board by a second camera to obtain a second outer contour feature of the backplane and a circuit feature of the interface; mapping the position of the circuit feature to the viewing angle of the first camera according to the corresponding relationship between the first outer contour feature and the second outer contour feature, and the position relationship between the second outer contour feature and the circuit feature, to obtain a visual recognition feature value;

[0010] Step S3: filtering the visual recognition feature values ​​in combination with a filtering algorithm, and assembling the flexible circuit board in combination with the filtered feature values.

[0011] Preferably, the step S1 includes the following sub-steps:

[0012] Step S1.1: performing rough positioning of the back plate of the flexible circuit board by using a first camera to obtain a rough positioning result of the back plate;

[0013] Step S1.2: finely locate the edge in the rough positioning result by using an edge contour detection algorithm to obtain a first outer contour feature of the backplane;

[0014] Step S1.3: Input the first outer contour feature into the servo system to complete the clamping of the flexible circuit board backplane.

[0015] Preferably, the first outer contour feature is a feature line or a feature point; the feature line includes a contour edge line, and the feature point includes an intersection point of a contour edge line, or a key point identified by a key point model.

[0016] Preferably, the step S2 includes the following sub-steps:

[0017] Step S2.1: positioning the flexible printed circuit board by a second camera to obtain a second outer contour feature of the backplane, a circuit feature of the interface, and a positional relationship between the two sets of features under the viewing angle of the second camera;

[0018] Step S2.2: Obtain the field of view sizes of the first camera and the second camera, and calculate the field of view size ratio between the two cameras;

[0019] Step S2.3: selecting the same reference feature from the first outer contour feature and the second outer contour feature, and obtaining a field of view rotation ratio according to an angular relationship between the two reference features;

[0020] Step S2.4: Calculate the position of the circuit feature under the first camera viewing angle, i.e., the visual recognition feature value, based on the positional relationship between the circuit feature and the second outer contour, the field of view size ratio, and the field of view rotation ratio.

[0021] Preferably, the filtering algorithm is a particle filter and a Kalman filter algorithm, and filtering the visual recognition feature value by combining the particle filter and the Kalman filter includes:

[0022] Step S3.1: Obtain visual recognition feature values ​​and initialize M particles with equal weights around the circuit feature position, each particle representing a possible state;

[0023] Step S3.2: Predict the next movement state of the particle according to the motion model, calculate the similarity between each particle and the observed feature value at this moment, and update the weight so that the particle that matches the observed value more closely gets a larger weight;

[0024] Step S3: Use some particles with large weights to calculate the state estimate and predict the covariance matrix of the state estimate as the prior estimate of step S4;

[0025] Step S4: Calculate the Kalman gain by using the state covariance matrix and the noise covariance matrix, and update the state estimate and the covariance matrix by using the gain;

[0026] Step S5: Iteratively update the particle weight and gain calculation to make the state estimate approach the true state and converge.

[0027] According to the present invention, a flexible circuit board assembly system based on multi-camera field of view visual servoing is provided, comprising:

[0028] Module M1: Positioning the flexible circuit board through the first camera, obtaining the first outer contour feature of the backplane, and grabbing the backplane according to the first outer contour feature;

[0029] Module M2: Position the flexible circuit board through the second camera to obtain the second outer contour feature of the backplane and the circuit feature of the interface. According to the correspondence between the first outer contour feature and the second outer contour feature, and the position relationship between the second outer contour feature and the circuit feature, the position of the circuit feature is mapped to the viewing angle of the first camera to obtain the visual recognition feature value.

[0030] Module M3: Filter the visual recognition feature values ​​in combination with the filtering algorithm, and assemble the flexible circuit board in combination with the filtered feature values.

[0031] Preferably, the module M1 includes the following submodules:

[0032] Module M1.1: Roughly position the backplane of the flexible circuit board through the first camera to obtain the rough positioning result of the backplane;

[0033] Module M1.2: Use edge contour detection algorithm to precisely locate the edge in the rough positioning result to obtain the first outer contour feature of the backplane;

[0034] Module M1.3: Input the first outer contour feature into the servo system to complete the clamping of the flexible circuit board backplane.

[0035] Preferably, the first outer contour feature is a feature line or a feature point; the feature line includes a contour edge line, and the feature point includes an intersection point of a contour edge line, or a key point identified by a key point model.

[0036] Preferably, the module M2 includes the following submodules:

[0037] Module M2.1: Position the flexible printed circuit board through the second camera to obtain the second outer contour feature of the backplane, the circuit feature of the interface, and the positional relationship between the two sets of features under the perspective of the second camera;

[0038] Module M2.2: Obtain the field of view of the first camera and the second camera, and calculate the field of view ratio between the two cameras;

[0039] Module M2.3: Select the same reference feature from the first outer contour feature and the second outer contour feature, and obtain the field of view rotation ratio according to the angular relationship between the two reference features;

[0040] Module M2.4: Calculate the position of the circuit feature under the first camera viewing angle, i.e., the visual recognition feature value, based on the positional relationship between the circuit feature and the second outer contour, the field of view size ratio, and the field of view rotation ratio.

[0041] According to a flexible circuit board assembly device provided by the present invention, the flexible circuit board assembly method based on multi-camera field of view visual servoing is executed when the assembly device is running.

[0042] Compared with the prior art, the present invention has the following beneficial effects:

[0043] 1. The present invention uses visual servo technology under the field of view of multiple cameras to accurately identify flexible circuit boards, solving the problem that the camera cannot directly obtain the circuit board features due to the obstruction of the field of view during the assembly process.

[0044] 2. The present invention uses a multi-stage visual algorithm to track and locate materials. Compared with single-stage positioning, multi-stage positioning can filter out irrelevant information, reduce the possibility of false detection, and have higher positioning accuracy. Multi-stage positioning can first quickly and preliminarily locate the area of ​​interest, and only needs to accurately locate the detection features within the area, thereby reducing the amount of calculation and improving calculation efficiency.

[0045] 3. The present invention uses contour features or key point features as the output of the positioning algorithm, which can be compatible with more types of materials.

[0046] 4. The present invention combines an improved visual servo method to optimize the control mode, solves the defect of low stability in traditional servo control, and completes the automatic assembly of flexible circuit boards more efficiently.

[0047] 5. The present invention solves the problem of damage to the flexible circuit board during grasping due to positioning errors and the problem of damage to the flexible circuit board due to assembly errors by high-precision positioning and assembly of the flexible circuit board. BRIEF DESCRIPTION OF THE DRAWINGS

[0048] Other features, objects and advantages of the present invention will become more apparent from the detailed description of non-limiting embodiments made with reference to the following drawings:

[0049] Figure 1 This is a flow chart of a flexible circuit board assembly method based on multi-camera field of view visual servoing of the present invention;

[0050] Figure 2 This is a schematic diagram of the rough positioning effect of the flexible circuit board in the present invention;

[0051] Figure 3 This is a schematic diagram of the precise positioning effect of the flexible circuit board in the present invention;

[0052] Figure 4 This is a schematic diagram of a flexible circuit board under the viewing angle of a second camera in the present invention;

[0053] Figure 5 It is a schematic diagram of the material positions during the assembly process of the flexible circuit board in the present invention. DETAILED DESCRIPTION

[0054] The present invention is described in detail below in conjunction with specific embodiments. The following embodiments will help those skilled in the art to further understand the present invention, but are not intended to limit the present invention in any form. It should be noted that, for those of ordinary skill in the art, several changes and improvements can also be made without departing from the concept of the present invention. These all belong to the protection scope of the present invention.

[0055] The present invention discloses a flexible circuit board assembly method based on multi-camera field of view visual servoing. The overall technical process of the method is as follows: Figure 1 As shown, the method includes three processes in sequence: identifying the backplane, identifying the circuit features, and assembling. The flexible circuit board includes a backplane and an interface. For the convenience of description, the side where the backplane of the flexible circuit board is located is defined as the back side, and the side where the interface is located is defined as the front side. First, it is necessary to identify the position of the flexible circuit board, and clamp the flexible circuit board to the assembly position through the tool on the back of the flexible circuit board. Secondly, identify the circuit features in the interface on the front side of the flexible circuit board, map the circuit features to the back side of the flexible circuit board, and use them as the input of the servo. Finally, the automated assembly is completed through servo control.

[0056] The flexible circuit board assembly method based on multi-camera field of view visual servoing disclosed in the present invention is further described below.

[0057] The backplane identification process is as follows:

[0058] First, the back image of the flexible circuit board at the first viewing angle is captured by the first camera. Due to the small volume of the material, the edge of the target is first roughly positioned using a coarse positioning algorithm to obtain a coarse positioning result of the backplane.

[0059] In a preferred embodiment, Figure 2 As shown, for a rectangular flexible circuit board, an object detection algorithm is used to roughly locate the four edges of the backplane to obtain the result predicted by the model, i.e., the part selected by the rectangular box in the figure. The object detection algorithm can be SSD, yolo series, R-CNN series and other algorithms. Of course, the object detection algorithm is not limited to the listed algorithms, and any algorithm that can achieve rough positioning of materials belongs to the protection scope of the present invention.

[0060] Secondly, the edge contour detection algorithm based on the AI ​​model is used to precisely locate the edge in the rough positioning result to obtain the first outer contour feature f of the backplane. l .

[0061] In a specific embodiment, the Figure 3 Taking the edge of the back panel as an example, the precise positioning algorithm can use Hough transform, LineNet, etc.

[0062] The advantages of using a coarse-fine two-level positioning algorithm over direct fine positioning are as follows: The first point is higher accuracy; because the coarse positioning algorithm has the characteristic of screening feature areas, that is, by learning and predicting the area of ​​interest, a large number of irrelevant background areas are excluded, and more attention is paid to the target area, thereby filtering irrelevant information and reducing missed detection. However, since the full image has a wide range and contains a large amount of irrelevant background information, this may interfere with precise positioning; noise, clutter or other target-like structures in the background may mislead the detection algorithm, resulting in a decrease in algorithm accuracy; therefore, if direct fine positioning is used, it will increase the difficulty of recognition. The second point is high computational efficiency; in the fine positioning stage, algorithms with high accuracy usually have higher complexity. If only the fine positioning method is used, the entire image needs to be traversed, and the computational efficiency is low. The coarse-fine positioning method can quickly and preliminarily locate the area of ​​interest first, and only the detection features need to be precisely located in the area.

[0063] The first outer contour feature f output by the edge contour detection algorithm l It can be the edge line of the backboard contour, the feature point of the backboard contour, or the key point information of the backboard; taking a rectangular backboard as an example, the four contour lines of the backboard are output:

[0064] Line0=[Line0x1,Line0y1,Line0x2,Line0y2]

[0065] Line1=[Line1x1,Line1y1,Line1x2,Line1y2]

[0066] Line2=[Line2x1,Line2y1,Line2x2,Line2y2]

[0067] Line3=[Line3x1,Line3y1,Line3x2,Line3y2]

[0068] Line0, Lne1, Line2, and Line3 are used as the output values ​​of the edge contour detection algorithm.

[0069] Output the feature points of the backplate contour:

[0070] In some cases, the output is based on the visual features of key points. The intersection of four edge lines can be calculated to obtain the coordinates of four key points. For example, the intersection of Line0 and Line1 is used to solve the linear equation system.

[0071] For Line0, let:

[0072] x1,y1=(Line0x1,Line0y1)

[0073] x2,y2=(Line0x2,Line0y2)

[0074] Similarly, for Line1:

[0075] x3,y3=(Line1x1,Line1y1)

[0076] x4,y4=(Line1x2,Line1y2)

[0077] The equation system can be obtained:

[0078] (y2-y1)x-(x2-x1)y=x1y2-x2y1

[0079] (y4-y3)x-(x4-x3)y=x3y4-x4y3

[0080] Solving the equations will yield the intersection of the two lines, R = (x, y). Find the four intersections in turn and sort them by calculating the center points of the four intersections, then sorting the intersections in a clockwise or counterclockwise direction by specifying an anchor vector.

[0081] Output the key points of the backplane: The key point model can be used to directly predict the positions of the key points of the backplane as visual output values.

[0082] For servo, the visual part needs to return feature values ​​in a fixed order, and the type of feature values ​​is not limited, for example, it can be the above-mentioned contour or key point.

[0083] Finally, after the servo system obtains the returned contour feature value, it calculates the contour value of the corresponding backplane position on the fixture in real time during the actual operation process. By using the servo control to make the current value approach the returned contour feature value, the goal of identifying and grasping the backplane can be achieved.

[0084] The process of identifying circuit characteristics is as follows:

[0085] The front side of the flexible circuit board is as follows Figure 4 As shown, since the inner circuit cannot be directly touched during actual assembly, the back panel can only be sucked up from the back of the flexible circuit board with a tool.

[0086] Figure 4 The figure shows the positional relationship of the materials in the assembly, with the flexible circuit board at the top and the interface at the bottom, and the circuit structure located in the interface at the bottom. Since the position of the circuit cannot be seen directly from above, it is necessary to first identify the key points of the circuit board features and map them to the position of the backplane during the assembly process.

[0087] Specifically, the method for mapping the circuit feature position to the backplane includes:

[0088] Step S1: Install a second camera under the flexible circuit board. The second viewing angle of the second camera is as follows: Figure 4 According to the method of identifying the backplane process, the interface circuit feature F under the second perspective is obtained. e , and the second outer contour feature F of the flexible circuit board backplane l , and the relative position relationship between the two sets of features in the image coordinate system of the second perspective can be obtained. Here, the circuit feature V e It can be the contour feature of the circuit structure at the interface or the position feature of the pins within the interface.

[0089] Step S2: Calculate the field of view ratio V of the two cameras by using the aspect ratio of the field of view of the two cameras. s .

[0090] Step S3: Select the same contour edge in the two cameras to calculate the angle relationship, and the field of view rotation ratio V can be obtained. r .

[0091] Step S4: Obtain the first outer contour feature f of the backplane from the first camera through model prediction l .

[0092] Step S5: Based on the circuit characteristics and the second outer contour characteristics F l The positional relationship and the field of view size ratio V s and the field of view rotation ratio V r , calculate the circuit characteristic F e The position in the first perspective is represented by f e .

[0093] Specifically, in step S1, we get F e =[(x1,y1),(x2,y2)],F l =[(a1, b1), (a2, b2)]. The positional relationship between the two sets of features can be represented by the distance D and the angle R. Usually, for the convenience of calculation, the two cameras can be installed in parallel, that is, the field of view rotation ratio V in step S3 r =1, then the position relationship can be simplified and represented by a vector as F e F l =V r [[(x1-a1),(y1-b1),[(x2-a2),(y2-b2)]], from F e Start by moving F along the x-axis and y-axis respectively e F l The calculated vector can represent the relationship between the two sets of features. The outer edge of the flexible circuit board is usually a rectangle. If the two cameras are not installed parallel, select the long side or short side of the outer contour recognized by the model in the two cameras and calculate the angle relationship.

[0094] In step S2, the camera field of view ratio is measured. Since the x and y direction vectors were used to calculate the position relationship in the previous step, the field of view ratio is calculated here by the field of view length-width ratio. Assuming that the current field of view size of the first camera is 180mm*60mm and the current field of view size of the second camera is 120mm*40mm, the field of view length-width ratio of the two cameras is the same, and V is obtained. s =A / B=1.5. If the aspect ratios of the two fields of view are inconsistent, two ratios need to be calculated respectively, and the x vector and the y vector are divided by the corresponding ratios respectively. Let step S4 calculate f l =[(a3,b3),(a4,b4)], then the result of step S5 is f e =f l -(F e F l / V s ).

[0095] The assembly process is as follows:

[0096] like Figure 5 As shown in the figure, the circuit characteristics f of the slot at the assembly target position are obtained through the coarse-fine positioning algorithm. s , and f obtained in the previous stepe The feature value F as the output of visual recognition output =[f e ,f s ]. These two sets of feature values ​​are returned to the robot in real time as visual output information. Subsequently, the robot is controlled by servo to assemble the flexible circuit board under the field of view of multiple cameras; the assembly here refers to the assembly of the interface, such as the assembly of the backplane plug and the slot at the assembly target position, or the assembly of the backplane slot and the plug at the assembly target position.

[0097] For traditional visual servoing, the control goal is to minimize the error R between the current image features and the desired target features:

[0098] R=S(P f ,a)-P t

[0099] P f Here is the image feature coordinate value of the visual output. f ,α) is the visual feature vector, which is used to describe the position of the target in the current image, where a is the camera internal parameter, P t is the target expected value. By selecting a control scheme and designing a speed controller, the relationship between the eigenvector and the speed variable is obtained by solving the Jacobian matrix. In addition, it is necessary to calculate the difference between the input image eigenvector and the target eigenvector to obtain the error signal. Based on the Jacobian matrix and the error signal, the posture and speed of the end motion of the robot arm can be further calculated, so that the robot can accurately track the target in the image.

[0100] Therefore, it can be seen that the traditional servo method is very dependent on visual information, but there are often two problems in complex scenes. First, the visual output may lose frames, that is, the visual result is not received at a certain moment; second, when the difference between the visual output frequency and the control frequency is too large, it will also affect the control of the robot.

[0101] In order to reduce the impact of the above two problems on visual servoing, the present invention uses filtering to post-process the visual output value. The specific method is: assuming that the visual return frequency is 20ms, the control frequency is 5ms, and the difference is 15ms. In these 15ms, the robot movement is controlled by the filtering prediction result at a control frequency of 5ms / time until the visual result is received, and then the visual result is used as the true value at this moment. Avoid the robot freezing or obvious wrong movement direction caused by the above two problems.

[0102] Hybrid filtering mainly uses the ideas of particle filtering and Kalman filtering. The complete process is as follows:

[0103] Step A1: Initialize M particles with equal weights around the circuit feature position, each particle represents a possible state. The size of M depends on the comprehensive consideration of the reliability of the visual output feature value and the operation time.

[0104] Step A2: Predict the next movement state of the particle according to the motion model, calculate the similarity between each particle and the observed feature value at this moment, and update the weight so that the particle that matches the observed value more closely gets a larger weight.

[0105] Step A3: Take a weighted average of the particles with large weights and calculate a new state value to represent the state-related values ​​such as the position and speed of the current state of the target feature. In addition, the covariance matrix of the predicted state estimate is used as a priori estimate for step 4. The covariance matrix represents the confidence of the state estimate, which is estimated by the particle weight value and the motion model. Here, the motion model can be selected according to the actual situation, for example, a nonlinear model, a Markov model, a Gaussian model, etc. can be selected.

[0106] Step A4: The Kalman gain is calculated by using the state covariance matrix and the noise covariance matrix. The previously estimated covariance matrix is ​​denoted as p k,k-1 , k and k-1 represent the current and previous moments respectively. The noise covariance matrix is ​​denoted as R k It is used to represent the current statistical characteristics of noise. If the noise comes from vibration or friction, it can be obtained through physical experimental measurement. If the source is unknown, the parameter value input can be adjusted by statistical average data. The state vector is converted into the matrix H of the measurement vector at this moment. k , is its transposed matrix. The calculation of Kalman gain can be expressed as:

[0107]

[0108] Step A5: Iteratively update the particle weight and gain calculation, which is achieved by comparing the similarity between the predicted state of the particle and the observed eigenvalue. After the weight is updated, the larger the particle weight is, the more it matches the observed value, and therefore the higher the priority. In the iteration, the state estimate and covariance matrix are continuously updated through the Kalman gain, so that the current state is close to the real state and converges, improving the accuracy and reliability of the visual eigenvalue for servo control.

[0109] The invention also provides a flexible circuit board assembly system based on multi-camera field of view visual servoing. The flexible circuit board assembly system based on multi-camera field of view visual servoing can be realized by executing the process steps of the flexible circuit board assembly method based on multi-camera field of view visual servoing, that is, those skilled in the art can understand the flexible circuit board assembly method based on multi-camera field of view visual servoing as a preferred implementation of the flexible circuit board assembly system based on multi-camera field of view visual servoing.

[0110] The present invention provides a flexible circuit board assembly system based on multi-camera field of view visual servoing, comprising:

[0111] Module M1: Position the flexible printed circuit board through the first camera to obtain the first outer contour feature of the back plate, and grab the back plate according to the first outer contour feature.

[0112] The module M1 includes the following submodules:

[0113] Module M1.1: Roughly position the backplane of the flexible circuit board through the first camera to obtain the rough positioning result of the backplane;

[0114] Module M1.2: Use edge contour detection algorithm to precisely locate the edge in the rough positioning result to obtain the first outer contour feature of the backplane;

[0115] Module M1.3: Input the first outer contour feature into the servo system to complete the clamping of the flexible circuit board backplane.

[0116] The first outer contour feature is a feature line or a feature point; the feature line includes a contour edge line, and the feature point includes an intersection point of a contour edge line, or a key point identified by a key point model.

[0117] Module M2: The flexible circuit board is positioned by the second camera to obtain the second outer contour feature of the backplane and the circuit feature of the interface. According to the correspondence between the first outer contour feature and the second outer contour feature, and the positional relationship between the second outer contour feature and the circuit feature, the position of the circuit feature is mapped to the viewing angle of the first camera to obtain the visual recognition feature value.

[0118] The module M2 includes the following submodules:

[0119] Module M2.1: Position the flexible printed circuit board through the second camera to obtain the second outer contour feature of the backplane, the circuit feature of the interface, and the positional relationship between the two sets of features under the perspective of the second camera;

[0120] Module M2.2: Obtain the field of view of the first camera and the second camera, and calculate the field of view ratio between the two cameras;

[0121] Module M2.3: Select the same reference feature from the first outer contour feature and the second outer contour feature, and obtain the field of view rotation ratio according to the angular relationship between the two reference features;

[0122] Module M2.4: Calculate the position of the circuit feature under the first camera viewing angle, i.e., the visual recognition feature value, based on the positional relationship between the circuit feature and the second outer contour, the field of view size ratio, and the field of view rotation ratio.

[0123] Module M3: Combine particle filtering and Kalman filtering to filter the visual recognition feature values, and assemble the flexible circuit board based on the filtered feature values. The filtering process includes:

[0124] Module M3.1: Obtain visual recognition feature values ​​and initialize M particles with equal weights around the circuit feature position, where each particle represents a possible state;

[0125] Module M3.2: Predict the next movement state of the particle based on the motion model, calculate the similarity between each particle and the observed feature value at the moment, and update the weight so that the particle that matches the observed value more closely gets a larger weight.

[0126] Module M3: Use some particles with large weights to calculate the state estimate and predict the covariance matrix of the state estimate as the prior estimate of step S4;

[0127] Module M4: The Kalman gain is calculated by the state covariance matrix and the noise covariance matrix, and the state estimate and the covariance matrix are updated by the gain;

[0128] Module M5: Iteratively update the particle weights and gain calculations to make the state estimate approach the true state and converge.

[0129] The present invention also discloses a flexible circuit board assembly device, which executes the above-mentioned flexible circuit board assembly method based on multi-camera field of view visual servoing when running.

[0130] Those skilled in the art know that, in addition to realizing the system and its various devices, modules, and units provided by the present invention in a purely computer-readable program code, it is entirely possible to realize the same functions in the form of logic gates, switches, application-specific integrated circuits, programmable logic controllers, and embedded microcontrollers by logically programming the method steps. Therefore, the system and its various devices, modules, and units provided by the present invention can be considered as a hardware component, and the devices, modules, and units included therein for realizing various functions can also be regarded as structures within the hardware component; the devices, modules, and units for realizing various functions can also be regarded as both software modules for realizing the method and structures within the hardware component.

[0131] In the description of the present application, it should be understood that the terms "up", "down", "front", "back", "left", "right", "vertical", "horizontal", "top", "bottom", "inside", "outside", etc., indicating orientations or positional relationships, are based on the orientations or positional relationships shown in the accompanying drawings, and are only for the convenience of describing the present application and simplifying the description, and do not indicate or imply that the referred device or element must have a specific orientation, be constructed and operated in a specific orientation, and therefore should not be understood as a limitation on the present application.

[0132] The above describes the specific embodiments of the present invention. It should be understood that the present invention is not limited to the above specific embodiments, and those skilled in the art can make various changes or modifications within the scope of the claims, which does not affect the essence of the present invention. In the absence of conflict, the embodiments of the present application and the features in the embodiments can be combined with each other arbitrarily.

Claims

1. A flexible circuit board assembly method based on multi-camera field of view visual servoing, characterized in that: The flexible circuit board includes a back plate and an interface, and the assembly method includes: Step S1: positioning the flexible circuit board by using a first camera to obtain a first outer contour feature of the backplane, and grabbing the backplane according to the first outer contour feature; Step S2: positioning the flexible printed circuit board by a second camera to obtain a second outer contour feature of the backplane and a circuit feature of the interface; mapping the position of the circuit feature to the viewing angle of the first camera according to the corresponding relationship between the first outer contour feature and the second outer contour feature, and the position relationship between the second outer contour feature and the circuit feature, to obtain a visual recognition feature value; Step S3: filtering the visual recognition feature values ​​in combination with a filtering algorithm, and assembling the flexible circuit board in combination with the filtered feature values.

2. The flexible circuit board assembly method based on multi-camera field of view visual servoing according to claim 1 is characterized in that: The step S1 comprises the following sub-steps: Step S1.1: performing rough positioning of the back plate of the flexible circuit board by using a first camera to obtain a rough positioning result of the back plate; Step S1.2: finely locate the edge in the rough positioning result by using an edge contour detection algorithm to obtain a first outer contour feature of the backplane; Step S1.3: Input the first outer contour feature into the servo system to complete the clamping of the flexible circuit board backplane.

3. The flexible circuit board assembly method based on multi-camera field of view visual servoing according to claim 1 is characterized in that: The first outer contour feature is a feature line or a feature point; the feature line includes a contour edge line, and the feature point includes an intersection point of a contour edge line, or a key point identified by a key point model.

4. The flexible circuit board assembly method based on multi-camera field of view visual servoing according to claim 1 is characterized in that: The step S2 comprises the following sub-steps: Step S2.1: positioning the flexible printed circuit board by a second camera to obtain a second outer contour feature of the backplane, a circuit feature of the interface, and a positional relationship between the two sets of features under the viewing angle of the second camera; Step S2.2: Obtain the field of view sizes of the first camera and the second camera, and calculate the field of view size ratio between the two cameras; Step S2.3: selecting the same reference feature from the first outer contour feature and the second outer contour feature, and obtaining a field of view rotation ratio according to an angular relationship between the two reference features; Step S2.4: Calculate the position of the circuit feature under the first camera viewing angle, i.e., the visual recognition feature value, based on the positional relationship between the circuit feature and the second outer contour, the field of view size ratio, and the field of view rotation ratio.

5. The flexible circuit board assembly method based on multi-camera field of view visual servoing according to claim 1 is characterized in that: The filtering algorithm is a particle filter and a Kalman filter algorithm. Combining the particle filter and the Kalman filter to filter the visual recognition feature value includes: Step S3.1: Obtain visual recognition feature values ​​and initialize M particles with equal weights around the circuit feature position, each particle representing a possible state; Step S3.2: Predict the next movement state of the particle according to the motion model, calculate the similarity between each particle and the observed feature value at this moment, and update the weight so that the particle that matches the observed value more closely gets a larger weight; Step S3: Use some particles with large weights to calculate the state estimate and predict the covariance matrix of the state estimate as the prior estimate of step S4; Step S4: Calculate the Kalman gain by using the state covariance matrix and the noise covariance matrix, and update the state estimate and the covariance matrix by using the gain; Step S5: Iteratively update the particle weight and gain calculation to make the state estimate approach the true state and converge.

6. A flexible circuit board assembly system based on multi-camera field of view visual servoing, characterized in that: include: Module M1: Positioning the flexible circuit board through the first camera, obtaining the first outer contour feature of the backplane, and grabbing the backplane according to the first outer contour feature; Module M2: Position the flexible circuit board through the second camera to obtain the second outer contour feature of the backplane and the circuit feature of the interface. According to the correspondence between the first outer contour feature and the second outer contour feature, and the position relationship between the second outer contour feature and the circuit feature, the position of the circuit feature is mapped to the viewing angle of the first camera to obtain the visual recognition feature value. Module M3: Filter the visual recognition feature values ​​in combination with the filtering algorithm, and assemble the flexible circuit board in combination with the filtered feature values.

7. The flexible circuit board assembly system based on multi-camera field of view visual servoing according to claim 6 is characterized in that: The module M1 Includes the following submodules: Module M1.1: Roughly position the backplane of the flexible circuit board through the first camera to obtain the rough positioning result of the backplane; Module M1.2: Use edge contour detection algorithm to precisely locate the edge in the rough positioning result to obtain the first outer contour feature of the backplane; Module M1.3: Input the first outer contour feature into the servo system to complete the clamping of the flexible circuit board backplane.

8. The flexible circuit board assembly system based on multi-camera field of view visual servoing according to claim 6 is characterized in that: The first outer contour feature is a feature line or a feature point; the feature line includes a contour edge line, and the feature point includes an intersection point of a contour edge line, or a key point identified by a key point model.

9. The flexible circuit board assembly system based on multi-camera field of view visual servoing according to claim 6, characterized in that: The module M2 Includes the following submodules: Module M2.1: Position the flexible printed circuit board through the second camera to obtain the second outer contour feature of the backplane, the circuit feature of the interface, and the positional relationship between the two sets of features under the perspective of the second camera; Module M2.2: Obtain the field of view of the first camera and the second camera, and calculate the field of view ratio between the two cameras; Module M2.3: Select the same reference feature from the first outer contour feature and the second outer contour feature, and obtain the field of view rotation ratio according to the angular relationship between the two reference features; Module M2.4: Calculate the position of the circuit feature under the first camera viewing angle, i.e., the visual recognition feature value, based on the positional relationship between the circuit feature and the second outer contour, the field of view size ratio, and the field of view rotation ratio.

10. A flexible circuit board assembly device, characterized in that: The assembly equipment executes the flexible circuit board assembly method based on multi-camera field of view visual servoing according to any one of claims 1 to 5 when in operation.

Images