Automatic assembly system, control method and storage medium for steel plate pressing of mobile phones

Abstract

This application relates to the fields of computer vision and robot control technology, and particularly to an automatic assembly system, control method, and storage medium for mobile phone steel plates. The system includes a fixing component for securing the mobile phone circuit board; an assembly unit for assembling the mobile phone steel plate to the target position on the mobile phone circuit board; a measuring unit for measuring the first pose of the mobile phone steel plate, the second pose of the mobile phone circuit board, image features of the latches and slots, and distance features of the target point on the mobile phone steel plate; and a controller for controlling the assembly unit to assemble the mobile phone steel plate based on the first pose, the second pose, and the image features, and for detecting whether the assembly of the mobile phone steel plate is complete based on the image features and distance features. This solves the problems in related technologies, such as the uncertain initial position of the steel plate, difficulty in accurate identification, and the inability to meet the assembly accuracy requirements of mobile phone steel plates due to the accumulation of various errors.

Description

Technical Field

[0001] This application relates to the fields of computer vision and robot control technology, and in particular to an automatic assembly system, control method and storage medium for mobile phone steel plates. Background Technology

[0002] With the continuous development of technology, automated assembly technology for mobile phone steel clamps has begun to attract attention. Steel clamps are a type of part used to press and hold components on mobile phone circuit boards, such as flexible flat cable connectors, cameras, and coaxial cable connectors. In mobile phone repair shops, the reinstallation of steel clamps still relies heavily on manual labor, resulting in significant wear and tear on workers. The process involves picking up the steel clamp, inserting its first clip into the corresponding slot on the circuit board, rotating and pressing the clamp, and then inserting the remaining clips into the slots. Automated reinstallation of steel clamps in repair scenarios faces three major challenges: first, the XY assembly accuracy of the steel clamp clips and circuit board slots is as high as 0.2mm; second, the initial position of the steel clamp is unpredictable in repair scenarios, requiring precise positioning of the clips and slots, but these are small and have inconspicuous features, making accurate identification difficult; and third, the complex clamping action can easily lead to abnormal contact with the circuit board during assembly, causing unexpected deformation of the parts. In summary, to implement automated steel plate reloading in maintenance scenarios, both measurement accuracy and operational flexibility must be considered.

[0003] Because the initial position of the steel clamping plate is uncertain in repair scenarios, it is impossible to achieve high-precision reassembly of the steel clamping plate using repetitive robot movements. To locate and manipulate the steel clamping plate at different initial positions, a robot capable of online measurement of the part's spatial pose is required. Conventional camera-calibrated vision robots suffer from accumulated errors from calibration, measurement, and robot motion, exceeding the assembly precision requirements of mobile phone steel clamping plates. Therefore, automated reassembly of mobile phone steel clamping plates in repair scenarios requires a robot measurement and control method that combines measurement accuracy with operational flexibility. Summary of the Invention

[0004] This application provides an automatic assembly system, control method, and storage medium for mobile phone steel plates to solve problems in related technologies, such as uncertain initial position of the steel plate, difficulty in accurate identification, and the inability to meet the assembly accuracy requirements of mobile phone steel plates due to the accumulation of various errors.

[0005] The first aspect of this application provides an automatic assembly system for a mobile phone steel plate, comprising: a fixing component for fixing a mobile phone circuit board; an assembly unit for assembling a mobile phone steel plate to a target position on the mobile phone circuit board; a measuring unit for measuring a first pose of the mobile phone steel plate, a second pose of the mobile phone circuit board, image features of the buckle and slot, and distance features of a target point on the mobile phone steel plate; and a controller for controlling the assembly unit to assemble the mobile phone steel plate according to the first pose, the second pose, and the image features, and for detecting whether the assembly of the mobile phone steel plate is complete according to the image features and the distance features.

[0006] Optionally, the assembly unit includes: an assembly robot with multi-degree-of-freedom motion; a suction cup air channel disposed at the front end of the assembly robot; and a flexible suction cup connected to the suction cup air channel for picking up the steel plate of the mobile phone.

[0007] Optionally, the measuring unit includes: a multi-degree-of-freedom gimbal; an image acquisition device mounted on the multi-degree-of-freedom gimbal; an X-direction linear slide rail mounted on the plane of the assembly robot base; a Y-direction linear slide rail mounted between the X-direction linear slide rails, wherein the Y-direction linear slide rails are movable along the X-direction between the X-direction linear slide rails; and a ranging device mounted on the Y-direction linear slide rails, wherein the ranging device is movable along the Y-direction on the Y-direction linear slide rails.

[0008] A second aspect of this application provides a control method for an automatic assembly system of a mobile phone steel plate. The method is used to control the automatic assembly system of the mobile phone steel plate as described above, and includes the following steps: fixing a mobile phone circuit board; measuring the first pose of the mobile phone steel plate, the second pose of the mobile phone circuit board, and the image features of the buckle on the mobile phone steel plate and the slot on the mobile phone circuit board; controlling the assembly part to assemble the mobile phone steel plate according to the first pose, the second pose, and the image features; and detecting whether the mobile phone steel plate is assembled completely according to the image features and the distance features of the target point on the mobile phone steel plate.

[0009] Optionally, controlling the assembly part to assemble the mobile phone steel plate according to the first pose, the second pose, and the image features includes: controlling the assembly part to pick up the mobile phone steel plate according to the first pose; placing the buckle of the mobile phone steel plate on the slot base at the target position according to the first pose and the second pose; guiding the alignment and insertion between the first buckle of the mobile phone steel plate and the first slot on the slot base based on the image features; and rotating and pressing the steel plate to complete the fastening between the second buckle of the mobile phone steel plate and the second slot on the slot base.

[0010] Optionally, the alignment and insertion between the first buckle of the mobile phone steel plate and the first card slot on the card slot base based on the image features includes: identifying the first buckle edge line and the first card slot edge line in the image features; abstracting multiple endpoints from the pixel sets of the first buckle edge line and the first card slot edge line respectively, and determining the card slot center line, card slot left edge line, card slot right edge line and card slot bottom line based on the multiple endpoints; using the vertical distance from the center point of the first buckle to the center line of the card slot as the deviation distance of the first buckle from the first card slot, using the left edge line and the right edge line of the card slot as the card slot boundary, adjusting the deviation distance based on the card slot boundary until the deviation distance is less than or equal to a preset distance, completing the alignment between the first buckle and the first card slot, aligning the first buckle with the first card slot, and completing the insertion between the first buckle and the first card slot when the distance between the center point of the first buckle and the bottom line of the card slot is less than the preset distance.

[0011] Optionally, the step of detecting whether the mobile phone steel plate is assembled based on the image features and the distance features includes: obtaining a model projection image of the mobile phone steel plate; marking multiple target points in the model projection image, and calculating the homography matrix from the model projection image to the actual image based on the image features during the assembly process, transforming the target points into the actual image; for the mobile phone steel plate after the first buckle is aligned and inserted, taking the position of each target point as the target position, measuring the height of the multiple points on the mobile phone steel plate, and taking the height as the target value; for the mobile phone steel plate after the second buckle is fastened, if the difference in horizontal distance between the position of each target point and the target position is less than a horizontal threshold, it is determined that the steel plate is in place in the horizontal direction; if the difference in height between the height of the multiple points and the target value is less than a height threshold, it is determined that the steel plate is in place in the height direction.

[0012] Optionally, measuring the first pose of the mobile phone steel plate and the second pose of the mobile phone circuit board includes: acquiring respective three-dimensional models of the mobile phone steel plate and the mobile phone circuit board; collecting the point cloud of the three-dimensional model onto the imaging plane to generate respective model projection images of the mobile phone steel plate and the mobile phone circuit board, and recording the correspondence between the pixels of the model projection image and the point cloud; segmenting the outline image based on the grayscale threshold of the respective physical images of the mobile phone steel plate and the mobile phone circuit board, and determining the corresponding pose based on the feature matching results of the model projection image and the outline image and the correspondence.

[0013] Optionally, before measuring the first pose of the mobile phone steel plate and the second pose of the mobile phone circuit board, the method further includes: adjusting the poses of the respective three-dimensional models of the mobile phone steel plate and the mobile phone circuit board so that the angular deviation between the model normal and the normal of the actual object is less than a preset deviation.

[0014] Optionally, measuring the image features of the buckle on the mobile phone steel plate includes: marking the buckle edge line on the outline of the model projection of the mobile phone steel plate; transforming the outline of the model projection to a physical image using the shape features of the buckle to form a first template image; performing feature matching based on a first actual image of the mobile phone steel plate during actual assembly and the first template image, calculating a first homography matrix from the first template image to the first actual image, and transforming the outline of the first template image to the first actual image based on the first homography matrix to obtain the image features of the buckle on the mobile phone steel plate.

[0015] Optionally, measuring the image features of the card slot on the mobile phone circuit board includes: marking the card slot edge line on the image of the mobile phone circuit board to form a second template image; performing feature matching based on the actual image of the mobile phone circuit board during assembly and the second template image, calculating a second homography matrix from the second template image to the actual image, and transforming the card slot edge line to the second actual image based on the second homography matrix to obtain the image features of the card slot on the mobile phone circuit board.

[0016] A third aspect of this application provides a computer-readable storage medium having a computer program stored thereon, which is executed by a processor to implement the control method of the automatic assembly system for mobile phone steel sheet as described in the above embodiments.

[0017] Therefore, this application has at least the following beneficial effects:

[0018] This application's embodiments utilize scale-invariant features of part images to match their shape and surface texture. A 3D model and a physical image are used to roughly determine the part's pose. Image features are precisely located through feature matching, and the image edges of the steel pressure plate buckle and circuit board slot are abstracted. Then, the positional relationship between the buckle and slot is calculated. A closed-loop control robotic arm completes the lateral alignment and longitudinal insertion of the buckle and slot. Finally, a monocular camera and a height gauge are used to detect whether the 3D coordinates of key points on the steel pressure plate reach the target range, thereby determining whether the steel pressure plate has been successfully assembled. This solves the technical problems in related technologies, such as the uncertain initial position of the steel pressure plate, difficulty in accurate identification, and the inability to meet the assembly accuracy requirements of mobile phone steel pressure plates due to the accumulation of various errors.

[0019] Additional aspects and advantages of this application will be set forth in part in the description which follows, and in part will be obvious from the description, or may be learned by practice of this application. Attached Figure Description

[0020] The above and / or additional aspects and advantages of this application will become apparent and readily understood from the following description of the embodiments taken in conjunction with the accompanying drawings, wherein:

[0021] Figure 1This is a schematic diagram of an automatic assembly system for mobile phone steel plates according to an embodiment of this application;

[0022] Figure 2 This is a schematic diagram of an automatic assembly system for mobile phone steel plates according to an embodiment of this application;

[0023] Figure 3 This is a flowchart of the control method for the automatic assembly system of mobile phone steel plate according to the embodiments of this application;

[0024] Figure 4 This is a three-dimensional model diagram of the steel pressure plate provided according to the embodiments of this application;

[0025] Figure 5 This is a diagram illustrating the fastening operation of the second clip of the steel pressure plate according to an embodiment of this application;

[0026] Figure 6 This is an image feature map of the buckle slot provided according to an embodiment of this application;

[0027] Figure 7 This is a flowchart illustrating the snap-fit ​​and slot alignment and insertion according to an embodiment of this application;

[0028] Figure 8 This is a schematic diagram of the initial pose of a three-dimensional model provided according to an embodiment of this application;

[0029] Figure 9 This is a schematic diagram showing the actual orientation of the circuit board and steel pressure plate according to an embodiment of this application;

[0030] Figure 10 This is a schematic diagram illustrating the edge markings of the buckle according to an embodiment of this application;

[0031] Figure 11 This is a drawing of a steel pressure plate template provided according to an embodiment of this application;

[0032] Figure 12 This is a diagram illustrating the positioning effect of the buckle slot edge according to an embodiment of this application.

[0033] Figure 13 This is a schematic diagram illustrating the calculation of the positional relationship between the buckle and the slot according to the embodiments of this application;

[0034] Figure 14 This is a flowchart of a control method for an automatic assembly system for steel plates in mobile phones according to an embodiment of this application.

[0035] Explanation of reference numerals in the attached drawings: 1-Assembly robot, 2-Suction cup air duct, 3-Flexible suction cup, 4-Steel pressure plate, 5-Clamp, 6-Mobile phone circuit board, 7-Six-degree-of-freedom gimbal, 8-Camera, 9-Laser rangefinder, 10-X-direction linear slide rail, 11-Y-direction linear slide rail. Detailed Implementation

[0036] The embodiments of this application are described in detail below. Examples of these embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals denote the same or similar elements or elements having the same or similar functions throughout. The embodiments described below with reference to the accompanying drawings are exemplary and intended to explain this application, and should not be construed as limiting this application.

[0037] The following description, with reference to the accompanying drawings, outlines an automatic assembly system, control method, and storage medium for mobile phone steel plates according to embodiments of this application. Addressing the problem mentioned in the background art of robot measurement and control methods that cannot simultaneously achieve both measurement accuracy and operational flexibility, this application provides an automatic assembly system for mobile phone steel plates. This system utilizes scale-invariant features of part images to match their shape and surface texture. It uses a 3D model and a physical image to roughly measure the part's pose, accurately locates image features through feature matching, and abstracts the image edges of the steel plate clips and circuit board slots. Then, it calculates the positional relationship between the clips and slots, and uses a closed-loop control robot arm to complete the lateral alignment and longitudinal insertion of the clips and slots. Finally, it uses a monocular camera and a height gauge to detect whether the 3D coordinates of key points on the steel plate have reached the target range, thereby determining whether the steel plate assembly is successful. This solves the problems in related technologies, such as uncertain initial positions of the steel plates, difficulty in accurate identification, and the inability to meet the assembly accuracy requirements of mobile phone steel plates due to the accumulation of various errors.

[0038] Specifically, Figure 1 This is a schematic diagram of the automatic assembly system for mobile phone steel plates provided in an embodiment of this application.

[0039] like Figure 1 As shown, the automatic assembly system 10 for steel plates in mobile phones includes: a fixing part 100, an assembly part 200, a measuring part 300, and a controller 400.

[0040] The fastener 100 is used to fix the mobile phone circuit board; the assembly part 200 is used to assemble the mobile phone steel plate to the target position of the mobile phone circuit board; the measuring part 300 is used to measure the first pose of the mobile phone steel plate, the second pose of the mobile phone circuit board, the image features of the buckle and the slot, and the distance features of the target point on the mobile phone steel plate; the controller 400 is used to control the assembly part to assemble the mobile phone steel plate according to the first pose, the second pose and the image features, and to detect whether the mobile phone steel plate is assembled successfully according to the image features and the distance features.

[0041] It is understood that the embodiments of this application can assemble the mobile phone steel plate to the target position of the mobile phone circuit board, control the assembly part to assemble the mobile phone steel plate according to the measured first pose, second pose and image features, and detect whether the mobile phone steel plate is assembled, thereby achieving precise positioning and ensuring successful assembly.

[0042] In this embodiment, the assembly unit 200 includes: an assembly robot 1 with multi-degree-of-freedom motion; a suction cup air channel 2 disposed at the front end of the assembly robot 1; and a flexible suction cup 3 connected to the suction cup air channel 2 for picking up the steel pressure plate 4 of the mobile phone.

[0043] The flexible suction cup 2 may include suction cups made of silicone, fluororubber, polyurethane, etc.

[0044] It is understood that in this embodiment of the application, a suction cup air channel 2 can be set at the front end of the assembly robot 1, and the connected flexible suction cup 3 can pick up the steel pressure plate 4 of the mobile phone. The assembly robot 1 can then pick up and assemble the phone, which is simpler and more efficient.

[0045] In this embodiment, the measurement unit 300 includes: a multi-degree-of-freedom gimbal; an image acquisition device mounted on the multi-degree-of-freedom gimbal; an X-direction linear slide rail 10 mounted on the plane of the assembly robot 1 base; a Y-direction linear slide rail 11 mounted between the X-direction linear slide rails 10, wherein the Y-direction linear slide rail 11 can move along the X-direction between the X-direction linear slide rails 10; and a ranging device mounted on the Y-direction linear slide rail 11, wherein the ranging device can move along the Y-direction on the Y-direction linear slide rail 11.

[0046] Among them, a multi-degree-of-freedom gimbal can be a six-degree-of-freedom gimbal, etc.

[0047] It is understood that, in the embodiments of this application, the X-direction linear slide rail 10 and the Y-direction linear slide rail 11 can be mounted on the plane of the robot base by the bracket, so that the Y-direction linear slide rail 11 moves along the X-direction and the ranging device moves along the Y-direction, thereby detecting whether the three-dimensional coordinates of the key points of the steel pressure plate have reached the target position and determining whether the steel pressure plate has been successfully assembled.

[0048] The automatic assembly system for mobile phone steel plates proposed in this application can match the shape and surface texture of a part by utilizing the scale-invariant features of the part image. It uses a 3D model and a physical image to roughly measure the part's pose, accurately locates image features through feature matching, and abstracts the image edges of the steel plate clips and circuit board slots. Then, it calculates the positional relationship between the clips and slots, and uses a closed-loop control robotic arm to complete the lateral alignment and longitudinal insertion of the clips and slots. Finally, it uses a monocular camera and a height gauge to detect whether the 3D coordinates of key points on the steel plate have reached the target range, thus determining whether the steel plate assembly is successful. This solves the problems in related technologies, such as the uncertain initial position of the steel plate, difficulty in accurate identification, and the inability to meet the assembly accuracy requirements of mobile phone steel plates due to the accumulation of various errors.

[0049] The automatic assembly system for mobile phone steel plates provided in this application will be described in detail below through specific embodiments, such as... Figure 2 As shown, the details are as follows:

[0050] During assembly, the mobile phone circuit board 6 is fixed to the table by the clamp 5. The camera 8 is fixed on the six-degree-of-freedom gimbal 7 to view the mobile phone circuit board 6 from above. The camera 8 performs visual measurement to guide the assembly robot 1 to operate the steel pressure plate 4 to complete the insertion and fastening of the clips. After the steel pressure plate fastening operation is completed, the camera 8 and the laser rangefinder 9 detect whether the three-dimensional coordinates of the key points of the steel pressure plate 4 have reached the target position to determine whether the steel pressure plate has been successfully assembled.

[0051] In summary, the embodiments of this application propose a method for monocular measurement of the spatial pose of mobile phone steel plates, image edge positioning of buckles and slots, visual guidance assembly, and assembly success detection. These methods combined can complete the automated assembly of mobile phone steel plates. The embodiments of this application can complete the fully automated assembly of mobile phone steel plates and can be used for automated assembly tasks of mobile phone steel plates or similar parts of various shapes.

[0052] Next, with reference to the accompanying drawings, a control method for an automatic assembly system for mobile phone steel plates according to an embodiment of this application is described.

[0053] Figure 3 This is a flowchart illustrating the control method of the automatic assembly system for mobile phone steel plates according to an embodiment of this application.

[0054] like Figure 3 As shown, the control method of the automatic assembly system for steel plates in mobile phones includes the following steps:

[0055] In step S101, the mobile phone circuit board is fixed.

[0056] It is understood that the embodiments of this application can fix the mobile phone circuit board, which can be the internal circuit board of the mobile phone, connecting various components and performing precise control of electronic components and information.

[0057] In step S102, the first pose of the mobile phone steel plate, the second pose of the mobile phone circuit board, and the image features of the buckle on the mobile phone steel plate and the slot on the mobile phone circuit board are measured.

[0058] Among them, pose can be the position and orientation in a spatial coordinate system.

[0059] It is understood that the embodiments of this application can measure the first pose of the steel pressure plate, the second pose of the circuit board, and the image features of the buckle and the slot on the mobile phone circuit board. The image features are accurately located through feature matching. If there is a deviation in the pose, the pose is controlled and corrected in time to adjust the pose to the optimal state.

[0060] In step S103, the assembly unit is controlled to assemble the mobile phone steel plate according to the first pose, the second pose and the image features, and the assembly of the mobile phone steel plate is detected according to the image features and the distance features of the target point on the mobile phone steel plate.

[0061] The target point can be any point on the steel plate of the mobile phone.

[0062] It is understood that the embodiments of this application can control the assembly part to assemble the mobile phone steel plate according to the first pose, the second pose and image features, and detect whether the mobile phone steel plate is assembled by the distance features of the target point. The distance features of the target point can be visually measured by the camera. For each type of steel plate, the corresponding assembly action is completed, thereby improving the repeatability, efficiency and accuracy of robot assembly.

[0063] In this embodiment, the assembly of the mobile phone steel plate is controlled by the assembly unit according to the first pose, the second pose, and image features. The assembly unit picks up the mobile phone steel plate according to the first pose; the buckle of the mobile phone steel plate is placed on the slot base at the target position according to the first pose and the second pose; the alignment and insertion between the first buckle of the mobile phone steel plate and the first slot on the slot base is guided based on image features; and the steel plate is rotated and pressed to complete the fastening between the second buckle of the mobile phone steel plate and the second slot on the slot base.

[0064] The target location can be the key point of the steel pressing plate.

[0065] It is understood that, in the embodiments of this application, the assembly part can pick up the mobile phone steel plate through the first posture control, and place the buckle of the mobile phone steel plate on the slot base at the target position with the second posture. The buckle slot is aligned and inserted by using image features, and finally the steel plate is rotated and pressed to complete the fastening action, thereby realizing automated assembly.

[0066] It should be noted that, as Figure 4 As shown, most steel pressure plates have two clips. The first clip requires insertion through rigid body movement, while the second clip is fastened after the steel pressure plate undergoes elastic deformation. Figure 5 As shown, during operation, the first clip of the steel pressure plate is first inserted into the corresponding slot. The second clip will mechanically interfere with the corresponding slot. Only after the steel pressure plate clip is deformed by pressing down can the second clip be inserted into the slot.

[0067] In this embodiment, the alignment and insertion between the first buckle of the mobile phone steel plate and the first card slot on the card slot base guided by image features includes: identifying the edge line of the first buckle and the edge line of the first card slot in the image features; abstracting multiple endpoints from the pixel points of the respective first buckle edge line and the first card slot edge line, and determining the card slot center line, the left edge line, the right edge line, and the bottom edge line based on the multiple endpoints; using the vertical distance from the center point of the first buckle to the center line of the card slot as the deviation distance of the first buckle from the first card slot, using the left edge line and the right edge line of the card slot as the card slot boundary, adjusting the deviation distance based on the card slot boundary until the deviation distance is less than or equal to a preset distance, thus completing the alignment between the first buckle and the first card slot; aligning the first buckle with the first card slot; and completing the insertion between the first buckle and the first card slot when the distance between the center point of the first buckle and the bottom edge line of the card slot is less than the preset distance.

[0068] The preset distance can be 0.2mm, etc.

[0069] It is understood that the embodiments of this application can identify the edge line of the first buckle and the edge line of the first slot, abstract multiple endpoints, determine the center line of the slot, the left side line of the slot, the right side line of the slot, and the bottom line of the slot, take the vertical distance from the center point of the first buckle to the center line of the slot as the deviation distance of the first buckle from the first slot, take the left side line and the right side line of the slot as the slot boundary, adjust the deviation distance to be less than or equal to the preset distance, complete the alignment between the first buckle and the first slot, and then insert the first buckle into the first slot. When the distance between the center point of the buckle and the bottom line of the slot is less than the preset distance, the insertion between the first buckle and the first slot is completed.

[0070] Specifically, such as Figure 6 As shown, after identifying the edge line of the buckle and slot, the relationship between the buckle and slot is first determined. The vertical distance from the center point of the buckle to the center line of the slot is taken as the distance of the buckle deviating from the slot, and the left and right lines of the slot are taken as the boundaries of the slot.

[0071] As an example, such as Figure 7 As shown, if the buckle falls outside the card slot boundary, adjust the buckle horizontally. The adjustment distance is the distance the buckle deviates from the card slot, aligning the center point of the buckle with the center line of the card slot. If the buckle is inside the card slot boundary, insert it vertically into the card slot until the distance between the center point of the buckle and the bottom line of the card slot is less than the set threshold.

[0072] In this embodiment, detecting whether the mobile phone steel plate assembly is complete based on image features and distance features includes: obtaining a model projection image of the mobile phone steel plate; marking multiple target points in the model projection image, and calculating the homography matrix from the model projection image to the actual image based on image features during the assembly process, transforming the target points into the actual image; for the mobile phone steel plate after the first buckle is aligned and inserted, taking the position of each target point as the target position, measuring the height of multiple points on the mobile phone steel plate, and taking the height as the target value; for the mobile phone steel plate after the second buckle is fastened, if the difference in horizontal distance between the position of each target point and the target position is less than a horizontal threshold, it is determined that the steel plate is in place in the horizontal direction; if the difference in height between the height of multiple points and the target value is less than a height threshold, it is determined that the steel plate is in place in the height direction.

[0073] The horizontal threshold can be 0.3mm, etc., and the height threshold can be 0.4mm, etc.

[0074] It is understood that the embodiments of this application can transform the target point into the real image based on the homography matrix of the image feature calculation model projection map to the real image, and then measure the height of the inserted mobile phone steel plate. For the mobile phone steel plate with the second buckle fastened, if the difference in horizontal distance between the position of each target point and the target position is less than the horizontal threshold, it is determined that the steel plate is in place in the horizontal direction. If the height difference between the height of multiple points and the target value is less than the height threshold, it is determined that the steel plate is in place in the height direction. Since each type of steel plate has its unique fastening operation, the embodiments of this application can control the robot to complete the corresponding fastening operation, thereby expanding its practicality.

[0075] Specifically, after the first clip is inserted into the slot, the steel clamping plate needs to be operated to insert the second clip into the slot. There are many steel clamping plates of different shapes on the mobile phone circuit board, each with its unique fastening action. For each type of steel clamping plate, the robot needs to be controlled to complete the corresponding fastening action.

[0076] After the robot completes all the fastening actions, it is necessary to check whether the steel pressure plate is successfully assembled. This inspection process is carried out in two steps: first, to check whether the steel pressure plate is in place in the horizontal direction, and second, to check whether the steel pressure plate is in place in the vertical direction.

[0077] The methods for checking whether the steel pressure plate is in the correct horizontal and vertical directions are as follows:

[0078] 1) Mark more than 3 key points in the projection drawing of the steel plate model. When positioning the buckle edge during the assembly process, use the scale-invariant feature matching to calculate the homography matrix from the model projection drawing to the actual drawing, and transform the key points into the actual drawing.

[0079] 2) For the successfully assembled steel pressure plate, record the position of each key point as the target position;

[0080] 3) Move the laser altimeter to a fixed location, select at least 3 locations, and measure their height as the target value;

[0081] 4) After the robot completes the fastening action, it takes a picture to detect the key points of the steel plate and compares them with the key points of successful assembly. If the distance of each key point from the target position is less than the set threshold, it is considered that the steel plate is in place in the horizontal direction.

[0082] 5) Move the laser height measuring instrument to the previously determined point, measure the height in sequence, and compare it with the corresponding target value. If the difference between each measured value and the corresponding target value is less than the set threshold, it is considered that the steel pressure plate is in place in the height direction.

[0083] In this embodiment of the application, measuring the first pose of the mobile phone steel plate and the second pose of the mobile phone circuit board includes: acquiring the respective three-dimensional models of the mobile phone steel plate and the mobile phone circuit board; collecting the point cloud of the three-dimensional model onto the imaging plane to generate the respective model projection images of the mobile phone steel plate and the mobile phone circuit board, and recording the correspondence between the pixels of the model projection image and the point cloud; segmenting the outline image based on the grayscale threshold of the respective physical images of the mobile phone steel plate and the mobile phone circuit board, and determining the corresponding pose based on the feature matching results and correspondence between the model projection image and the outline image.

[0084] It is understood that the embodiments of this application can match model points and image pixels by point cloud projection fusion image feature matching to obtain the pose of the projection relative to the object, and then calculate the pose of the object in the model projection coordinate system.

[0085] Specifically, the core of the pose measurement method in this application is to match the three-dimensional model space points, referred to as "model points", with the photo pixels, and use the matching results to solve the Perspective-n-Point problem (hereinafter referred to as "PnP problem") to obtain the pose of the camera relative to the object, and then calculate the pose of the object in the camera coordinate system.

[0086] The PnP problem can be solved and coordinate transformation can be achieved using existing formulas, but matching model points with image pixels is a problem that urgently needs to be solved.

[0087] This application embodiment uses point cloud projection fusion image feature matching to match model points and image pixels. The specific method is as follows:

[0088] 1) Perform dense point cloud sampling on the 3D model of the steel plate, such as... Figure 8 As shown, the initial pose of the 3D model is obtained;

[0089] 2) Projecting point clouds onto an imaging plane using the pinhole imaging principle, such as... Figure 9 As shown, a projection image of the steel plate is generated, and the correspondence between the pixels of the projection image and the point cloud is recorded;

[0090] 3) Take a picture of the steel press against a black background, and use threshold segmentation to extract the shape of the steel press;

[0091] 4) Perform feature matching on the projection image and outline image of the steel plate, and then match the model points and outline image pixels through the correspondence between the projection image and the point cloud.

[0092] In the feature matching process between the projection image and the outline image, scale-invariant features of the images are extracted, and the matching relationship between pixels in the projection image and the outline image is established based on these scale-invariant features. After matching the model points and the outline image pixels, the pose of the actual steel sheet can be calculated by solving the PnP problem.

[0093] To improve the reliability of pose measurement, the posture of the steel plate model should be adjusted so that the deviation between the normal of the model and the normal of the actual object is as small as possible.

[0094] The circuit board pose measurement is also performed using a similar method. When matching the model projection and the physical shape, the selected feature point is the center of the circuit board through hole. This feature is more significant and stable than the shape feature, and the pose measurement result is more accurate.

[0095] In this embodiment of the application, before measuring the first pose of the mobile phone steel plate and the second pose of the mobile phone circuit board, the method further includes: adjusting the pose of the respective three-dimensional models of the mobile phone steel plate and the mobile phone circuit board so that the angular deviation between the model normal and the normal of the actual object is less than a preset deviation.

[0096] The preset deviation can be 1 degree, etc.

[0097] It is understood that the embodiments of this application can adjust the posture of the three-dimensional model so that the angular deviation between the normal of the model and the normal of the object is less than a preset deviation, thereby improving the accuracy of the measurement.

[0098] In this embodiment of the application, measuring the image features of the buckle on the mobile phone steel plate includes: marking the buckle edge line on the outline of the model projection of the mobile phone steel plate; transforming the outline of the model projection to the physical image using the shape features of the buckle to form a first template image; performing feature matching based on the first actual image and the first template image of the mobile phone steel plate during actual assembly, calculating the first homography matrix from the first template image to the first actual image, and transforming the outline of the first template image to the first actual image based on the first homography matrix to obtain the image features of the buckle on the mobile phone steel plate.

[0099] It is understood that, in the embodiments of this application, the shape features of the buckle can be used to form a first template image and a first real image for feature matching, calculate the first homography matrix, obtain the image features of the buckle, and achieve precise positioning of the buckle edge.

[0100] In this embodiment of the application, measuring the image features of the card slot on the mobile phone circuit board includes: marking the card slot edge line on the image of the mobile phone circuit board to form a second template image; performing feature matching based on the actual image of the mobile phone circuit board during actual assembly and the second template image, calculating the second homography matrix from the second template image to the actual image, and transforming the card slot edge line to the second actual image based on the second homography matrix to obtain the image features of the card slot on the mobile phone circuit board.

[0101] It is understood that, in the embodiments of this application, the image of the mobile phone circuit board can be located first, and then a second template image can be formed through the card slot. The template image is then matched with the actual image to calculate the second homography matrix and obtain the image features of the card slot, thereby achieving accurate positioning of the card slot edge.

[0102] Specifically, because the buckle and slot are tiny and have trivial features, it is difficult to accurately locate the buckle and slot edges directly from the image. Therefore, it is necessary to perform buckle and slot image feature localization. The method for locating the buckle edge is as follows:

[0103] 1) such as Figure 10 As shown, mark the snap-on edge lines on the outline of the steel plate projection;

[0104] 2) Transform the outline of the projected image to the physical image using shape feature matching, such as... Figure 11 As shown, a template diagram is formed;

[0105] 3) Take actual assembly photos, use the template image and the actual assembly photos to perform scale-invariant feature matching, and calculate the homography matrix from the template image to the actual assembly photos;

[0106] 4) Use the homography matrix described above to transform the outline of the template drawing (with the snap-fit ​​edges marked) into the actual drawing.

[0107] The specific method for positioning the edge line of the card slot is as follows:

[0108] 1) Take a photo of the circuit board and mark the card slot edges on it to use as a template;

[0109] 2) Take photos of the circuit board during actual assembly, perform scale-invariant feature matching with the template image, and calculate the homography matrix from the template image to the circuit board photo;

[0110] 3) Use the homography matrix described above to transform the card slot edge lines into the circuit board photograph.

[0111] As an example, using a 2K resolution camera at an object distance of approximately 300mm, the edge line of the latch slot is located against a complex circuit background, such as... Figure 12 As shown, the positioning error of the buckle slot edge is 1 pixel, which translates to a physical size of approximately 0.04 mm.

[0112] It should be noted that the specific process for calculating the positional relationship of the features in the buckle and slot image is as follows: Figure 13 As shown, during the actual operation of the steel pressing plate, due to measurement errors, the edge line of the steel pressing plate buckle is not strictly parallel to the edge line of the circuit board slot, but will have a slight deviation angle δθ.

[0113] After extracting the edge line of the buckle slot from the image, four points p1, p2, p3, and p4 are abstracted from the pixel set of the slot edge line as ideal endpoints, and four points p5, p6, p7, and p8 are abstracted from the pixel set of the buckle edge line as ideal endpoints. The buckle center point p0 = (p5 + p6) / 2. The point p0 passes through the midpoint of p1 and p2. 10 And the midpoint p of p3p4 11 Draw the center line l0 of the card slot, draw the left side line l1 of the card slot through p1 and p4, and draw the right side line l2 of the card slot through p2 and p3.

[0114] Let pixel p i The coordinates are represented by (u) i ,v i ) indicates that the straight line l j Using equation A j u+B j v+C j =0 means that, where A j B is the coefficient of u. j C is the coefficient of v. j As constants, the equations for the center line and left and right edges of the slot are expressed as follows:

[0115] l0:(v 10 -v 11 )u+(u 11 -u 10 )v+u 10 v 11 -u 11 v 10 =0

[0116] l1:(v1-v4)u+(u4-u1)v+u1v4-u4v1=0

[0117] l2:(v2-v3)u+(u3-u2)v+u2v3-u3v2=0

[0118] pixel p i (u i ,v iSubstitute the line l j The result obtained on the left side of the equation is represented by l. j (p i If ), then the following formula can be used to determine whether the buckle is inside the boundary of the slot.

[0119] l j (p i )·l j (p 10 ), i = 5, 6, 7, 8, j = 1, 2

[0120] In the above formula, there are 8 combinations based on the different values ​​of i and j, resulting in 8 expressions. If all 8 expressions are greater than 0, then...

[0121]

[0122] At this point, it is assumed that the buckle is located inside the boundary of the slot, and the buckle is inserted vertically.

[0123] If any one of the eight expressions is less than 0, that is...

[0124]

[0125] This indicates that the latch has exceeded the card slot boundary. At this point, adjust the latch horizontally. The adjustment distance is the distance from the latch center point p0(u0,v0) to the card slot center line l0 multiplied by the ratio of the physical distance to the pixel distance (pre-calibrated).

[0126]

[0127] Where A0 is the coefficient of u0, B0 is the coefficient of v0, and C0 is a constant.

[0128] Move the latch laterally along the line containing p1 and p2, so that p0 moves closer to l0.

[0129] The control method of the automatic assembly system for mobile phone steel plates provided in this application will be described in detail below through specific embodiments, such as... Figure 14 As shown, the details are as follows:

[0130] Before the system is officially put into operation, the camera's intrinsic parameters and spatial pose relative to the robot's base coordinate system are calibrated in advance. During system operation, the poses of the steel pressure plate and the circuit board are measured first. Then, the suction cup at the end of the robot's operation is moved to pick up the steel pressure plate. Based on the pose measurement results, the steel pressure plate is operated to place its clips on the circuit board's slot base. Then, the robot is controlled while taking pictures, and the image features are used to guide the clip slots to align and insert. Finally, the steel pressure plate is rotated and pressed to complete the fastening. After the robot's fastening action is completed, image recognition and laser ranging are used to detect whether the steel pressure plate is successfully assembled.

[0131] The core components of the automated assembly system include four parts: steel plate and circuit board pose measurement, buckle and slot image feature positioning, visual guidance buckle and slot alignment and insertion, and steel plate fastening completion inspection.

[0132] In summary, the embodiments of this application are based on a 3D model and use a monocular camera to measure the spatial pose of the mobile phone steel plate. The hardware is simple, and the method is universal, convenient, and easy to deploy quickly. By using the shape information of the 3D model, the edge line of the buckle slot is accurately located using image feature matching, which can effectively monitor the assembly process of the mobile phone steel plate. The robot is guided to complete the assembly of the mobile phone steel plate based on the positional relationship of image features, and the assembly process is simple and efficient. After the assembly is completed, a monocular camera and a laser sensor are used to detect whether the steel plate is properly fastened, forming a fully closed loop of automated assembly.

[0133] It should be noted that the foregoing explanation of the automatic assembly system for mobile phone steel plates also applies to the control method of the automatic assembly system for mobile phone steel plates in this embodiment, and will not be repeated here.

[0134] The control method for the automatic assembly system of mobile phone steel plate according to the embodiments of this application can match the shape and surface texture of the part by using the scale-invariant features of the part image. The part pose is roughly measured using a 3D model and a physical image. Image features are accurately located through feature matching, and the image edges of the steel plate buckle and circuit board slot are abstracted. Then, the positional relationship between the buckle and slot is calculated. A closed-loop control robotic arm completes the lateral alignment and longitudinal insertion of the buckle and slot. Finally, a monocular camera and a height gauge are used to detect whether the 3D coordinates of key points of the steel plate have reached the target range, thereby determining whether the steel plate has been successfully assembled. This solves the problems in related technologies, such as the uncertain initial position of the steel plate, difficulty in accurate identification, and the inability to meet the assembly accuracy requirements of mobile phone steel plates due to the accumulation of various errors.

[0135] This application also provides a computer-readable storage medium storing a computer program thereon, which, when executed by a processor, implements the control method of the above-described automatic assembly system for mobile phone steel plates.

[0136] In the description of this specification, the references to terms such as "one embodiment," "some embodiments," "example," "specific example," or "some examples," etc., indicate that a specific feature, structure, material, or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of this application. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples. Moreover, without contradiction, those skilled in the art can combine and integrate the different embodiments or examples described in this specification, as well as the features of different embodiments or examples.

[0137] Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include at least one of that feature. In the description of this application, "N" means at least two, such as two, three, etc., unless otherwise explicitly specified.

[0138] Any process or method described in the flowchart or otherwise herein can be understood as representing a module, segment, or portion of code comprising one or N executable instructions for implementing custom logic functions or processes, and the scope of the preferred embodiments of this application includes additional implementations in which functions may be performed not in the order shown or discussed, including substantially simultaneously or in reverse order depending on the functions involved, as should be understood by those skilled in the art to which embodiments of this application pertain.

[0139] It should be understood that the various parts of this application can be implemented using hardware, software, firmware, or a combination thereof. In the above embodiments, the N steps or methods can be implemented using software or firmware stored in memory and executed by a suitable instruction execution system. For example, if implemented in hardware, as in another embodiment, it can be implemented using any one or a combination of the following techniques known in the art: discrete logic circuits having logic gates for implementing logical functions on data signals, application-specific integrated circuits (ASICs) having suitable combinational logic gates, programmable gate arrays (FPGAs), field-programmable gate arrays (FPGAs), etc.

[0140] Those skilled in the art will understand that all or part of the steps of the methods in the above embodiments can be implemented by a program instructing related hardware. The program can be stored in a computer-readable storage medium, and when executed, the program includes one or a combination of the steps of the method embodiments.

[0141] Although embodiments of this application have been shown and described above, it is understood that the above embodiments are exemplary and should not be construed as limiting this application. Those skilled in the art can make changes, modifications, substitutions and variations to the above embodiments within the scope of this application.

Claims

1. A control method for an automatic assembly system for steel sheet pressing in mobile phones, characterized in that, The method includes the following steps: Fasteners, used to secure the phone's circuit board; The assembly section assembles the mobile phone steel plate onto the target position of the mobile phone circuit board; The measurement unit measures the first pose of the mobile phone steel pressure plate, the second pose of the mobile phone circuit board, and the image features of the buckle on the mobile phone steel pressure plate and the slot on the mobile phone circuit board. The assembly part is controlled to assemble the mobile phone steel plate according to the first pose, the second pose and the image features, and the assembly part is controlled to assemble the mobile phone steel plate according to the image features and the distance features of the target point on the mobile phone steel plate. Based on the image features, the alignment and insertion between the first buckle of the mobile phone steel plate and the first slot on the slot base is guided, including: Identify the first buckle edge line and the first slot edge line in the image features; Multiple endpoints are abstracted from the pixel sets of the first buckle edge line and the first card slot edge line, and the card slot center line, card slot left line, card slot right line and card slot bottom line are determined based on the multiple endpoints. The vertical distance from the center point of the first buckle to the center line of the slot is taken as the deviation distance of the first buckle from the first slot. The left and right lines of the slot are taken as the slot boundary. The deviation distance is adjusted based on the slot boundary until the deviation distance is less than or equal to a preset distance. The alignment between the first buckle and the first slot is completed. The first buckle is aligned with the first slot. When the distance between the center point of the first buckle and the bottom line of the slot is less than the preset distance, the insertion between the first buckle and the first slot is completed. Detecting whether the mobile phone steel plate is assembled based on the image features and distance features includes: Obtain the model projection image of the steel plate of the mobile phone; Multiple target points are marked in the model projection map, and the homography matrix from the model projection map to the actual map is calculated based on the image features during the assembly process to transform the target points into the actual map; For the mobile phone steel plate after the first buckle is aligned and inserted, the position of each target point is taken as the target position, and the height of the distance from multiple points of the mobile phone steel plate is measured as the target value; For the steel pressure plate of the mobile phone after the second buckle is installed, if the difference in horizontal distance between the position of each target point and the target position is less than the horizontal threshold, it is determined that the steel pressure plate is in place in the horizontal direction. If the difference in height between the height of multiple points and the target value is less than the height threshold, it is determined that the steel pressure plate is in place in the height direction.

2. The control method of the automatic assembly system for mobile phone steel plates according to claim 1, characterized in that, The step of controlling the assembly part to assemble the mobile phone steel plate according to the first pose, the second pose, and the image features includes: The assembly part is controlled to pick up the mobile phone steel plate according to the first pose. According to the first pose and the second pose, the buckle of the mobile phone steel pressure plate is placed on the slot base at the target position; Based on the image features, guide the alignment and insertion between the first buckle of the mobile phone steel pressure plate and the first slot on the card slot base, and rotate and press the steel pressure plate to complete the fastening between the second buckle of the mobile phone steel pressure plate and the second slot on the card slot base.

3. The control method of the automatic assembly system for mobile phone steel pressing plates according to claim 1, characterized in that, Measuring the first pose of the mobile phone steel pressure plate and the second pose of the mobile phone circuit board includes: Obtain the respective three-dimensional models of the mobile phone steel plate and the mobile phone circuit board; The point cloud of the 3D model is collected onto the imaging plane to generate model projection images of the mobile phone steel plate and the mobile phone circuit board, and the correspondence between the pixels of the model projection image and the point cloud is recorded. The outline image is segmented based on the grayscale threshold of the physical images of the mobile phone steel plate and the mobile phone circuit board, and the corresponding pose is determined based on the feature matching results of the model projection image and the outline image and the correspondence.

4. The control method of the automatic assembly system for mobile phone steel plates according to claim 3, characterized in that, Before measuring the first pose of the mobile phone steel pressure plate and the second pose of the mobile phone circuit board, the following steps are also included: Adjust the orientation of the respective three-dimensional models of the mobile phone steel plate and the mobile phone circuit board so that the angular deviation between the model normal and the physical object normal is less than a preset deviation.

5. The control method of the automatic assembly system for mobile phone steel pressing plates according to claim 1, characterized in that, Measuring the image features of the latches on the steel pressure plate of the mobile phone includes: Mark the buckle edge line on the outline of the model projection of the mobile phone steel plate; The outline of the model projection is transformed into the physical image using the shape features of the buckle to form the first template image; Based on the first actual image and the first template image of the mobile phone steel plate during actual assembly, feature matching is performed to calculate the first homography matrix from the first template image to the first actual image. Based on the first homography matrix, the contour of the first template image is transformed to the first actual image to obtain the image features of the buckle on the mobile phone steel plate.

6. The control method of the automatic assembly system for mobile phone steel plates according to claim 1, characterized in that, Measuring the image features of the card slot on the mobile phone circuit board includes: Marking the card slot edge lines on the image of the mobile phone circuit board forms a second template image; Based on the actual assembly diagram of the mobile phone circuit board and the second template diagram, feature matching is performed to calculate the second homography matrix from the second template diagram to the actual diagram. Based on the second homography matrix, the card slot edge line is transformed into the second actual diagram to obtain the image features of the card slot on the mobile phone circuit board.

7. An automatic assembly system for steel pressing plates in mobile phones, characterized in that, For implementing the method as described in any one of claims 1-6, comprising: Fasteners are used to secure the mobile phone circuit board. An assembly section is used to assemble the mobile phone steel plate onto the target position of the mobile phone circuit board. The measuring unit is used to measure the first pose of the mobile phone steel plate, the second pose of the mobile phone circuit board, the image features of the buckle and the slot, and the distance features of the target point on the mobile phone steel plate; The controller is used to control the assembly part to assemble the mobile phone steel plate according to the first pose, the second pose and the image features, and to detect whether the mobile phone steel plate is assembled completely according to the image features and the distance features.

8. The automatic assembly system for mobile phone steel plates according to claim 7, characterized in that, The assembly unit includes: Assembly robots with multiple degrees of freedom of motion; The suction cup air passage is located at the front end of the assembly robot; The flexible suction cup, connected to the suction cup air passage, is used to pick up the steel pressure plate of the mobile phone.

9. The automatic assembly system for mobile phone steel plates according to claim 8, characterized in that, The measuring unit includes: Multi-degree-of-freedom gimbal; Image acquisition device mounted on the multi-degree-of-freedom gimbal; A linear slide rail in the X direction is provided on the plane of the assembly robot base; A Y-direction linear slide rail is disposed between the X-direction linear slide rails, wherein the Y-direction linear slide rails are movable along the X-direction between the X-direction linear slide rails; A ranging device is installed on the linear slide rail in the Y direction, wherein the ranging device can move along the Y direction on the linear slide rail in the Y direction.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, The program is executed by the processor to implement the control method of the automatic assembly system for mobile phone steel plates as described in any one of claims 1-6.

Images