Automatic Installation Method for Intelligent Electric Meter Module Based on Machine Vision

Abstract

The present invention relates to an automatic installation method for intelligent meter modules based on machine vision. First, a visual recognition algorithm is executed for module recognition; then, a hand-eye calibration algorithm is executed; secondly, a robot control algorithm; by executing the visual recognition algorithm, the grasping and installation of the visual manipulator are realized; in the visual recognition algorithm, there is a target detection module for detecting and recognizing the communication module of the intelligent meter of the State Grid; the communication module is white in color and placed in a black tray; the visual recognition algorithm is developed based on the OpenCV algorithm library. First, the grayscale image collected by the camera is binarized; then, the binary image is dilated morphologically to eliminate black hole noise, and the Canny edge detection algorithm is used to preliminarily screen the contours in the image through a double threshold of the edge length. The present invention is reasonable in design, compact in structure and convenient to use.

Description

Technical Field

[0001] The present invention relates to an automatic installation method for intelligent meter modules based on machine vision. Background Art

[0002] Currently, with the rapid development of the automatic installation system of a six-axis robotic arm equipped with visual sensing, it provides a development idea for the intelligence and flexibility of installation-type equipment.

[0003] Md. Hazrat Ali et al. from Nazarbayev University developed an Eye-in-Hand vision system by installing a camera at the end of the robotic arm of the robot. Sland and Rick et al. developed a pick-and-place machine application system based on visual perception technology. The team of Professor Zhong Chongquan from Dalian University of Technology applied visual perception technology to a glue spraying robot. By visually measuring, the actual position of the glue spraying point was accurately calculated, and the industrial robot could be controlled to find the glue spraying point without teaching, greatly improving the glue spraying efficiency; Xu Changyuan from South China University of Technology adopted binocular stereo vision measurement technology to achieve full-degree-of-freedom measurement of objects and applied this technology to the industrial robot for handling automotive parts, with a measurement accuracy reaching 0.5 mm; Zou Xiugong from the University of Electronic Science and Technology achieved feature positioning of docking plugs and PCB socket boards through monocular vision detection, realizing the assembly of straight-through components on the PCB board, and its positioning error could reach 0.02 mm.

[0004] Current intelligent meter module assembly equipment, when performing assembly, especially during detection, only repeats the stored operation procedures, with a single working condition and requires manual cooperation to complete. The resulting problems are: relatively strict requirements for the dimensional accuracy of meters produced by different meter case manufacturers, and slight dimensional differences may lead to module plugging and unplugging failures. The equipment also has strict requirements for the positioning accuracy of the electric meter, and positioning errors will also cause module plugging and unplugging failures.

[0005] In the above problems, in the light case, the equipment frequently alarms and the output decreases, and in the severe case, the detection equipment and the electric meter collide and damage each other. Summary of the Invention

[0006] In view of the above, the technical problem to be solved by the present invention generally is to provide an automatic installation method for intelligent meter modules based on machine vision that is reasonable in design, low in cost, durable, safe and reliable, simple to operate, time-saving and labor-saving, cost-saving, compact in structure and convenient to use; the detailed technical problems to be solved and the beneficial effects obtained are specifically described in the following content and in combination with the specific embodiments.

[0007] To solve the above problems, the technical solution adopted by the present invention is:

[0008] An automatic installation method for intelligent meter modules based on machine vision. The method performs the following steps. First, a visual recognition algorithm is executed for module recognition. Then, a hand-eye calibration algorithm is executed. Secondly, a robot control algorithm.

[0009] By executing the visual recognition algorithm, the grasping and installation of the visual robotic arm are realized.

[0010] In the visual recognition algorithm, it includes an object detection module for detecting and recognizing the communication module of the State Grid intelligent meter. The communication module is white in color and is placed in a black tray.

[0011] The visual recognition algorithm is developed based on the OpenCV algorithm library. First, the grayscale value collected by the camera is Figure 2 thresholded. Then, the binary image is dilated morphologically to eliminate black hole noise. The Canny edge detection algorithm is used to preliminarily screen the contours in the image through a double threshold of the edge length. Next, among the current contours, all closed contours adopt the quadrilateral fitting algorithm. By judging the distance between the center points of the quadrilaterals, the distances less than the set threshold are deleted. Thus, the fitting rectangles of all modules in the image and the center points of the rectangles are obtained.

[0012] Among them, this method is based on the Windows platform and is developed using the Visual Studio IDE based on the C# language.

[0013] In the thresholding, the collected image is a signal structure similar to a double peak. The method adopted for thresholding is the Otsu method.

[0014] First, with the help of σ 2 =Pf×(Mf - M) 2 +Pb×(Mb - M) 2 ; Formula (1);

[0015] The average pixel value of the default background pixels is 0, that is, Mb = 0, M = Mf / 2, and Pf + Pb is the total number of pixels in the current image. So, formula (1) is modified to:

[0016]

[0017] Among them, the quadrilateral fitting algorithm adopts the Douglas - Peucker algorithm for fitting the polygons of the closed contours. The polygon fitting is finally distinguished by the distance between the center points of the polygons and the variance of the offset between the points on the polygon and the actual contour points, and is used after fitting and screening.

[0018] In the execution of the hand - eye calibration algorithm, the hand - eye calibration algorithm combines the automated nine - point calibration algorithm for visual target detection, and adopts Eye - in - hand and Eye - to - hand;

[0019] In Eye-in-hand, in S1, the hand-eye calibration is abstractly associated with the transformation relationship between two coordinate systems, that is, the relationship between the image coordinate system and the base coordinate system of the robotic arm. Through nine pairs of corresponding points, the affine transformation matrix is directly solved and obtained.

[0020] In Eye-in-hand, in S1.1, calibrate the relationship between the camera and the center of the robotic arm flange.

[0021] First, fix the camera at the photographing point 1 and place a target object at the center position of the camera image at this time. Then, while ensuring that the camera moves to nine points and, at the same time, the target object is within the camera's field of view, determine the center coordinates of the photographed and recognized target object, and record the coordinate point of the robotic arm that is symmetric to the photographing point with respect to the photographing point. Then, the symmetric coordinate point is used as the robotic arm coordinate point. Secondly, calculate the affine transformation matrix between the nine groups of points through the affine transformation function calculated by opencv to achieve the automatic calibration of Eye-in-hand. Among them, the hand-eye calibration affine transformation matrix is through the following formula:

[0022]

[0023] Where: x---column of the pixel coordinate system; y---row of the pixel coordinate system; θ represents the rotation angle of the coordinate system; tx--translation in the x direction of the coordinate system; ty---translation in the y direction of the coordinate system; sx---scaling in the x direction of the coordinate system; sy---scaling in the y direction of the coordinate system; x1---x coordinate of the robotic arm coordinate system; y1---y coordinate of the robotic arm coordinate system. Thus, the rotation angle θ, the offset (tx, ty), and the scaling ratio (sx, sy) are calculated.

[0024] S1.2, calibrate the relationship between the center of the flange and the center of the gripper.

[0025] After calibrating the relationship between the camera and the center of the robotic arm flange, first, through the target detection algorithm, calculate the image coordinates of the target object at this time, multiply by the affine transformation matrix of formula (3), and obtain the corresponding coordinates of the center of the robotic arm end flange corresponding to the image coordinates at this time and record them as coordinates (x1, y1). Then, manually move the robotic arm to the position where the fixture grips the target object and record the coordinates (x2, y2), calculate the corresponding offset (x1 - x2, y1 - y2) between the two coordinate points, update the offset to the affine transformation matrix of formula (3), and establish a calibration relationship between the camera coordinate system and the center of the gripper.

[0026] In formula (1), the calibration relationship is divided into rotation, translation, and scaling. Without changing the photographing pose, modify the value of the new translation matrix on the original affine transformation matrix of formula (3), that is

[0027]

[0028] where x3 is the x - coordinate of the robotic arm tool coordinate system; y3 is the y - coordinate of the robotic arm tool coordinate system.

[0029] In Eye - in - hand, calibration steps are performed for different photographing points.

[0030] First, manually control the robotic arm. Under the condition of ensuring that the pose of the end - effector of the robotic arm remains unchanged, move the camera to the photographing point 2 on the same horizontal plane. Place the communication module in the camera's field of view, and obtain the center pixel coordinates of the module through the target detection module. Calculate the coordinate points in the robotic arm base coordinate system after transformation through the original affine transformation matrix and record them as coordinate point 1. Manually control the robotic arm. Under the condition of ensuring that the pose of the end - effector of the robotic arm remains unchanged, move the robotic arm to the center point position of the current module, record the current robotic arm coordinate point and record it as coordinate point 2. Fifthly, calculate the XY - direction offset between coordinate point 1 and 2. Finally, accumulate the offset into the original affine transformation matrix to obtain a new affine transformation matrix, that is, the affine transformation matrix of the current camera photographing point 2.

[0031] In Eye - to - hand, S2.1, first calibrate the hand - eye. First, after marking at the camera photographing point 1, control the camera to photograph and calibrate at the photographing point 2 according to the set path. Then, the robotic arm grabs the target object and moves nine times on the same plane directly above the camera. During the movement, locate the center pixel coordinates of the bottom end of the target object recognized by the camera 2, and at the same time, the control system reads the robotic arm coordinates of the robotic arm at nine positions, records nine groups of corresponding points between coordinate systems, and realizes the solution of the affine transformation matrix.

[0032] S2.2, then calibrate the tool. First, under the condition of ensuring the pose remains unchanged, the robotic arm grabs the target object and reaches the installation position and records the installation point 1, that is, coordinate point 1. Then, after the fixture grabs the target object, it reaches the photographing point of the camera photographing point 2, records the photographing point coordinate 1 - P1(x1, y1) of the camera photographing point 2. Obtain the image coordinates of the target object at this time through target detection, and convert them into the robotic arm coordinate system coordinates 2 - P2(x2, y2) through the hand - eye calibration affine transformation matrix. By calculating the offset between coordinate 2 and coordinate 1, apply this offset to the installation point 1 - A1(x3, y3), and then the final installation position A2(x4, y4) can be obtained.

[0033]

[0034] x1 is the robotic arm x - coordinate of the camera photographing point during calibration;

[0035] y1 is the robotic arm y - coordinate of the camera photographing point during calibration;

[0036] x2---The x coordinate of the robotic arm after affine transformation of the current image coordinates;

[0037] y2---The y coordinate of the robotic arm after affine transformation of the current image coordinates;

[0038] x3---The x coordinate of the robotic arm at the gripper mounting point during calibration;

[0039] y3---The y coordinate of the robotic arm at the gripper mounting point during calibration;

[0040] x4---The x coordinate of the robotic arm at the final mounting point;

[0041] y4---The y coordinate of the robotic arm at the final mounting point.

[0042] In the robot control algorithm, the SDK interface provided by the Aubo robot is re - encapsulated in C# language. The SDK interface includes the robot's point - to - point axis movement interface, the robot's current position acquisition interface, and the robot's current status acquisition interface;

[0043] The hardware system of the robot includes a base. There is a robotic arm on the base, a robot head on the robotic arm, and a camera and a fixture are set on the robot head; The fixture has a gripper for grasping the module; The gripper is provided with a telescopic cylinder and a clamping cylinder for driving the telescopic and clamping of the gripper. The fixture clamps the module through the clamping cylinder; The telescopic cylinder is equipped with a position sensor for detecting whether the module is installed in place. After judging that the installation is completed, if the module is installed in place, the telescopic cylinder is in the fully extended limit state and the position sensor has a signal state. If the module is not installed in place, that is, the pins of the module are not aligned, the telescopic cylinder is in an incompletely extended state, that is, the position sensor signal is in a no - signal state;

[0044] One of the cameras is mounted on the fixture, and the shooting direction is from top to bottom for module grasping, recognition and positioning; Another camera is mounted on the transportation platform of the module, and the shooting direction is from bottom to top for module installation, recognition and positioning;

[0045] The hardware system also includes a meter positioning system for positioning the meter at a fixed installation position. The meter positioning system includes a manipulator for the suction and flipping of the meter; The manipulator for the suction and flipping of the meter is equipped with a rotary cylinder to realize the flipping of the meter. The manipulator has suction cups to realize the simultaneous suction and flipping of two meters.

[0046] The design of the present invention is reasonable, which improves the detection accuracy, increases the output, and avoids collision damage and many other effects.

[0047] The beneficial effects of the present invention are not limited to this description. For better understanding, a more detailed description is given in the specific implementation part. Brief Description of the Drawings

[0048] Figure 1 It is a schematic diagram of the vision positioning and installation mechanism of the present invention.

[0049] Figure 2 It is a schematic diagram of the vision positioning algorithm of the present invention.

[0050] Figure 3 It is the vision algorithm process of the present invention.

[0051] Figure 4 It is the flow chart of the Eye-in-hand calibration algorithm of the present invention.

[0052] Figure 5 It is the flow chart of the multi-photographing point algorithm of the present invention.

[0053] Figure 6 It is the schematic diagram of the Eye-to-hand calibration process of the present invention.

[0054] Figure 7 Schematic diagram of the three-dimensional structure of the carrier module assembly system for electric energy meters. Detailed implementation manner

[0055] The vision positioning and installation mechanism of the present invention is shown in Figures 1-7 ,

[0056] The present invention adopts a software system and a hardware system.

[0057] Among them, the software system is developed on the Windows platform using the Visual Studio IDE based on the C# language, and it includes three parts: vision recognition algorithm, hand-eye calibration algorithm, and robot control algorithm. The overall algorithm flow chart is as shown in the process Figure 2 .

[0058] The vision recognition algorithm is used for module recognition, as shown in the vision algorithm process Figure 3 :

[0059] Among them, the target detection module is mainly used to detect and recognize the communication module of the smart electric energy meter of the State Grid. Among them, the communication module is cuboid in shape, white in color, and placed in a black tray. The overall algorithm idea of the target recognition algorithm is as shown in Figure 3 : The vision algorithm is developed based on the OpenCV algorithm library. First, the grayscale image collected by the camera Figure 2Value quantization: The current image has a signal structure similar to a double-peak. The method used for binarization is THRESH_OTSU (Otsu's method). Then, the binary image is dilated morphologically to eliminate the black hole noise inside the module affected by factors such as lighting. The Canny edge detection algorithm is used to preliminarily screen the contours in the image through a double threshold for the edge length. Next, considering that the module itself is approximately rectangular, for all closed contours in the current contour, a quadrilateral (polygon) fitting algorithm is used. To prevent one contour from fitting multiple quadrilaterals, those with a relatively short distance between the center points of the quadrilaterals and less than the set threshold are deleted. Finally, the fitted rectangles of all modules in the image and the center points of the rectangles are obtained.

[0060] Among them, the Otsu's method used for binarization is different from the formula before improvement. The formula before improvement is as follows:

[0061] σ 2 = Pf × (Mf - M) 2 + Pb × (Mb - M) 2 Formula (1);

[0062] Among them, σ2---variance of gray values; Pf – number of target pixels; Mf - average gray value of the foreground; M - overall average gray value of the image; Pb - number of background pixels; Mb - average gray value of the background;

[0063] Since the collected background is a black image, the average pixel value of the default background pixels is set to 0, that is, Mb = 0, M = Mf / 2, and Pf + Pb is the total number of pixels in the current image. Therefore, the formula is finally modified to:

[0064]

[0065] The entire formula calculation is optimized, reducing the detection time.

[0066] The polygon fitting algorithm used therein is the Douglas - Peucker algorithm, which is used to fit the polygons of closed contours. The polygon fitting combination is finally distinguished by the distance between the center points of the polygons and the variance of the offset between the points on the polygon and the actual contour points. It is used after fitting and screening.

[0067] Such as Figure 4, The hand-eye calibration algorithm can achieve various tasks such as grasping and installation involving the cooperation of the entire set of equipment with a machine vision robotic arm. It combines the Eye-in-hand and Eye-to-hand methods, and at the same time improves the previous hand-eye calibration algorithm using a calibration board, updating it to an automated nine-point calibration algorithm combined with visual target detection. The specific idea of the present invention is as follows: First, it reduces the difficulty of calibrating the on-site camera, abstractly associates hand-eye calibration with the conversion relationship between two coordinate systems, that is, the relationship between the image coordinate system and the robotic arm base coordinate system, and directly solves for the affine transformation matrix through nine pairs of corresponding points. The specific implementation process of Eye-in-hand can be seen in the Eye-in-hand flowchart.

[0068] First, calibrate the relationship between the camera and the center of the robotic arm flange;

[0069] First, fix the camera at the photo-taking point 1, and place a target at the center position of the camera image at this time (approximate center is okay); then, on the premise of ensuring that the target does not exceed the camera's field of view while the camera moves nine points, determine the center coordinates of the photographed and recognized target, and at the same time, record the coordinate point symmetric to the current robotic arm coordinate point with respect to the photo-taking point. This symmetric coordinate point is used as the robotic arm coordinate point. Finally, calculate the affine transformation matrix between the nine groups of points through the affine transformation function of opencv to achieve the automated calibration of Eye-in-hand. Among them, the hand-eye calibration matrix is calculated by the following formula:

[0070]

[0071] Where: x---column of the pixel coordinate system; y---row of the pixel coordinate system; θ represents the rotation angle of the coordinate system; tx--translation in the x direction of the coordinate system; ty---translation in the y direction of the coordinate system; sx---scaling in the x direction of the coordinate system; sy---scaling in the y direction of the coordinate system; x1---x coordinate of the robotic arm coordinate system; y1---y coordinate of the robotic arm coordinate system;

[0072] Thus, calculate the rotation angle θ, the offset (tx, ty), and the scaling ratio (sx, sy); by selecting nine points

[0073] Under the condition of ensuring that the target does not exceed the field of view range, randomly select nine different coordinate points to eliminate singular values and improve the calculation accuracy at the same time.

[0074] Then, calibrate the relationship between the center of the flange and the center of the gripper;

[0075] After calibrating the relationship between the camera and the center of the robotic arm flange, first, through the target detection algorithm, calculate the image coordinates of the target object at this time. Multiply them by the affine transformation matrix in formula (3) to obtain the corresponding coordinates of the center of the robotic arm end flange corresponding to the image coordinates at this time and record them as coordinates (x1, y1). Manually move the robotic arm to the position where the fixture holds the target object and record the coordinates (x2, y2). Calculate the corresponding offset (x1 - x2, y1 - y2) between the two coordinate points, and update the offset to the original affine transformation matrix in formula (3), then the calibration relationship between the camera coordinate system and the gripper center can be established.

[0076] In formula (1), the calibration relationship is divided into rotation and translation. Under the condition of ensuring that the photographing pose remains unchanged, that is, the relative rotation relationship of the coordinate system does not change, that is, only the value of the new translation matrix needs to be modified on the original affine transformation matrix in formula (3), that is

[0077]

[0078] where x3---the x coordinate of the robotic arm tool coordinate system;

[0079] y3---the y coordinate of the robotic arm tool coordinate system.

[0080] The unknowns in T1 are the offsets of the X and Y axes respectively. Only one set of corresponding values is needed to obtain the two unknowns, greatly simplifying the entire calibration process.

[0081] (1) Calibration method for different photographing points:

[0082] Manually control the robotic arm. Under the condition of ensuring that the pose of the robotic arm end remains unchanged, move the camera to photographing point 2 on the same horizontal plane. Place the communication module in the camera's field of view and obtain the center pixel coordinates of the module through the target detection module. Calculate the coordinate point in the robotic arm base coordinate system after conversion through the original affine transformation matrix and record it as coordinate point 1. Manually control the robotic arm. Under the condition of ensuring that the pose of the robotic arm end remains unchanged, move the robotic arm to the center point position of the current module and record the current robotic arm coordinate point and record it as coordinate point 2. Fifth step, calculate the XY-direction offset between coordinate point 1 and 2. Finally, accumulate the offset to the original affine transformation matrix to obtain a new affine transformation matrix, that is, the affine transformation matrix of the current photographing point 2.

[0083] Camera 2: Eye-to-hand See the Eye-to-hand calibration flow chart.

[0084] Calibrate the hand-eye first;

[0085] First, after punctuating at the camera photographing point 1, control the camera to photograph from the lower part to the upper part at the camera photographing point 2 for calibration. Then, use the robotic arm to pick up the target object and move it nine times on the same plane directly above the camera (randomly move to nine positions under the condition that the target object does not exceed the field of view). During the movement, locate the camera 2 to photograph and identify the pixel coordinates of the bottom center of the target object. At the same time, the control system reads the robotic arm coordinates of the robotic arm at the nine positions. At this time, it is equivalent to recording nine groups of corresponding points between coordinate systems, and then the solution of the affine transformation matrix can be realized.

[0086] This method realizes the automatic calibration of Eye-to-hand, without manual participation, and simplifies the calibration process.

[0087] Re-calibration tool;

[0088] First, under the condition of ensuring the same pose, use the robotic arm to pick up the target object and reach the installation position and record the installation point 1, that is, the coordinate point 1. In the actual process, after the fixture picks up the target object, it reaches the photographing point of the camera photographing point 2, and records the photographing point coordinate 1P1(x1,y1) of the camera photographing point 2. Through target detection, the image coordinates of the target object at this time are obtained, and after being converted by the hand-eye calibration affine transformation matrix into the robotic arm coordinate system coordinates 2P2(x2,y2). By calculating the offset between coordinate 2 and coordinate 1, and applying this offset to the installation point 1A1(x3,y3), the final installation position A2(x4,y4) can be obtained.

[0089]

[0090] x1---The robotic arm x coordinate of the camera photographing point during calibration; y1---The robotic arm y coordinate of the camera photographing point during calibration; x2---The robotic arm x coordinate after affine transformation of the current image coordinates; y2---The robotic arm y coordinate after affine transformation of the current image coordinates; x3---The robotic arm x coordinate of the gripper installation point during calibration; y3---The robotic arm y coordinate of the gripper installation point during calibration; x4---The robotic arm x coordinate of the final installation point; y4---The robotic arm y coordinate of the final installation point;

[0091] In the robot control algorithm, mainly through the C# language to re-encapsulate the SDK interface provided by the Aubo robot, mainly including the robotic arm point-to-point axis movement interface, the robotic arm current position acquisition interface and the robotic arm current state acquisition interface. According to the on-site environment, write a path control program to ensure accurate installation of the module without collision.

[0092] The hardware system applying the above algorithm: mainly includes a base 2, on which there is a four-axis robotic arm 1, on the robotic arm 1 there is a robotic arm head 7, and on the robotic arm head 7 there are 2 Hikvision 500w grayscale cameras and a set of independently designed fixtures.

[0093] As shown Figure 1 in the figure: First, the fixture adopts the idea of simultaneous grasping by two claws, and can grasp two modules at the same time, effectively improving the working rhythm. Second, the clamping claws adopt a telescopic cylinder 6 and a clamping cylinder 5 in cooperation to achieve a flexible clamping method that can automatically extend and clamp. Among them, the specially curved fixture 4 is used to cooperate with the clamping cylinder 5 to finally clamp the module 3. The telescopic cylinder 6 can also effectively detect whether the module is installed in place by adjusting the position of the in-place sensor. That is, after the installation is completed, if the installation is normal, the telescopic cylinder is in a fully extended state, and at this time, the in-place sensor should be in a signal state. If the installation is not in place, that is, the module pins are not correctly aligned, that is, at this time, the telescopic cylinder is in an incompletely extended state, that is, the in-place sensor signal is in a non-signal state. This design can achieve a closed loop for the entire installation process. A camera 8 is mounted on the fixture, and the shooting direction is from top to bottom, which is used for the function of module grasping, recognition and positioning. Another one is mounted on the module transportation platform, and the shooting direction is from bottom to top, which is used for the function of module installation recognition and positioning, and the position is arbitrary.

[0094] The present invention also includes an electric meter positioning system, which is mainly used to position the electric meter at a fixed installation position, and mainly includes a manipulator for sucking and flipping the cover, a positioning mechanism, and a stopping mechanism. Among them, the manipulator for sucking and flipping the cover can be flipped by a rotary cylinder, and can realize the sucking and flipping of two electric meters at the same time. The positioning mechanism and the stopping mechanism mainly include cylinders to limit the plane offset and play a positioning role.

[0095] Robot: In the table box transfer and positioning mechanism of this system, a welding robot installation frame is set up, which is specifically used for the grasping and placing of the electric energy meter between the table box and the carrier module assembly mechanism.

[0096] Camera: This system includes two cameras. The vision camera A is installed in the carrier module gripper mechanism through a camera mounting plate, which is used for the positioning of the electric energy meter and simultaneously reads the bar code information of the carrier module. The vision camera B is installed in the carrier module assembly mechanism, which is responsible for the visual positioning of the electric energy meter carrier module, binds the electric energy meter information with the carrier module information after loading the electric energy meter.

[0097] Gripper and meter: This system includes an electric energy meter gripper mechanism and a carrier module gripper mechanism. The electric energy meter gripper mechanism is used for the grasping and placing of the electric energy meter in the table box, and the carrier module gripper is used for the grasping and loading of the electric energy meter carrier module.

[0098] Module tooling: This module is responsible for the bearing and positioning of the communication module, enabling the robot to present a better pose and higher accuracy during grasping.

[0099] The design of the present invention is reasonable, low in cost, strong and durable, safe and reliable, simple to operate, time-saving and labor-saving, cost-saving, compact in structure and convenient to use.

[0100] The present invention is fully described for a clearer disclosure, and the prior art will not be enumerated one by one.

[0101] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that they can still modify the technical solutions described in the foregoing embodiments, or perform equivalent replacements on some of the technical features; it is obvious for those skilled in the art to combine multiple technical solutions of the present invention. These modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. An automatic installation method for an intelligent electricity meter module based on machine vision, characterized in that: The method performs the following steps. First, it executes a visual recognition algorithm for module recognition. Then, it executes a hand-eye calibration algorithm. Next, it executes a robot control algorithm. By executing the visual recognition algorithm, the grasping and installation of the visual robotic arm are realized. The communication module of the State Grid smart meter is detected and recognized through the visual recognition algorithm. The color of the communication module is white and it is placed in a black tray. For the visual recognition algorithm, first, the grayscale image collected by the camera is binarized. Then, the binary image is dilated morphologically to eliminate black hole noise. The Canny edge detection algorithm is used, and the contours in the image are preliminarily screened through a double threshold of the edge length. Next, among the current contours, all closed contours are fitted with a quadrilateral algorithm. By judging the distance between the center points of the quadrilaterals, the distances less than the set threshold are deleted. Thus, the fitted rectangle of the communication module in the image and the center point of the rectangle are obtained. In the binarization, the collected grayscale image has a signal structure similar to a double peak, and the Otsu method is used for binarization. First, with the aid of σ 2 = Pf × (Mf - M) 2 + Pb × (Mb - M) 2 Equation (1); Among them, σ 2 – Variance of gray value; Pf – Number of target pixels; Mf - Average foreground gray value; M - Average gray value of the entire image; Pb - Number of background pixels; Mb - Average background gray value; The average pixel value of the default background pixels is set to 0, that is, Mb = 0, M = Mf / 2, and Pf + Pb is the total number of pixels in the current image. Therefore, formula (1) is simplified to: The hardware system of the robot includes a base (2). There is a robotic arm (1) on the base (2), a head (7) on the robotic arm (1), and a camera (8) and a fixture (4) are set on the head (7). The fixture (4) has jaws for grasping the module. The jaws are provided with a telescopic cylinder (6) and a clamping cylinder (5) for driving the telescopic and clamping of the jaws. The fixture (4) clamps the communication module (3) through the clamping cylinder (5). The telescopic cylinder (6) is equipped with a position sensor for detecting whether the communication module (3) is installed in place. After judging that the installation is completed, if the communication module (3) is installed in place, the telescopic cylinder (6) is in the fully extended state and the position sensor has a signal. If the communication module (3) is not installed in place, that is, the pins of the communication module (3) are not aligned, the telescopic cylinder (6) is in an incompletely extended state, that is, the position sensor has no signal. One of the cameras (8) is mounted on the fixture (4) and shoots from top to bottom for the grasping recognition and positioning of the communication module (3). The other camera (8) is mounted on the transportation platform of the communication module (3) and shoots from bottom to top for the installation recognition and positioning of the communication module (3). The hardware system also includes a meter positioning system for positioning the meter at a fixed installation position. The meter positioning system includes a manipulator for the suction and flipping of the meter cover. The manipulator for the suction and flipping of the meter cover is equipped with a rotary cylinder to realize the flipping of the meter. The manipulator has suction cups to realize the suction and flipping of two meters simultaneously.

2. The automatic installation method of the intelligent electricity meter module based on machine vision according to claim 1, characterized in that: The visual recognition algorithm is developed based on the OpenCV algorithm library. Among them, this automatic installation method is based on the Windows platform and is developed using the Visual Studio IDE based on the C# language.

3. The automatic installation method of the intelligent meter module based on machine vision according to claim 1, wherein: Among them, The quadrilateral fitting algorithm uses the Douglas-Peucker algorithm to fit polygons for closed contours, and polygon fitting is finally distinguished by the distance between the center points of the polygons and the variance of the offsets between the points on the polygons and the actual contour points, and is used after fitting and screening.

4. The automatic installation method of the intelligent electric meter module based on machine vision according to claim 1, characterized in that: In the execution of the hand-eye calibration algorithm, the hand-eye calibration algorithm, combined with the automated nine-point calibration algorithm for visual target detection, uses Eye-in-hand and Eye-to-hand; In Eye-in-hand, S1 abstractly associates hand-eye calibration with the transformation relationship between two coordinate systems, that is, the relationship between the image coordinate system and the robotic arm base coordinate system, and directly solves for the affine transformation matrix through nine groups of corresponding points.

Images