An intelligent wiring device

Abstract

This specification provides an intelligent wiring device, which relates to the field of power electronics technology. A camera group is set on the workbench of this device, and a global camera, an end camera, and a gripper are set on the robotic arm. The gripper is controlled by a control device to pick up a wire and place it in the middle of the camera group, and at the same time, images of each camera in the camera group are obtained; the wire markings on the wire surface are recognized according to the images of each camera; the markings of the connection terminals connected to both ends of the wire are determined according to the wiring diagram; path planning is carried out according to the markings of the connection terminals connected to both ends of the wire and the images taken by the global camera; the end of the robotic arm is controlled to move to the position of the target connection terminal, and the gripper is controlled to insert one end of the wire into the target connection hole. This solution can automatically pick up the wire, automatically determine the wiring position, automatically move to the wiring position and automatically perform the wiring operation, reducing the workload of manual labor, and at the same time avoiding problems such as high error rate, low efficiency, poor consistency, and great maintenance difficulty caused by manual operation.

Description

Technical Field

[0001] The present application relates to the field of power electronics technology, and in particular to an intelligent wiring device. Background Art

[0002] The wiring technology of electrical cabinets is an important part in the field of power engineering and automation control. It involves connecting various electrical components according to the design requirements to achieve the normal operation of the system.

[0003] Currently, almost all wiring technology in the industry is implemented by traditional manual methods. Manual wiring has the following disadvantages: high human error rate, low efficiency, safety risks, poor consistency, and difficult maintenance. Summary of the invention

[0004] This manual provides an intelligent wiring device to overcome the disadvantages of manual wiring.

[0005] In order to solve the above technical problems, the first aspect of the present specification provides an intelligent wiring device, including: a workbench, the workbench is used to place a board to be wired; a camera group is arranged on one side of the workbench, and the camera group is used to capture an image of the surface of the wire; a mechanical arm, the mechanical arm is provided with a global camera, and the global camera is used to capture an image of the surface of the workbench; a gripper and an end camera are arranged at the end of the mechanical arm, the gripper is used to grab the wire and insert the wire into a target wiring hole, and the end camera is used to capture an image of the end position of the gripper; a control device is used to control the gripper to grab the wire and place it in the middle of the camera group, and at the same time obtain the image of each camera in the camera group; recognize the wire mark on the surface of the wire according to the image of each camera in the camera group; obtain a wiring diagram, and determine the mark of the wiring terminal connected at both ends of the wire according to the wiring diagram; perform path planning according to the mark of the wiring terminal connected at both ends of the wire and the image captured by the global camera; control the end of the mechanical arm to move to the target wiring terminal position, and control the gripper to insert one end of the wire into the target wiring hole.

[0006] In some embodiments, path planning is performed based on the identification of the wiring terminals at both ends of the wire and the image of the global camera, including: determining the target area where the target wiring terminal is located based on the identification of the wiring terminals at both ends of the wire, the surface of the wiring board body is divided into areas according to the wiring range of the main module, and the surface of the wiring board body is provided with area division identification; determining the position of the target area in the image taken by the global camera, and further determining the spatial coordinates of the target area; determining the first travel path of the robot arm based on the current coordinates of the end of the robot arm and the spatial coordinates of the target area, and determining whether the target area is reached based on the image taken by the end camera; after the end of the robot arm reaches the target area, identifying the identification of the target wiring terminal from the target image taken by the end camera; determining the second travel path of the end of the robot arm based on the position of the end camera when taking the target image and the position of the target terminal identification in the target image, and determining whether the position of the target wiring terminal is reached based on the image taken by the end camera; after reaching the position of the target wiring terminal, identifying the target wiring hole closest to the target wiring terminal identification from the image taken by the camera end; and controlling the end of the robot arm to reach the position of the target wiring hole in the image.

[0007] In some embodiments, the camera group includes three cameras, the shooting directions of the three cameras are toward the same position, the three cameras are arranged along the same plane, and adjacent cameras are arranged at an angle of 120°.

[0008] In some embodiments, signals on the surface of the conductor are recognized based on images of each camera in a camera group, including: capturing an image of a portion of the conductor from an image captured by each camera; splicing the images of the conductor captured from the images captured by each camera to obtain a composite image; and recognizing a character sequence from the composite image, and using the character sequence as the line number of the conductor.

[0009] In some embodiments, before recognizing a character sequence from a composite image, a character recognition model for recognizing each character in a character sequence from a composite image is trained in the following manner: obtaining character images taken by each camera in a camera group when a target character is printed on a conductive line and an arrangement direction of the character is at a preset angle to an extension direction of the conductive line, and the conductive line is in a variety of postures relative to the camera group; determining a character composite image corresponding to each posture, a character spliced ​​image corresponding to one posture being a composite image formed by splicing character images taken by each camera in the camera group when the conductive line is in the posture relative to the camera group; using the character composite images corresponding to each posture as training samples to form a training sample set; and training the character recognition model using the training sample set.

[0010] In some embodiments, a wiring diagram is obtained, and identifications of wiring terminals at both ends of the wire are determined based on the wiring diagram, including: when a physical wiring diagram is placed on a workbench, obtaining an image taken by a global camera; and identifying identifications of wiring terminals connected to both ends of the target wire from the image taken by the global camera.

[0011] In some embodiments, a wiring diagram is obtained, and identifications of wiring terminals at both ends of the wire are determined based on the wiring diagram, including: obtaining an electronic wiring relationship file, wherein the wiring relationship file includes a correspondence between each wire identification and a wiring terminal identification; and determining identifications of the wiring terminals connected to both ends of the target wire identification based on the correspondence.

[0012] In some embodiments, the end of the robotic arm includes a first gripper and a second gripper, and the two grippers are spaced a predetermined distance apart.

[0013] In some embodiments, the end of the robotic arm further includes a wire winding mechanism, which is used to wind the wires clamped by the two grippers around the target pillar.

[0014] In some embodiments, the winding mechanism includes: a first pillar, a hook is provided at the end of the first pillar; a motor is provided on the robotic arm, the front end of the first pillar is provided on the rotating shaft of the motor, and the front end of the first pillar is retractable.

[0015] In some embodiments, the first support is a support for positioning one of the grippers.

[0016] In some embodiments, the motor has a first working state and a second working state, and the rotation directions of the first working state and the second working state are opposite; when the motor is in the first working state, the motor drives the first pillar to rotate in a first direction so that the wire is wound around the first pillar; when the motor is in the second working state, the motor drives the first pillar to rotate in a second direction so that the wire loops on the first pillar are spread out.

[0017] In some embodiments, the end of the robotic arm also includes two wire fixing components; one wire fixing component is located between the two gripping clamps, and the other wire fixing component is located on one side of the two gripping clamps; the wire fixing component includes a first wire wheel and a second wire wheel, the first wire wheel and the second wire wheel are arranged tangentially, and the edges of the first wire wheel and the second wire wheel are recessed to form a through hole matching the shape of the wire.

[0018] In some embodiments, a torque sensor is provided at the rotating shaft of the motor, and the torque sensor is used to detect the torque of the motor rotation; when the control device controls the motor to start rotating so that the wire is wound around the first pillar, it determines whether to control the motor to stop rotating according to the detection value of the torque sensor.

[0019] In some embodiments, a screwdriver is also provided at the end of the robotic arm; after the control device controls the gripper to insert one end of the wire into the target wiring hole, it also controls the screwdriver to insert into the screw hole of the target wiring hole and tighten the screw.

[0020] In some embodiments, a tension sensor is also provided on the second pillar used to set a gripping clamp; after the control device controls the gripping clamp to insert the target end of the wire into the target wiring hole, the control device controls the gripping clamp on the second pillar to pull the wire connected to the target end; when the tension value detected by the tension sensor reaches a preset tension value and the target end of the wire is still not pulled out, it is determined that the wiring of the target end of the wire is completed.

[0021] The intelligent wiring device provided in this specification is provided with a camera group on a workbench, a global camera, a terminal camera, and a gripper on a mechanical arm. The gripper is controlled by a control device to grip the wire and place it in the middle of the camera group, and the images of each camera in the camera group are obtained at the same time; the wire identification on the surface of the wire is recognized according to the images of each camera in the camera group; the wiring diagram is obtained, and the identification of the wiring terminals connected at both ends of the wire is determined according to the wiring diagram; the path is planned according to the identification of the wiring terminals connected at both ends of the wire and the image taken by the global camera; the end of the mechanical arm is controlled to move to the target wiring terminal position, and the gripper is controlled to insert one end of the wire into the target wiring hole. The intelligent wiring device can automatically grip the wire, automatically determine the wiring position, automatically move to the wiring position, and automatically perform the wiring operation. It has a high degree of automation, thereby reducing the workload of manual work, and avoiding the problems of high error rate, low efficiency, safety risks, poor consistency, and difficult maintenance caused by manual operation. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] In order to more clearly illustrate the implementation methods of the present application or the technical solutions in the prior art, the drawings required for use in the implementation methods or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some implementation methods recorded in this application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.

[0023] Figure 1 The overall structural diagram of the intelligent wiring device provided for this manual;

[0024] Figure 2 A schematic diagram showing that the arrangement direction of the character sequence is parallel to the extension direction of the wire;

[0025] Figure 3 A schematic diagram showing that the arrangement direction of the character sequence and the extension direction of the wire form a certain angle;

[0026] Figure 4 A schematic diagram of the camera set provided for this manual;

[0027] Figure 5 A schematic diagram of the end of a robotic arm;

[0028] Figure 6 Another schematic diagram of the end of the robotic arm;

[0029] Figure 7 A schematic diagram of a wire placement rack on a workbench;

[0030] Figure 8 Another schematic diagram of a wire placement rack on a workbench;

[0031] Fig. 9 A schematic diagram of a path planning;

[0032] Fig.10 This is a schematic diagram of the structure of the control device provided in this manual.

[0033] above Figures 1 to 8 Reference numerals:

[0034] 10-workbench, 11-board to be wired, 12-camera group, 121-first camera, 122-second camera, 123-third camera, 124-circular positioning reference part, 20-mechanical arm, 21-global camera, 22-gripping clamp, 221-first gripping clamp, 222-second gripping clamp, 23-end camera, 241-first wire fixing component, 242-second wire fixing component, 25-screwdriver, 30-control device, 40-wire placement rack, 41-wire support. DETAILED DESCRIPTION

[0035] In order to enable those skilled in the art to better understand the technical solutions in this application, the technical solutions in the embodiments of this application will be clearly and completely described below in conjunction with the drawings in the embodiments of this application. Obviously, the described embodiments are only part of the embodiments of this application, not all of the embodiments. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of this application.

[0036] In order to overcome the drawbacks of manual wiring, this specification provides an intelligent wiring device, which includes a workbench 10 , a mechanical arm 20 and a control device 30 .

[0037] like Figure 1 As shown, the workbench 10 is used to place a board body 11 to be wired. The board body 11 to be wired may include at least one of a PCB (Printed Circuit Board), a breadboard, and a perforated board, and wiring terminals arranged thereon.

[0038] Terminals can include plug-in terminals or screw-type terminals. Plug-in terminals consist of a plastic insulating shell that wraps around an internal metal clip. Screw-type terminals have structural components such as screws for tightening wires.

[0039] A camera group 12 is provided on one side of the workbench, and the camera group 12 is used to capture images of the surface of the wire. The camera group 12 can be 2 cameras, 3 cameras or more cameras, and these cameras can be arranged in a regular array, and the wire identification is obtained according to the images of each camera in the camera group.

[0040] A global camera 21 is provided on the robot arm 20. The global camera 21 is used to capture an image of the workbench surface. A gripper 22 and an end camera 23 are provided at the end of the robot arm 20. The gripper 22 is used to grab the wire and insert the wire into the target wiring hole. The end camera 23 is used to capture an image of the end position of the gripper.

[0041] The control device 30 is used to control the gripper 22 to clamp the wire and place it in the middle of the camera group 12, and at the same time obtain the image of each camera in the camera group 12; identify the wire identification on the surface of the wire according to the image of each camera in the camera group 12; obtain the wiring diagram, and determine the identification of the wiring terminals connected at both ends of the wire according to the wiring diagram; perform path planning according to the identification of the wiring terminals connected at both ends of the wire and the image taken by the global camera 21; control the end of the robot arm 20 to move to the target terminal position, and control the gripper to insert one end of the wire into the target wiring hole.

[0042] The conductor surface is provided with a conductor mark. The conductor mark is a series of character sequences consisting of at least one of letters, numbers, and symbols. The arrangement direction of the character sequence can be parallel to the extension direction of the conductor, such as Figure 2 As shown; it can also be at a certain angle with the extension direction of the wire (for example, the angle is greater than 10° or 15°), such as Figure 3 shown.

[0043] In the case where the arrangement direction of the character sequence is parallel to the extension direction of the wire, at least one camera in the camera group can capture the complete wire identification, and of course there may be cameras that cannot capture the wire identification at all or only capture part of the wire identification. In this case, the control device 30 can first identify whether the image captured by each camera in the camera group contains a complete wire identification (for example, whether the number of digits of the wire is a predetermined number of digits, whether each character in the wire identification is complete, etc.), and then perform character recognition on the camera image containing the complete wire identification to obtain the wire identification.

[0044] In the case where the arrangement direction of the character sequence forms a certain angle with the extension direction of the wire, and at least one camera in the camera group can capture the complete wire identification, the method described in the previous paragraph can also be used to identify the wire identification. In the case where the angle is large and no camera in the camera group can capture the complete wire identification, the wire identification can be identified in combination with the images captured by each camera group in the camera group. For example, the wire identification has a total of 4 characters. The first 2 characters of the wire identification are identified from the image captured by one camera, and the last 2 characters of the wire identification are identified from the image captured by another camera. The first 2 characters and the last 2 characters are concatenated to obtain a complete wire identification with 4 characters.

[0045] A character recognition model for recognizing characters from an image is trained in the following manner: character images taken by a camera at various postures relative to a conductive line when each character is printed on the conductive line and the character arrangement direction is at a preset angle to the extension direction of the conductive line are obtained as training samples to form a training sample set; the character recognition model is trained using the training sample set.

[0046] In some embodiments, the signal of the surface of the wire can be recognized according to the image of each camera in the camera group through the following S11 to S13.

[0047] S11: Extracting images of the wire portion from the images captured by each camera.

[0048] The shape of the wire in the image is generally a long strip, which is a relatively regular shape and is relatively easy to intercept.

[0049] S12: stitching the wire images captured from the images taken by each camera to obtain a composite image.

[0050] The wire images captured from the images taken by each camera usually have the same inclination angle as the reference edges fixed on the camera image (such as the horizontal and vertical edges of the image). When stitching, the images of adjacent cameras can be partially overlapped until the edge lines of the wires overlap to obtain a composite image.

[0051] S13: Identify a character sequence from the composite image, and use the character sequence as the wire number of the wire.

[0052] Furthermore, before S13, a character recognition module for recognizing each character in a character sequence from a synthetic image may be trained in the following manner from S14 to S17.

[0053] S14: acquiring character images taken by each camera in the camera group when the target character is printed on the wire and the character arrangement direction is at a preset angle to the extension direction of the wire, and the wire is in various postures relative to the camera group.

[0054] The preset angle may be the angle between the actual guide mark and the extension direction of the wire.

[0055] S15: Determine the character composite images corresponding to each posture, wherein the character splicing image corresponding to one posture is a composite image formed by splicing the character images taken by each camera in the camera group when the wire presents the posture relative to the camera group.

[0056] S16: Using the character composite images corresponding to each posture as training samples to form a training sample set.

[0057] S17: Training the character recognition model using the training sample set.

[0058] Since the surface of the wire is in an arc state, the characters on the edge of the image taken by the camera may be distorted. By overlapping the images taken by adjacent cameras, the problem of image distortion can be overcome, and the problem that one camera cannot determine whether it has captured the wire mark can be overcome.

[0059] The above-mentioned character images are spliced ​​to obtain a composite image, which can be a plane image or a three-dimensional image. Three-dimensional splicing refers to three-dimensional modeling based on the images taken by three cameras, obtaining multiple arc surfaces of the wire surface, and splicing the multiple arc surfaces to obtain a three-dimensional image of the wire surface. Then, the wire mark is identified from the surface of the three-dimensional image.

[0060] In some embodiments, the camera group 12 includes three cameras, specifically a first camera 121, a second camera 122, and a third camera 123. The shooting directions of the three cameras face the same position, the three cameras are arranged along the same plane, and adjacent cameras are arranged at an angle of 120°. Figure 1 As shown, the camera assembly 12 can be fixed at one end of the bracket.

[0061] When the camera group captures an image of the wire surface, the gripper at the end of the robotic arm 20 can clamp the wire in a straight line and place the wire at the intersection of multiple camera shooting directions, and make the extension direction of the wire perpendicular to the plane where the multiple cameras are located. Figure 4 The thick straight line in the figure represents the wire, the “×” on the straight line represents the clamping position of the clamp, and the dotted line represents the extending direction of the wire.

[0062] In some embodiments, in order to facilitate the robot arm 20 to clamp the wire to the above position and posture more accurately, a circular positioning reference piece 124 can be set between the multiple cameras of the camera group.

[0063] In some embodiments, the camera group can be in standby mode and take images periodically. The control device identifies whether there are wires in the images taken by each camera in the camera group. If there are wires in the images taken by the cameras, the wire markers on the surface of the wires are identified and the identified wire markers are obtained. When the wire markers are identified, the control device controls the end of the robot arm 20 to hold the wire and move in a direction away from the camera group so as to move along the planned path.

[0064] In some embodiments, the intersection of the shooting directions of the cameras in the camera group may also be located (for example, Figure 4 A trigger switch is set at position A in the gripper. When the gripper clamps the wire and places it at the target posture (i.e., the wire is placed at the intersection of the shooting directions of the cameras in the camera group, and the extension direction of the wire is perpendicular to the plane where the multiple cameras are located), the gripper triggers the trigger switch to give a shooting signal, and the control device controls each camera in the camera group to shoot in response to the shooting signal, and identifies the wire identifier based on the image captured by each camera. Before the control device identifies the wire identifier, the control device 30 can control the robot arm 20 to maintain the original posture to keep the posture of the wire unchanged, or the control device 30 can control the robot arm 20 to adjust the posture so that the placement posture of the wire is closer to the target posture.

[0065] In the case that the control device 30 still cannot obtain the correct wire identification after recognizing the images taken by each camera in the camera group, the control device 30 can control the robot arm 20 to adjust its posture so that the placement posture of the wire is closer to the target posture and control each camera in the camera group 12 to retake images after the robot arm 20 adjusts its posture, and then the control device 30 re-recognizes the images taken by each camera in the camera group 12. Repeat the above process until the control device 30 recognizes the wire identification, and then the control device 30 controls the end of the robot arm 20 to clamp the wire and move in the direction away from the camera group to move along the planned path.

[0066] The trigger switch can be a push-button trigger switch or an inductive trigger switch. For example, in the case of a push-button trigger switch, the gripper can touch the button. In the case of an inductive trigger switch, the trigger switch can emit a laser, and when the gripper clamps the wire to the target position, the end component of the robot arm 20 reflects the laser, and the control device 30 can determine whether to generate a trigger signal based on whether the reflected laser is received.

[0067] In some cases, when the robot arm 20 moves along the planned path, the end of the robot arm 20 will also reflect the laser. For this, the camera group can be set on the side of the workbench, with a distance difference from the area of ​​the board to be wired. In this way, the control device 30 can further determine whether to generate a trigger signal when receiving the reflected laser and combining whether the time difference between sending and receiving the laser is within a predetermined time length.

[0068] The global camera 21 is used to capture images of the workbench surface. A board to be connected may be placed on the workbench surface, so that the global camera 21 can capture images of the board surface to be connected, and the images of the board surface to be connected can be used for path planning.

[0069] Before the control device 30 moves along the planned path, the physical wiring diagram can be placed on the workbench surface so that the global camera 21 can capture the image of the physical wiring diagram. The control device 30 can identify the wiring relationship from the image of the physical wiring diagram. The wiring relationship includes the wire identification and the identification of the terminal at both ends of the wire corresponding to the wire identification. A wire identification corresponds to an identification of a terminal, and the wire represented by the wire identification is connected to the terminal. The wiring relationship can be used for path planning.

[0070] In some embodiments, the control device 30 may also directly obtain an electronic wiring diagram and identify the wiring relationship from the electronic wiring diagram, or the control device 30 may directly obtain an electronic wiring relationship file. For example, the above-mentioned obtaining of the wiring diagram and determining the identification of the wiring terminals at both ends of the wire according to the wiring diagram may be implemented in the following manner: obtaining an electronic wiring relationship file, wherein the wiring relationship file includes the correspondence between each wire identification and the wiring terminal identification; and determining the identification of the wiring terminals connected to both ends of the target wire identification according to the correspondence.

[0071] There are at least two grippers 22 disposed at the end of the mechanical arm 20: a first gripper and a second gripper. The first gripper and the second gripper are disposed at a predetermined distance, and the two grippers are used to clamp the two ends of the wire respectively.

[0072] Some wires are long, and the drooping of the long wires usually causes inconvenience in plugging in the wires, and the end of the mechanical arm is usually not designed to be wide enough to allow the two grippers to straighten the wires. Therefore, the intelligent wiring device provided in this specification is designed with a winding mechanism, which is used to wind the wires clamped by the two grippers around the target pillar.

[0073] In some embodiments, the winding mechanism includes a first pillar and a motor, a hook is provided at the end of the first pillar, the motor is provided on the mechanical arm, the front end of the first pillar is provided on the rotating shaft of the motor, and the front end of the first pillar can be extended. When the two grippers respectively clamp the two ends of the wire, the first pillar is extended to the wire position, the hook hooks the wire, and then the first pillar rotates to drive the wire to be wound around the first pillar.

[0074] The motor has a first working state and a second working state, and the first working state has a rotation direction opposite to that of the second working state. When the motor is in the first working state, the motor drives the first pillar to rotate in a first direction so that the wire is wound around the first pillar. When the motor is in the second working state, the motor drives the first pillar to rotate in a second direction so that the coils of the wire on the first pillar are spread out.

[0075] The first support column may be disposed between the two gripping clamps. The first support column may also be the first support column for disposing one of the gripping clamps. Figure 5 As shown, the first gripper 221 is arranged on the pillar B, the second gripper 222 is arranged on the pillar C, the pillar B is arranged on the rotating shaft of the motor D, and the pillar B can be used as the first pillar. The control device 30 can realize winding by controlling the motor D to rotate. For example, the control device 30 can first control the first gripper 221 to clamp the wire, and then control the motor D to rotate forward so that the wire is wound on the pillar B. When the wire needs to be released after one or both ends of the wire are plugged in, the control device 30 can control the motor D to rotate in reverse so that the wire wound on the pillar B is unwound.

[0076] In some embodiments, Figure 6 As shown, the end of the robot arm also includes two wire fixing components: a first wire fixing component 241 and a second wire fixing component 242, wherein one wire fixing component is arranged between the two gripping clamps, and the other wire fixing component is arranged on one side of the two gripping clamps.

[0077] The wire fixing assembly comprises a first wire wheel and a second wire wheel. The first wire wheel and the second wire wheel are arranged tangentially. The edges of the first wire wheel and the second wire wheel are recessed to form a through hole matching the shape of the wire.

[0078] Before the clamp grabs the wire, the two wire wheels move in opposite directions. After the clamp is aligned with the wire position, the two wire wheels move in opposite directions to clamp the wire wheels. During the winding and unwinding of the wire, the two wire wheels can not only tighten the wire to prevent it from falling, but also make it easier to move the wire. During the winding and unwinding of the wire, the rotation direction and rotation speed of the motor and wire wheels are matched to achieve precise control.

[0079] In some embodiments, a torque sensor is provided at the rotating shaft of the motor, and the torque sensor is used to detect the torque of the motor rotation. When the control device 30 controls the motor to start rotating so that the wire is wound around the first pillar, it determines whether to control the motor to stop rotating according to the detection value of the torque sensor.

[0080] When the wires are wound until the wires between the two grippers are straightened, the motor may be damaged if it rotates again, and the torque may increase sharply. In this regard, it can be determined whether the wires between the two grippers have been straightened based on the torque when the motor rotates, and the motor can be controlled to stop rotating if it has been straightened. For example, when the detection value of the torque sensor is greater than the preset torque, the control device 30 can control the motor to stop rotating when the detection value of the torque sensor is greater than the preset torque.

[0081] In some embodiments, Figure 5 and Figure 6 As shown, the end of the mechanical arm is also provided with a screwdriver 25. After the control device 30 controls the gripper to insert one end of the wire into the target wiring hole, it also controls the screwdriver 25 to insert into the screw hole of the target wiring hole and tighten the screw.

[0082] In some embodiments, a tension sensor is also provided on the second pillar for setting a gripper. After the control device 30 controls the gripper to insert the target end of the wire into the target wiring hole, the control device 30 can control the gripper on the second pillar to pull the wire connected to the target end. When the tension value detected by the tension sensor reaches a preset tension value and the target end of the wire is still not pulled out, it is determined that the target end of the wire is wired.

[0083] In some embodiments, the control device 30 can obtain images from the global camera 21 and / or the terminal camera 23, and detect whether the target end of the wire is pulled out by recognizing the images taken by the camera. Specifically, the control device 30 can obtain an image after controlling the gripper to pull the wire connected to the target end, and recognize the target end of the wire from the image. If the target end is recognized, it means that the target end of the wire is pulled out; if the target end is not recognized, it means that the target end of the wire is well connected.

[0084] The control device 30 can also use the images at each first moment and the images at the second moment as a set of comparison images, input the set of comparison images into a pre-trained recognition model, and identify whether the target end of the wire is pulled out through the recognition model. The images at the first moment refer to the images at each moment after the control device 30 starts to control the gripper to pull the wire connected to the target end, and the images at the second moment refer to the images at a moment after the control device 30 controls the gripper to insert the target end of the wire into the target wiring hole and before the control device 30 starts to control the gripper to pull the wire connected to the target end. The images at the first moment and the images at the second moment can both be taken by the global camera 21, or can both be taken by the end camera 23. When taking the images at the first moment and the images at the second moment, the position of the camera can be stationary or slightly changed.

[0085] In some embodiments, the control device 30 can determine whether the target end of the wire is pulled out according to the change in the detection value of the tension sensor. Before being pulled out, the detection value of the tension sensor gradually increases, and at the moment of being pulled out, the tension detection value becomes 0. That is to say, after the control device 30 controls the gripper to pull the wire connected to the target end, it starts to obtain the detection value of the tension sensor. If the detection value of the tension sensor suddenly becomes 0 before reaching the preset tension value, it means that the target end of the wire is pulled out; if the detection value of the tension sensor does not decrease before reaching the preset tension value, it means that the target end is well connected. After the detection value of the tension sensor reaches the preset tension value, the control device 30 can control the gripper to stop pulling the wire.

[0086] The wire can be manually picked up and placed in position with two grippers.

[0087] In some embodiments, a wire rack 40 may be provided on the workbench 10. The wire rack 40 includes at least two wire supports 41. Each wire support 41 includes a wire clamp for clamping a wire. The wires to be plugged may be pre-placed on the wire rack 40 manually or mechanically.

[0088] like Figure 7 As shown, there can be only one wire placement rack 40 on the workbench, so the wires need to be placed on the wire placement rack 40 one by one manually or mechanically, and a wire needs to be put down manually or mechanically after a wire on the wire placement rack 40 is clamped away.

[0089] like Figure 8As shown, there may be multiple wire racks 40 on the workbench, and multiple wire racks 40 may be placed in a predetermined area, and the predetermined area may be, for example, a wire placement box. Then, a batch of wires may be pre-placed on the wire racks 40 manually or mechanically. The control device 30 controls the manipulator 20 to move above the predetermined area, and controls the global camera to capture an image of the predetermined area, identifies each wire rack 40 with wires placed thereon from the image captured by the global camera, and determines the target wire that currently needs to be clamped, performs path planning according to the position of the target wire in the image captured by the global camera, and controls the manipulator 20 to move to the position of the target wire along the planned path and clamp the target wire.

[0090] In some embodiments, path planning based on the identification of the connection terminals connected to both ends of the wire and the image of the global camera may include the following S21 to S27, such as Fig. 9 shown.

[0091] S21: determining a target area where a target terminal is located according to identifications of the terminal blocks at both ends of the wire, wherein the surface of the wiring board is divided into areas according to the wiring range of the main modules, and area division identifications are provided on the surface of the wiring board.

[0092] The surface of the board to be connected can be represented by a rectangular frame, area blocks marked with different colors, etc. to indicate the wiring range of the main module, and the identification of each divided area is set on the surface of the board to be connected.

[0093] The target terminal is one of the two terminals to which the two ends of the wire are connected.

[0094] In some embodiments, the control device 30 may obtain the correspondence between each terminal identifier and the region identifier in advance, so that the identifier of the region where the target terminal is located may be directly determined according to the identifier of the target terminal and the correspondence.

[0095] In some embodiments, the identification of each terminal block may be associated with the regional identification of the area where the terminal block is located. For example, the first or last digits of the identification of the terminal block are the regional identification of the area where the terminal block is located. Then, the identification of the area where the target terminal block is located may be directly determined based on the identification of the target terminal block and the associated relationship.

[0096] By setting an association between the identification of each terminal and the regional identification of the area where it is located, the control device 30 can determine the identification of the area where the target terminal is located based only on the correspondence between the wire identification and the terminal identification, thereby reducing the processing process of the control device 30 and lowering the requirements of the intelligent wiring equipment for wiring-related files.

[0097] S22: Determine the position of the target area in the image captured by the global camera, and further determine the spatial coordinates of the target area.

[0098] The spatial coordinates of the target area determined based on the image captured by the global camera can be an approximate position and do not need to be very accurate. For example, S22 can estimate the direction and approximate distance to the target area based on the image captured by the global camera, and determine the spatial coordinates of the target area accordingly.

[0099] S23: Determine a first travel path of the robotic arm according to the current coordinates of the end of the robotic arm and the spatial coordinates of the target area, and determine whether the target area is reached according to the image taken by the end camera.

[0100] Whether the target area has been reached can be determined based on whether the target area logo or the specific area color of the target area is recognized in the image captured by the terminal camera.

[0101] The first travel path refers to the path that the end of the robot arm travels from the current position to the target area. In other words, the first travel path is used to guide the end of the robot arm to move from the current position to the approximate position of the target area.

[0102] S24: After the end of the robot arm reaches the target area, the identification of the target terminal is identified from the target image taken by the end camera.

[0103] S25: Determine the second travel path of the end of the robot arm according to the position of the end camera when capturing the target image and the position of the target terminal mark in the target image, and determine whether the position of the target terminal is reached according to the image captured by the end camera.

[0104] The second travel path refers to the path that the end of the robot arm takes to move to the target terminal in a more refined manner after reaching the approximate position of the target area.

[0105] The shooting range of the terminal camera is limited, so the terminal camera may not be able to shoot the target terminal before reaching the target area. Usually, the terminal camera can shoot the target terminal after reaching the target area.

[0106] Normally, the range captured by the global camera is wider and the image contains more content. The image processing workload required to determine whether the target terminal has been reached based on the image captured by the global camera is large. At this time, switching to use the image captured by the end camera to determine whether the target terminal has been reached can reduce the workload of image processing.

[0107] In addition, the control device 30 controls the gripper to clamp and insert the wire through the image taken by the terminal camera. At this time, it switches from determining whether the target terminal has been reached based on the image taken by the terminal camera to determining based on the global camera, which can facilitate subsequent operations.

[0108] S26: After reaching the position of the target wiring terminal, identify the target wiring hole closest to the target wiring terminal mark from the image captured by the end of the camera.

[0109] The control device 30 cannot determine where the target wiring hole is based only on the image captured by the terminal camera. The target wiring terminal mark is usually set next to the target wiring hole and is closest to the target wiring hole. Based on this principle, the control device 30 can identify the circle (i.e., the hole) closest to the target wiring terminal mark and use it as the target wiring hole.

[0110] S27: Control the end of the robot arm to reach the position of the target wiring hole in the image.

[0111] The spatial coordinates of the target wiring hole can be determined according to the position of the target wiring hole in the image taken by the end camera, and then the end of the robot arm is controlled to move to the spatial coordinate position.

[0112] The control device may be composed of multiple processors, such as a graphics processor, a controller of the robot body, and controllers of various components at the end of the robot.

[0113] The embodiment of the present invention further provides a control device, which may be an industrial computer. Fig.10 As shown, the control device may include a processor 1001 and a memory 1002, wherein the processor 1001 and the memory 1002 may be connected via a bus or other means. Fig.10 The example of connecting through bus is taken in the following.

[0114] The processor 1001 may be a central processing unit (CPU). The processor 1001 may also be other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, or a combination of the above chips.

[0115] The memory 1002 is a non-transitory computer-readable storage medium that can be used to store non-transitory software programs, non-transitory computer executable programs and modules. The processor 1001 executes various functional applications and data processing of the processor by running the non-transitory software programs, instructions and modules stored in the memory 1002, that is, to realize the intelligent wiring device in the above method embodiment.

[0116] The memory 1002 may include a program storage area and a data storage area, wherein the program storage area may store an operating system, an application required by at least one function; the data storage area may store data created by the processor 1001, etc. In addition, the memory 1002 may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one disk storage device, a flash memory device, or other non-volatile solid-state storage device. In some embodiments, the memory 1002 may optionally include a memory remotely arranged relative to the processor 1001, and these remote memories may be connected to the processor 1001 via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0117] The one or more modules are stored in the memory 1002 and when executed by the processor 1001, the above-mentioned intelligent wiring device is realized.

[0118] The specific details of the above-mentioned control device can be understood by referring to the corresponding descriptions and effects in the method embodiment, and will not be repeated here.

[0119] The present specification also provides a computer storage medium, wherein the computer storage medium stores computer program instructions, and when the computer program instructions are executed, the method steps in the above-mentioned intelligent wiring device are implemented.

[0120] The present specification also provides a computer program product, which includes a computer program. When the computer program is executed by a processor, the method steps in the above-mentioned intelligent wiring device are implemented.

[0121] Those skilled in the art can understand that the implementation of all or part of the processes in the above-mentioned embodiments can be completed by instructing the relevant hardware through a computer program, and the program can be stored in a computer-readable storage medium, and when the program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, the storage medium can be a disk, an optical disk, a read-only memory (ROM), a random access memory (RAM), a flash memory, a hard disk drive (HDD) or a solid-state drive (SSD), etc.; the storage medium can also include a combination of the above-mentioned types of memory.

[0122] Each embodiment in this specification is described in a progressive manner, and the same or similar parts between the various embodiments can be referenced to each other, and each embodiment focuses on the differences from other embodiments.

[0123] The systems, devices, modules or units described in the above embodiments may be implemented by computer chips or entities, or by products with certain functions.

[0124] For the convenience of description, the above device is described in various units according to their functions. Of course, when implementing the present application, the functions of each unit can be implemented in the same or multiple software and / or hardware.

[0125] It can be known from the description of the above implementation methods that those skilled in the art can clearly understand that the present application can be implemented by means of software plus a necessary general hardware platform. Based on such an understanding, the technical solution of the present application can be essentially or partly contributed to the prior art in the form of a software product, which can be stored in a storage medium such as ROM / RAM, a magnetic disk, an optical disk, etc., and includes several instructions for enabling a computer device (which can be a personal computer, a server, or a network device, etc.) to execute certain parts of the methods of each implementation method of the present application.

[0126] The present application may be described in the general context of computer-executable instructions executed by a computer, such as program modules. Generally, program modules include routines, programs, objects, components, data structures, etc. that perform specific tasks or implement specific abstract data types. The present application may also be practiced in distributed computing environments where tasks are performed by remote processing devices connected through a communication network. In a distributed computing environment, program modules may be located in local and remote computer storage media, including storage devices.

[0127] Although the present application has been described through embodiments, those skilled in the art will appreciate that there are many modifications and variations to the present application without departing from the spirit of the present application, and it is intended that the appended claims include these modifications and variations without departing from the spirit of the present application.

Claims

1. An intelligent wiring device, characterized in that: include: A workbench is provided on which a board to be wired is placed; a camera group is provided on one side of the workbench, and the camera group is used to capture an image of the surface of the wire; A robotic arm, wherein a global camera is provided on the robotic arm, and the global camera is used to capture an image of the workbench surface; a gripper and an end camera are provided at the end of the robotic arm, wherein the gripper is used to grab the wire and insert the wire into the target wiring hole, and the end camera is used to capture an image of the end position of the gripper; A control device, used to control the gripper to grip the wire and place it in the middle of the camera group, and simultaneously obtain images of each camera in the camera group; identify wire identifiers on the surface of the wire according to the images of each camera in the camera group; obtain a wiring diagram, and determine identifiers of the wiring terminals connected to both ends of the wire according to the wiring diagram; Path planning is performed according to the identification of the wiring terminals connected at both ends of the wire and the image captured by the global camera; the end of the robot arm is controlled to move to the target wiring terminal position, and the gripper is controlled to insert one end of the wire into the target wiring hole; The path planning is performed according to the identification of the wiring terminals connected at both ends of the wire and the image captured by the global camera, including: Determine the target area where the target terminal is located according to the identification of the terminal at both ends of the wire, the surface of the wiring board body is divided into areas according to the wiring range of the main module, and the surface of the wiring board body is provided with area division identification; Determine the position of the target area in the image captured by the global camera, and further determine the spatial coordinates of the target area; Determine a first travel path of the robotic arm according to the current coordinates of the end of the robotic arm and the spatial coordinates of the target area, and determine whether the target area is reached according to the image taken by the end camera; After the end of the robot arm reaches the target area, identifying the identification of the target terminal from the target image taken by the end camera; Determine a second travel path of the end of the robot arm according to the position of the end camera when capturing the target image and the position of the target terminal mark in the target image, and determine whether the position of the target terminal is reached according to the image captured by the end camera; After reaching the position of the target wiring terminal, the target wiring hole closest to the target wiring terminal mark is identified from the image captured by the camera end; Control the end of the robot arm to reach the position of the target wiring hole in the image.

2. The intelligent wiring device according to claim 1, characterized in that: The camera group includes three cameras, the shooting directions of the three cameras are toward the same position, the three cameras are arranged along the same plane, and adjacent cameras are arranged at an angle of 120°.

3. The intelligent wiring device according to claim 1, characterized in that: The signals on the surface of the conductor are recognized based on the images of each camera in the camera group, including: Extracting an image of the wire portion from the images captured by each camera; The wire images captured from the images taken by each camera are stitched together to obtain a composite image; A character sequence is recognized from the composite image and used as the wire number of the wire.

4. The intelligent wiring device according to claim 3, characterized in that: Before recognizing a character sequence from a synthetic image, a character recognition model for recognizing each character in the character sequence from the synthetic image is trained in the following manner: Acquire character images taken by each camera in the camera group when the target character is printed on the wire and the character arrangement direction is at a preset angle to the extension direction of the wire, and the wire is in various postures relative to the camera group; Determine a character composite image corresponding to each posture, wherein the character splicing image corresponding to one posture is a composite image formed by splicing character images taken by each camera in the camera group when the wire presents the posture relative to the camera group; The character synthetic images corresponding to each posture are respectively used as training samples to form a training sample set; The character recognition model is trained using the training sample set.

5. The intelligent wiring device according to claim 1, characterized in that: Obtaining a wiring diagram, and determining identifications of wiring terminals at both ends of the wire according to the wiring diagram, including: When the physical wiring diagram is placed on the workbench, an image captured by the global camera is obtained; The identifications of the connection terminals connected to the two ends of the target wire are identified from the image captured by the global camera.

6. The intelligent wiring device according to claim 5, characterized in that: Obtaining a wiring diagram, and determining identifications of wiring terminals at both ends of the wire according to the wiring diagram, including: Obtaining an electronic version of a wiring relationship file, wherein the wiring relationship file includes a correspondence between each wire identifier and a wiring terminal identifier; The identifiers of the connection terminals connected to the two ends of the target wire identifier are determined according to the corresponding relationship.

7. The intelligent wiring device according to claim 1, characterized in that: The end of the mechanical arm comprises a first gripper and a second gripper, and the two grippers are arranged at a predetermined distance.

8. The intelligent wiring device according to claim 7, characterized in that: The end of the mechanical arm also includes a winding mechanism, which is used to wind the wires clamped by the two grippers around the target pillar.

9. The intelligent wiring device according to claim 8, characterized in that: The winding mechanism comprises: A first pillar, wherein a hook is disposed at an end of the first pillar; The motor is arranged on the mechanical arm, the front end of the first support column is arranged on the rotating shaft of the motor, and the front end of the first support column is retractable.

10. The intelligent wiring device according to claim 9, characterized in that: The first support column is a support column for setting one of the gripping clamps.

11. The intelligent wiring device according to claim 9, characterized in that: The motor has a first working state and a second working state, and the rotation directions of the first working state and the second working state are opposite; When the motor is in a first working state, the motor drives the first pillar to rotate along a first direction so that the wire is wound around the first pillar; When the motor is in the second working state, the motor drives the first pillar to rotate along the second direction so that the wire loops on the first pillar are spread out.

12. The intelligent wiring device according to claim 8, characterized in that: The end of the mechanical arm also includes two wire fixing components; one wire fixing component is located between the two gripping clamps, and the other wire fixing component is located on one side of the two gripping clamps; The wire fixing assembly comprises a first wire wheel and a second wire wheel. The first wire wheel and the second wire wheel are arranged tangentially. The edges of the first wire wheel and the second wire wheel are recessed to form a through hole matching the shape of the wire.

13. The intelligent wiring device according to claim 9, characterized in that: A torque sensor is provided at the rotating shaft of the motor, and the torque sensor is used to detect the torque of the motor rotation; The control device controls the motor to start rotating so that the wire is wound around the first support column, and determines whether to control the motor to stop rotating according to the detection value of the torque sensor.

14. The intelligent wiring device according to claim 1, characterized in that: A screwdriver is also provided at the end of the mechanical arm; After the control device controls the gripper to insert one end of the wire into the target wiring hole, the control device also controls the screwdriver to insert into the screw hole of the target wiring hole and tighten the screw.

15. The intelligent wiring device according to claim 1, characterized in that: A tension sensor is also provided on the second pillar used to set a gripper; After the control device controls the grasping clip to insert the target end of the wire into the target wiring hole, the control device controls the grasping clip on the second support to pull the wire connected to the target end; When the tension value detected by the tension sensor reaches a preset tension value and the target end of the wire is still not pulled out, it is determined that the connection of the target end of the wire is completed.

Images