A method and system for locking a screw hole

Abstract

The embodiment of the application discloses a screw hole locking method and system, the method comprises: obtaining the screw hole area position feature by screw hole recognition according to the global image obtained by the global camera; screening the screw hole center coordinate area in the plate edge within the screw hole area position feature; controlling the pressing mechanism to press and fix the plate at the optimal pressing position; dividing the working area of each station based on the screw hole center coordinate area; planning the locking path of all screw holes in the working area of each station respectively; guiding the electric screwdriver head carrying the binocular stereo camera to move based on the planned screw hole locking path, positioning the screw hole center according to the depth image profile of each screw hole, and obtaining the image center coordinate of each screw hole; converting the spatial coordinates according to the image center coordinate of each screw hole, so that the electric screwdriver head locks according to the screw hole spatial center coordinate. The screw hole locking is efficiently and accurately performed.

Description

Technical Field

[0001] This application relates to the field of mechanical assembly technology, specifically to a screw hole fastening method and system. Background Technology

[0002] Screw fastening is a crucial step in the furniture manufacturing assembly process. With the rapid urbanization in China in recent years, the continuous expansion of commercial housing transactions, and the upgrading of residents' consumption structure and the shift in consumption concepts towards fashion, environmental protection, and multifunctionality, the market share and scale of the customized furniture industry are growing larger and larger. The manual screw fastening process for furniture, especially customized furniture, can no longer meet the needs of industrial transformation and upgrading due to the high labor intensity, low efficiency, and high cost of workers. Summary of the Invention

[0003] Therefore, embodiments of this application provide a screw hole fastening method and system that can perform screw hole fastening efficiently and accurately.

[0004] To achieve the above objectives, the embodiments of this application provide the following technical solutions:

[0005] According to a first aspect of the embodiments of this application, a screw hole fastening method is provided, the method comprising:

[0006] Screw hole identification is performed based on global images acquired by the global camera to obtain the location features of the screw hole region;

[0007] Perform edge detection of the plate material within the location features of the screw hole area, and filter the screw hole center coordinate region within the edge of the plate material.

[0008] A network diagram is formed based on the coordinate region of the screw hole center, and the optimal pressing position is determined based on the network diagram and the preset network segmentation algorithm. The pressing mechanism is then controlled to press and fix the plate at the optimal pressing position.

[0009] The working areas of each station are divided based on the coordinate region of the screw hole center.

[0010] Plan the fastening path for all screw holes within the working area of ​​each workstation;

[0011] The screwdriver bits carrying binocular stereo cameras are guided to move based on the planned screw hole fastening paths, and local images of each screw hole in their respective moving areas are acquired.

[0012] Contour extraction is performed on the local image of each screw hole to obtain the depth image contour map of each screw hole;

[0013] The center of each screw hole is located based on the depth image contour map, and the coordinates of the center of each screw hole are obtained.

[0014] The spatial coordinates are transformed based on the center coordinates of the image for each screw hole, so that the electric screwdriver bit can be locked according to the spatial center coordinates of the screw hole.

[0015] Optionally, before performing screw hole recognition based on the global image acquired by the global camera, the method further includes:

[0016] The minimum bounding matrix of the board edge is obtained from the global image acquired by the global camera;

[0017] Detect the number of contours of the minimum outer matrix of the plate edge and determine whether the plates are connected;

[0018] If no boards are connected, proceed to the next step; if boards are connected, send an alarm message.

[0019] Optionally, dividing the working areas of each of the multiple stations based on the coordinate region of the screw hole center includes:

[0020] Transform the screw holes into nodes in the network, the paths between the centers of the screw holes into edges in the network, and the distance between the centers of the screw holes into the weights of the edges, thus obtaining a set of nodes;

[0021] Based on the weight matrix, sort the edges in the nodes from largest to smallest weight; select the node with the largest weight as the seed node for the working range of each workstation.

[0022] Find the node with the minimum weight among the unassigned nodes and assign it to the work area of ​​the seed node, marking it as an assigned node, until all nodes have been marked as assigned nodes, thus obtaining the work areas of each of the multiple workstations after division.

[0023] Optionally, the step of locating the center of each screw hole based on the depth image contour map of each screw hole to obtain the image center coordinates of each screw hole includes:

[0024] A discrete parameter space is established between the maximum and minimum values ​​of the center coordinates and radius of the contour map of the depth image of each screw hole. The transformation is performed on each point in the candidate region of the target detection in the parameter space, and the center coordinates and radius values ​​of all points are calculated respectively to obtain the number of screw holes and the image center coordinates of each screw hole.

[0025] Optionally, the step of performing plate edge detection within the location features of the screw hole area and filtering the screw hole center coordinate region within the plate edge includes:

[0026] The positional features of the screw hole area are convolved along the X-axis and Y-axis using X-axis and Y-axis templates respectively. The center of the template is aligned with a pixel in the image, and the points around that pixel are multiplied with the coefficients on the template. Then, gradient calculation is performed to obtain the coordinates of the edge area of ​​the board material.

[0027] Draw horizontal rays from the coordinates of each screw hole, and filter the screw holes within the edge of the board according to the number of times the screw hole rays intersect with the edge of the board to form the coordinate region of the screw hole center.

[0028] Optionally, the step of identifying screw holes in the global image to obtain the location features of the screw hole region includes:

[0029] Perform connected component analysis on the global image to extract the chain code table and linear list of each region;

[0030] Calculate the length, perimeter, area, and center of the region boundary profile using the chain code table and the linear table, respectively.

[0031] Based on the length, perimeter, area, and center of the region's boundary contour, the screw hole region is located to obtain its positional characteristics.

[0032] Optionally, the step of planning the fastening path for all screw holes within the working area of ​​each workstation includes:

[0033] The ant colony algorithm is used to calculate the coordinates of the center of each screw hole in the working area of ​​the electric screwdriver bit and return the shortest distance to the original location, which is used as the screw hole locking path for each screw hole.

[0034] Optionally, forming a network diagram based on the image center coordinates of each screw hole includes:

[0035] The coordinates of the center of the image for each screw hole are used as nodes in the network graph;

[0036] The lines connecting the screw holes are used as edges in the network graph;

[0037] The distance between the screw holes is used as the weight of the edge in the network graph.

[0038] Optionally, determining the optimal compaction position based on the network graph and a preset network segmentation algorithm includes:

[0039] The network graph is divided into multiple sub-networks;

[0040] The optimal compaction position is determined based on the median line between the boundary points of adjacent sub-networks.

[0041] Optionally, the network graph is divided into multiple sub-networks, including:

[0042] Determine the number of subnetworks to be divided;

[0043] The edges in the network graph are sorted from largest to smallest weight; the two nodes corresponding to the edges with larger weights at the top are taken as seed nodes, and the number of seed nodes is equal to the number of subnetworks.

[0044] If the node with the smallest weight when connected to the seed node is found among the remaining unassigned nodes, and the number of nodes in the sub-network corresponding to the seed node is less than a preset threshold, then the node is assigned to the sub-network corresponding to the seed node and marked as an assigned node.

[0045] Repeat the node assignment process until all remaining nodes are marked as assigned nodes, resulting in multiple sub-networks.

[0046] Optionally, determining the optimal compaction position based on the median line between adjacent sub-network boundary points includes:

[0047] Sort the nodes in each subnetwork in ascending order according to their X-axis coordinates. The first node after sorting is taken as the left critical node of the subnetwork, and the last node after sorting is taken as the right critical node of the subnetwork.

[0048] The optimal clamping position of each clamping bar is obtained by using the median line between the right critical node of the subnetwork with the smaller x-axis coordinate and the left critical node of the adjacent right subnetwork.

[0049] According to a second aspect of the embodiments of this application, a screw hole fastening system is provided, the system comprising:

[0050] The screw hole coarse positioning module is used to identify screw holes based on the global image acquired by the global camera and obtain the positional features of the screw hole area;

[0051] The edge detection module is used to perform edge detection of the board material within the location features of the screw hole area and to filter the screw hole center coordinate area within the edge of the board material.

[0052] The clamping module is used to form a network diagram based on the coordinate region of the screw hole center, determine the optimal clamping position based on the network diagram and a preset network segmentation algorithm, and control the clamping mechanism to clamp and fix the plate at the optimal clamping position.

[0053] A multi-station partitioning module is used to divide the working areas of each of the multiple stations based on the coordinate region of the screw hole center.

[0054] The fastening path planning module is used to plan the fastening path for all screw holes in the working area of ​​each workstation.

[0055] The local image acquisition module is used to guide the electric screwdriver bit carrying the binocular stereo camera to move based on the planned screw hole fastening path, and to acquire local images of each screw hole in its respective moving area;

[0056] The contour extraction module is used to extract the contour of the local image of each screw hole to obtain the depth image contour map of each screw hole;

[0057] The screw hole center positioning module is used to locate the center of each screw hole based on the depth image contour map, and obtain the image center coordinates of each screw hole;

[0058] The locking module is used to transform the spatial coordinates according to the center coordinates of the image of each screw hole, so that the electric screwdriver bit can lock according to the spatial center coordinates of the screw hole.

[0059] According to a third aspect of the present application, an electronic device is provided, comprising: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the method described in the first aspect above.

[0060] According to a fourth aspect of the embodiments of this application, a computer-readable storage medium is provided having computer-readable instructions stored thereon, which can be executed by a processor to implement the method described in the first aspect above.

[0061] In summary, this application provides a screw hole fastening method and system. The method involves identifying screw holes based on global images acquired by a global camera to obtain the location features of the screw hole region; detecting the edge of the sheet metal within the location features to filter the screw hole center coordinate regions within the sheet metal edge; forming a network diagram based on the screw hole center coordinate regions; determining the optimal clamping position based on the network diagram and a preset network segmentation algorithm; controlling the clamping mechanism to clamp and fix the sheet metal at the optimal clamping position; dividing the work areas of multiple stations based on the screw hole center coordinate regions; planning fastening paths for all screw holes within the work area of ​​each station; guiding the movement of an electric screwdriver bit equipped with a binocular stereo camera based on the planned screw hole fastening paths, and acquiring local images of each screw hole within its respective movement area; extracting the contour of each screw hole's local image to obtain a depth image contour map of each screw hole; locating the screw hole center based on the depth image contour map of each screw hole to obtain the image center coordinates of each screw hole; and performing spatial coordinate transformation based on the image center coordinates of each screw hole so that the electric screwdriver bit fastens the screw hole according to its spatial center coordinates. Perform screw hole fastening efficiently and accurately. Attached Figure Description

[0062] To more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are merely exemplary, and those skilled in the art can derive other embodiments based on the provided drawings without creative effort.

[0063] The structures, proportions, sizes, etc. illustrated in this specification are only for the purpose of assisting those skilled in the art in understanding and reading the content disclosed herein, and are not intended to limit the conditions under which the present invention can be implemented. Therefore, they have no substantial technical significance. Any modifications to the structure, changes in the proportions, or adjustments to the size, without affecting the effects and objectives that the present invention can produce, should still fall within the scope of the technical content disclosed in the present invention.

[0064] Figure 1 This is a schematic flowchart of a screw hole fastening method provided in an embodiment of this application;

[0065] Figure 2 The device structure diagram provided in the embodiments of this application;

[0066] Figure 3 Detailed view of the gantry frame carrying the locking wire mechanism provided in the embodiments of this application;

[0067] Figure 4 Detailed design diagram of the locking wire mechanism carrying a partial binocular stereo camera provided in an embodiment of this application;

[0068] Figure 5 A flowchart of the screw hole coarse identification and positioning method provided in the embodiments of this application;

[0069] Figure 6 This is a basic flowchart of the ant colony algorithm provided in the embodiments of this application;

[0070] Figure 7 A block diagram of a screw hole fastening system provided in this application embodiment;

[0071] Figure 8 This illustration shows a structural schematic diagram of an electronic device provided in an embodiment of this application;

[0072] Figure 9 A schematic diagram of a computer-readable storage medium provided in an embodiment of this application is shown. Detailed Implementation

[0073] The following specific embodiments illustrate the implementation of the present invention. Those skilled in the art can easily understand other advantages and effects of the present invention from the content disclosed in this specification. Obviously, the described embodiments are only some, not all, of the embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0074] The characteristics of custom furniture cabinet panels are: firstly, a large range of sizes (400mm-1800mm in length, 400mm-700mm in width, and 12mm-15mm in thickness); and secondly, a variety of panel shapes and hole distributions. In other words, due to differences in rooms, the hole positions of each order are different, which places very high demands on the flexibility of the screw fastening device.

[0075] Currently, most screw fastening devices (methods) require workers to pre-position each screw hole and then use a screw fastening machine to fasten the screws at the set positions. This method does not utilize machine vision and cannot directly identify the screw hole positions. Therefore, it is only suitable for relatively standardized workpieces and requires customized fixtures to fix the workpieces. When using this method to process workpieces with many variations, such as customized furniture cabinet panels, it is necessary to repeatedly set the hole positions on the equipment, repeatedly adjust, and even remake the fixtures, which is inefficient and costly.

[0076] Based on the above background, this application proposes a method for precise screw hole positioning based on multi-vision sensing and binocular vision technology to address the need for automatic screw fastening on customized cabinet panels. Multi-vision sensing technology efficiently identifies the distribution of workpiece holes, plans the motion path of a local camera, and accurately identifies the hole positions. Binocular vision technology excludes already screwed circular workpieces, accurately locates the screw holes requiring fastening, and performs screw fastening efficiently and accurately.

[0077] Figure 1 This application illustrates a threaded hole fastening method flow according to an embodiment of the present application. The method includes:

[0078] Step 101: Identify screw holes based on the global image acquired by the global camera to obtain the location features of the screw hole region;

[0079] Step 102: Perform edge detection of the board material within the location features of the screw hole area, and filter the screw hole center coordinate area within the edge of the board material;

[0080] Step 103: Form a network diagram based on the coordinate region of the screw hole center, determine the optimal pressing position based on the network diagram and the preset network segmentation algorithm, and control the pressing mechanism to press and fix the plate at the optimal pressing position;

[0081] Step 104: Divide the working areas of each of the multiple workstations based on the coordinate region of the screw hole center;

[0082] Step 105: Plan the fastening path for all screw holes within the working area of ​​each workstation;

[0083] Step 106: Guide the electric screwdriver bit carrying the stereo camera to move based on the planned screw hole fastening path, and acquire local images of each screw hole in its respective moving area;

[0084] Step 107: Extract the contour of the local image of each screw hole to obtain the depth image contour map of each screw hole;

[0085] Step 108: Locate the center of each screw hole based on the depth image contour map, and obtain the image center coordinates of each screw hole;

[0086] Step 109: Perform spatial coordinate transformation based on the image center coordinates of each screw hole so that the electric screwdriver bit can be locked according to the spatial center coordinates of the screw hole.

[0087] In one possible implementation, before performing screw hole identification based on the global image acquired by the global camera in step 101, the method further includes:

[0088] The minimum bounding matrix of the board edge is obtained from the global image acquired by the global camera; the number of contours in the minimum bounding matrix of the board edge is detected, and it is determined whether the boards are connected; if no boards are connected, proceed to the next step; if boards are connected, an alarm message is sent.

[0089] In one possible implementation, step 101, which involves identifying screw holes based on the global image acquired by the global camera to obtain the location features of the screw hole region, includes:

[0090] Connectivity analysis is performed on the global image to extract the chain code table and linear list of each region; the length, perimeter, area and center of the region boundary contour are calculated according to the chain code table and linear list respectively; the screw hole region is located according to the length, perimeter, area and center of the region boundary contour to obtain the positional features of the screw hole region.

[0091] In one possible implementation, step 102, which involves performing plate edge detection within the location features of the screw hole region and filtering the screw hole center coordinate region within the plate edge, includes:

[0092] The positional features of the screw hole region are convolved along the X-axis and Y-axis using X-axis and Y-axis templates respectively. The center of the template is aligned with a certain pixel in the image, and the points around that pixel are multiplied with the coefficients on the template. Gradient calculation is then performed to obtain the coordinates of the edge region of the board. Horizontal rays are drawn from the coordinates of each screw hole, and the screw holes within the edge of the board are selected based on the number of intersections between the screw hole rays and the edge of the board to form the coordinate region of the screw hole center.

[0093] In one possible implementation, step 103, forming a network diagram based on the image center coordinates of each screw hole, includes:

[0094] The coordinates of the center of each screw hole in the image are used as nodes in the network graph; the lines connecting the screw holes are used as edges in the network graph; and the distance between the screw holes is used as the weight of the edges in the network graph.

[0095] In one possible implementation, step 103, determining the optimal pressing position based on the network graph and a preset network segmentation algorithm, includes:

[0096] The network graph is divided into multiple sub-networks; the optimal compaction position is determined based on the median line between the boundary points of adjacent sub-networks.

[0097] In one possible implementation, the network graph is divided into multiple sub-networks, including:

[0098] Determine the number of sub-networks to be divided; sort the edges in the network graph from largest to smallest weight; select the two nodes corresponding to the edges with the largest weights at the top as seed nodes, and the number of seed nodes is equal to the number of sub-networks; find the node with the smallest weight when connected to a seed node among the remaining unassigned nodes, and if the number of nodes in the sub-network corresponding to the seed node is less than a preset threshold, then assign the node to the sub-network corresponding to the seed node and mark the node as an assigned node; repeat the node assignment step until all remaining nodes are marked as assigned nodes, thus obtaining multiple sub-networks.

[0099] In one possible implementation, determining the optimal compaction position based on the median line between adjacent sub-network boundary points includes:

[0100] The nodes within each subnetwork are sorted in ascending order according to their X-axis coordinates. The first node after sorting is taken as the left critical node of the subnetwork, and the last node after sorting is taken as the right critical node of the subnetwork. The optimal pressing position of each pressing bar is obtained based on the median line between the right critical node of the subnetwork with the smaller X-axis coordinate and the left critical node of the adjacent right subnetwork.

[0101] In one possible implementation, step 104, dividing the working areas of each of the multiple stations based on the coordinate region of the screw hole center, includes:

[0102] The screw holes are transformed into nodes in a network, the paths between the centers of the screw holes are the edges in the network, and the distance between the centers of the screw holes is the weight of the edge, resulting in a node set. According to the weight matrix, the edges in the nodes are sorted in descending order of weight. The node with the largest weight is selected as the seed node for the working area of ​​each workstation. Among the unassigned nodes, the node with the smallest weight that is generated with the seed node is found, and the node with the smallest weight is assigned to the working area of ​​the seed node and marked as an assigned node. This process is repeated until all nodes have been marked as assigned nodes, resulting in the working areas of each of the multiple workstations after division.

[0103] In one possible implementation, step 105, which involves planning the fastening paths for all screw holes within the working area of ​​each workstation, includes:

[0104] The ant colony algorithm is used to calculate the coordinates of the center of each screw hole in the working area of ​​the electric screwdriver bit and return the shortest distance to the original location, which is used as the screw hole locking path for each screw hole.

[0105] In one possible implementation, step 108, locating the center of each screw hole based on the depth image contour map of each screw hole to obtain the image center coordinates of each screw hole, includes:

[0106] A discrete parameter space is established between the maximum and minimum values ​​of the center coordinates and radius of the contour map of the depth image of each screw hole. The transformation is performed on each point in the candidate region of the target detection in the parameter space, and the center coordinates and radius values ​​of all points are calculated respectively to obtain the number of screw holes and the image center coordinates of each screw hole.

[0107] The design structure of the global camera and the dual-station local binocular stereo camera is described below. Then, taking the two-station example, the following methods are introduced: edge detection method for irregular plates, coarse identification and positioning method for screw holes, dual-station working range division method based on network segmentation algorithm, dual-station locking path planning method based on global vision, and fine identification and positioning method for screw holes.

[0108] The screw hole positioning method provided in this application is applicable to the overall design structure of mechanical equipment, such as... Figure 2 As shown, the designed wire-locking machine includes a large platform with a controllable belt conveyor mechanism for transferring and supporting wooden boards. Two independent gantry frames, movable along the X-axis, are also arranged on the platform. Wire-locking mechanisms, movable along the Y-axis, are mounted on the gantry frames. These mechanisms are equipped with local binocular stereo cameras that move back and forth along the Y-axis with the wire-locking mechanisms (e.g., [image of camera]). Figure 3 As shown). A rotary locking mechanism that moves up and down along the Z-axis is arranged on the locking wire mechanism (e.g., Figure 4 (As shown).

[0109] A global camera is positioned above the platform, illuminating the entire platform and identifying the wooden planks on it. The global camera is mounted on a camera bracket, which is connected to the wire-locking machine frame. The camera bracket is adjustable in height along the Z-axis and in position along the X-axis.

[0110] First, the minimum bounding matrix of the board edge is obtained based on the global image acquired by the global camera. The number of contours in the minimum bounding matrix is ​​then detected to determine if boards are connected. If the number exceeds 1, boards are connected, and an alarm is triggered. After manual separation of the connected boards, the process of obtaining the minimum bounding matrix of the board edge from the global image is repeated. If the number of contours is 1, no boards are connected. If no boards are connected, the Blob algorithm is used to identify screw holes in the global image to obtain the location features of the screw hole region. The Sobel algorithm is then used to detect the board edge within the location features of the screw hole region, filtering out the screw hole center coordinate regions within the board edge.

[0111] Simultaneously, a network diagram is formed based on the coordinate region of the screw hole center, and the optimal pressing position is determined based on the network diagram and the preset network segmentation algorithm. The pressing mechanism is then controlled to press and fix the plate at the optimal pressing position.

[0112] The optimal division of the working area of ​​the dual-station camera system was calculated using a network segmentation algorithm. Then, an ant colony algorithm was used to plan the locking paths for all screw holes within the working areas of each local camera. Based on the optimal locking paths for their respective working areas, the two cameras were guided to complete the following steps in sequence: local image acquisition for each screw hole within their respective working areas; acquisition of the screw hole's local depth image contour map; precise location of the screw hole's center coordinates using the Hough algorithm; screw hole center coordinate transformation; and locking of the screw hole at the designated center coordinates using an electric screwdriver. The entire process concluded after both local stereo cameras had completed the acquisition, detection, and locking steps for each screw hole within their respective working areas.

[0113] The irregular plate edge detection method provided in the embodiments of this application will be described in detail below.

[0114] Edge detection of irregularly shaped panels was performed using the Sobel operator. As a discrete difference operator, the Sobel operator calculates the weighted difference between the gray values ​​of each pixel's four neighborhoods (top, bottom, left, and right) and finds the extreme value at the edge, thus detecting the edge. This operator not only produces good detection results but also effectively suppresses noise.

[0115] The original image is convolved using two templates along the X and Y axes, respectively, from top to bottom and from left to right. The center of the template is aligned with a pixel in the image, and the coefficients of the surrounding pixels are multiplied by the coefficients on the template. The X-axis and Y-axis templates are as follows:

[0116]

[0117] Then the gradient value at the center point O is G(x) and G(y) are as follows: G(x) = (x3 + 2x6 + x9) - (x1 + 2x4 + x7); G(y) = (x1 + 2x2 + x3) - (x7 + 2x8 + x9). x1 to x9 are the neighboring pixels of the center point O, distributed as follows:

[0118]

[0119] The following is a detailed description of the screw hole coarse identification and positioning method provided in the embodiments of this application.

[0120] The Blob algorithm is used for coarse identification and localization of screw holes. The Blob algorithm is used to detect images acquired by a global camera, obtaining information such as the position and orientation of circular screw holes and the topological relationships between them (there may be inclusion relationships between the circles to be identified, but no inclusion relationship exists in the context of this application embodiment). Based on this information, the target can be identified. Shape features are extracted accurately and quickly, and the target area is coarsely located. The Blob algorithm quickly and accurately extracts multiple connected regions through connected component analysis, thereby obtaining the chain code table and linear list of each region. Based on the chain code table and linear list, the boundary contour length and perimeter of the region, as well as feature parameters such as area and centroid, are calculated respectively. The specific process is as follows... Figure 5 As shown.

[0121] It should be noted that if filtering is for circular areas, four criteria must be met:

[0122] (1) Grayscale criteria. The marking circle used is black on the outside and white on the inside, so the center of gravity of the correct marking circle is white and the grayscale value is larger.

[0123] (2) Size Criteria. The perimeter of the region analyzed by Blob analysis is used as the criterion. The perimeter C of the target region A satisfies the following condition: C min <C<C max .

[0124] (3) Circularity criterion. Define the perimeter of the target region A as Len. a Area a If the perimeter and area satisfy Therefore, region A belongs to the candidate marker circle.

[0125] (4) Least squares fitting error. After the least squares fitting is completed, the points involved in the fitting are transformed to a new coordinate system with the center of the fitting circle as the origin, the major axis as the X-axis, and the minor axis as the Y-axis. The fitting error of each point is calculated as follows: Discard items with fitting errors exceeding the mean.

[0126] Precise screw hole positioning is achieved using multi-vision sensing and binocular vision technology. However, positioning alone is insufficient for screw fastening; workpiece clamping and fixation are also crucial prerequisites. Therefore, a network diagram is generated based on the positioned screw holes, and the optimal clamping position is determined using this network diagram and a pre-defined network segmentation algorithm.

[0127] First, based on the network diagram G constructed after the screw holes are located, the screw holes are treated as nodes in the network diagram, and the node positions are the coordinates of the screw hole centers. The lines connecting the centers of the screw holes are treated as edges in the network diagram, and the distances between the centers of the screw holes are treated as the weights of the edges in the network diagram. The node set in the diagram is C(G).

[0128] Furthermore, the constructed network graph is divided into multiple sub-networks.

[0129] Specifically, the number of sub-networks to be divided can be determined based on the number of robotic arms used. For example, two robotic arms can be used to divide two sub-networks, and four robotic arms can be used to divide four sub-networks. The number of sub-networks can be set according to the actual situation, and this application embodiment does not make specific limitations.

[0130] Sort the edges in the network graph according to their weights from largest to smallest to obtain a sorted array EdgeList. Select the two nodes corresponding to the edges with the largest weights at the top of the list as seed nodes. The number of seed nodes is equal to the number of subnetworks. Therefore, the number of edges with larger weights to select can be determined based on the fact that the number of seed nodes is equal to the number of subnetworks. Taking two subnetworks as an example, two seed nodes are needed. Only the edge with the largest weight needs to be selected. The edge with the largest weight is selected as EdgeList[0], represented as e(c1,c2), which means that the weights generated by nodes c1 and c2 are the largest. c1 and c2 are selected as the seed nodes of the two subnetworks.

[0131] If, among the remaining unassigned nodes, the node with the lowest weight when connected to the seed node is found, and the number of nodes in the subnetwork corresponding to the seed node is less than a preset threshold, then this node is assigned to the subnetwork corresponding to the seed node and marked as an assigned node. If, after assignment, the number of nodes in the subnetwork reaches the threshold, then the remaining unassigned nodes are assigned to the subnetworks of other seed nodes.

[0132] For example, find the node {c|c∈C(G)} among the remaining unassigned nodes that produces the minimum weight with seed nodes c1 and c2. Assume c... k Together with c2, we can generate the minimum weight, and c k The subnetwork Region_List[2] belonging to the seed node c2 is marked c k For nodes that have already been assigned; if assigned to a node... Then node c2 will no longer be used as a seed node.

[0133] Repeat the node assignment process until all remaining nodes are marked as assigned nodes, resulting in multiple sub-networks, Region_List.

[0134] Among them, Region_m i Let represent the number of nodes in the i-th subnetwork, n represent the number of subnetworks, n-1 represent the number of clamping bars, and m represent the number of screw holes. Where 1 ≤ i ≤ n. E represents the weight matrix of the network, i.e., the distance matrix between the screw holes. i×j This represents the weight of the edge between node i and node j, where 1 ≤ i ≤ m and 1 ≤ j ≤ m. It should be noted that E... i×j and E j×i Equivalent. Region_List represents the list of subnetworks after partitioning, and the list of nodes / holes in each subnetwork. Region_List[i] represents the i-th subnetwork.

[0135] Furthermore, the optimal compaction position is determined based on the median line of the boundary points between adjacent sub-networks. The median line is chosen to ensure that the pressure on both sides of the compaction position is balanced as much as possible, and to ensure that the distance between the compaction position and the boundary node is large enough.

[0136] First, the nodes within each sub-network are sorted in ascending order based on their X-axis coordinates. The first node in the sorted sequence is designated as the left critical node of that sub-network, and the last node is designated as the right critical node. The optimal clamping position for each clamping bar is then determined by the median line between the right critical node of the sub-network with the smaller X-axis coordinate and the left critical node of the adjacent right sub-network.

[0137] For example, the x-axis coordinate of the right critical node of the i-th sub-network is represented as: The x-axis coordinate of the left critical node is represented as The optimal pressing position of the i-th pressing bar is at the x-axis coordinate of The position is determined. This step is repeated until the (n-1)th pressing position has been calculated. The algorithm then ends.

[0138] After locating the screw holes in the global image of the board material, the screw holes are treated as nodes in a network, and the paths between the centers of the screw holes are considered as edges in the network. The path length represents the weight of the edge, thus forming a network graph of the screw holes. The problem of finding the optimal clamping position is abstracted into the problem of calculating the optimal position based on the network segmentation results. Specifically, the network is first divided into a specified number of sub-networks, and then the median line between the boundary points of adjacent sub-networks is found as the optimal clamping position. Then, the clamping mechanism is controlled to clamp and fix the board material at the optimal clamping position.

[0139] This method eliminates the need for specialized fixtures, significantly reducing processing costs and improving efficiency. It also eliminates the need for manual fixture adjustments before processing, further enhancing efficiency and equipment usability. Furthermore, combined with visual perception algorithms, it enables the clamping and fixing of materials of different sizes and shapes.

[0140] The following is a detailed description of the planning of the working range of a dual-station based on the network segmentation algorithm provided in the embodiments of this application.

[0141] The problem of dividing the working area of ​​a dual-station machine is abstracted as a network partitioning problem. Screw holes are considered nodes in the network, and the paths between the centers of the screw holes are considered edges. The path length is the weight of the network edge. The resulting sub-networks correspond to the working area of ​​a single-station machine. This network partitioning algorithm ensures that different sub-networks are far apart, while nodes within the same sub-network are close together. This guarantees a large distance between working areas of different stations and close proximity of screw holes within the same working area of ​​a station, ensuring that the screw holes assigned to each electric screwdriver bit are as close as possible to each other, thus improving locking efficiency.

[0142] The number of screw holes to be locked is m, and the number of workstations is n, meaning the number of nodes in the network is m, which ultimately needs to be divided into n work areas / subnetworks. First, n seed points are selected. Then, based on proximity, the remaining nodes are assigned to the nearest seed point until all nodes are assigned. To ensure a balanced number of screw holes allocated to each workstation, the number of nodes in each work area / subnetwork is limited to less than or equal to... The specific algorithm is described below:

[0143] The algorithm's relevant parameters are defined as follows:

[0144] (1) CircleList: A list of nodes in the network, i.e., a list of all screw holes, with a quantity of m;

[0145] (2) n: The final number of work areas / sub-networks, i.e. the number of workstations. In this project context, n = 2.

[0146] (3)Region_m i The number of nodes in the i-th work area / sub-network, i.e., the number of screw holes to be fastened at the i-th workstation. Where 1≤i≤n;

[0147] (4) E: The weight matrix of the network, i.e., the distance matrix between the screw holes, E i×j Let E represent the weight of the edge between node i and node j, where 1 ≤ i ≤ m, 1 ≤ j ≤ m, and E i×j and E j×i equivalence;

[0148] (5) Region_List: That is, the list of work areas / subnetworks after division. Each subnetwork is the list of nodes in the subnetwork, that is, the list of screw holes in each work area. Region_List[i] represents the i-th work area / subnetwork.

[0149] (6) w(e): The weight of edge e.

[0150] The algorithm steps are as follows:

[0151] Step 1: Construct a weighted graph G. The method is as follows: convert the screw holes into nodes in the network. The node position is the coordinate of the screw hole center, the line connecting the screw hole centers is considered an edge in the network, and the distance between the screw hole centers is the edge weight. The node set in the graph is C(G).

[0152] Step 2: Based on the weight matrix E, sort the edges in G in descending order of weight into an array EdgeList, which contains a total of m×(m-1) / 2 edges;

[0153] Step 3: EdgeList[0] with the largest weight, denoted as e(c1,c2), represents the largest weight generated by nodes c1 and c2. Select c1 and c2 as the seed nodes for the two working ranges;

[0154] Step 4: Find the node {c│c∈C(G)} among the remaining unowned nodes that has the minimum weight with seed nodes c1 and c2. k Together with c2, we can generate the minimum weight, and c k The region of seed node c2 is located in Region_List[2], marked c. k For nodes that have already been assigned; if assigned to a node... Delete seed c2;

[0155] Repeat step 4 until all remaining nodes have been marked as the home node, and obtain the partitioning result, that is, G is divided into n working regions / sub-networks; the algorithm ends and the partitioned working region result Region_List is obtained.

[0156] A network segmentation algorithm is used to intelligently divide the working area of ​​the dual-station system, ensuring that the distance between screw holes within the working area of ​​each station is close, shortening the movement path of the electric screwdriver bit, and reducing the fastening time. At the same time, it ensures that the workload of the screw holes to be fastened in the two stations is balanced, reducing the time difference between the two stations, enabling the two stations to work in parallel and improving fastening efficiency.

[0157] The following is a detailed description of the dual-station locking path planning method based on global vision provided in the embodiments of this application.

[0158] The problem of planning the locking path for screw holes can be abstracted as the Traveling Salesman Problem (TSP). The TSP was first proposed by Dantzig in 1959 and is defined as follows: A salesman needs to travel to n cities to sell goods. Starting from an arbitrary city, the distance between any two cities i and j is w. ij Finally, we need to return to the starting point and visit each city only once. The shortest path can be mathematically modeled as: y = min∑ i≠j w ij x ij The constraints are:

[0159]

[0160] Where x ij x ∈ 0, 1, i ≠ j. ij Let x be the decision variable, if ij If x is 1, it represents the path from city i to city j. ij A value of 0 indicates that the path was not selected, and |S| represents the number of elements in set S.

[0161] This application employs the ant colony algorithm to solve this problem. Proposed in the 1990s, the ant colony algorithm is a biomimetic algorithm derived by simulating the pathfinding behavior of ants in nature. Ants, during their movement, leave pheromones along their paths to transmit information. They can also sense these pheromones and use them to guide their movement. The shorter the path, the more ants traverse it, leaving behind more pheromones, and the greater the probability that subsequent ants will choose that path.

[0162] The basic algorithm flow is as follows Figure 6 As shown in the embodiment of this application, the ant colony algorithm is used to plan the paths for the screw holes within the control areas of two local stereo cameras. Then, based on the respective path plans, the local images of the screw holes from both local cameras are simultaneously acquired, accurately detected, located, and locked. The ant colony algorithm has the characteristics of distributed computation, positive feedback, and heuristic search, and is essentially a heuristic global optimization algorithm in evolutionary algorithms.

[0163] The optimal path planning problem for screw fastening is abstracted into the classic Traveling Salesman Problem (TSP). The ant colony algorithm is used to calculate the shortest distance for the screwdriver bit to traverse the center coordinates of each screw hole in the working area and return to the original position. The screw fastening is completed by following the shortest path, which reduces fastening time and improves fastening efficiency.

[0164] The following is a detailed description of the screw hole fine identification and positioning method provided in the embodiments of this application.

[0165] Based on the planned screw fastening path, the local binocular camera is guided to sequentially acquire local depth images in a targeted manner, and then the contour map of the depth image is obtained. The Hough transform algorithm is used on the contour map for fine identification and localization of the screw holes, and then the screw fastening is completed. The Hough transform is an algorithm based on grayscale images, using contours as features, while also incorporating gradient information. It identifies objects through a voting method, and its circle detection in hidden conditions has anti-interference capabilities, allowing for the acquisition of the target's position information. Therefore, the Hough transform algorithm is chosen as the method for fine identification and localization. Hough transform detection can be summarized as follows:

[0166] (1) Establish a discrete parameter space between reasonable maximum and minimum values ​​of the center coordinates x, y and the radius r;

[0167] (2) Create an accumulator A(x, y, r) and set it to 0;

[0168] (3) In the XY image space, perform Hough transform on each point in the candidate region of the target detection, calculate all (x, y, r) values, and add 1 to the corresponding accumulator, i.e. A(x, y, r) = A(x, y, r) + 1;

[0169] (4) After the accumulation ends, the number of points cocircling at (x, y, r) is A(x, y, r) after voting based on the value of A(x, y, r). (x, y, r) represents the position of the circle.

[0170] Using the above algorithm, the position of the center of the screw hole can be obtained, and then the spatial coordinates can be transformed to guide the electric screwdriver bit to fasten the screw.

[0171] By combining a global camera and a dual-station local binocular stereo camera, a multi-sensor approach is adopted. First, the global image acquired by the global camera is used for coarse localization of the screw holes. Then, a network segmentation algorithm is used to divide the working area of ​​the dual-station cameras. Next, an ant colony algorithm is used to plan the fastening path for the dual-station cameras. Based on the planned path, the local cameras are guided to acquire local images of the screw holes and accurately locate the screw holes to be fastened in the local images. During the fastening process, the dual-station binocular stereo cameras work in parallel, which accelerates the efficiency and accuracy of screw hole identification, localization, and fastening.

[0172] This application proposes a screw-locking design structure combining a global camera and a dual-station local binocular stereo camera, and a method for precise screw hole positioning based on multi-vision sensing and binocular vision technology, effectively addressing the shortcomings of existing technologies. The main technical problems solved include: based on global vision, solving the edge detection problem of irregular materials, the coarse identification and positioning problem of screw holes, the intelligent division of the working range of the dual-station system, and the planning of the optimal path for screw fastening in the dual-station system; based on dual-station binocular stereo vision, solving the identification problem of screw holes to be fastened in local depth images, and the precise positioning problem of the identified screw holes to be fastened. Using a binocular stereo camera to acquire local screw hole images can fully utilize depth information and avoid misjudging already fastened circular workpieces (nuts, washers, etc.).

[0173] In summary, this application provides a screw hole fastening method. The method involves identifying screw holes based on global images acquired by a global camera to obtain the location features of the screw hole region; performing edge detection on the sheet metal within the location features of the screw hole region to filter the screw hole center coordinate regions within the sheet metal edge; forming a network diagram based on the screw hole center coordinate regions; determining the optimal clamping position based on the network diagram and a preset network segmentation algorithm; controlling the clamping mechanism to clamp and fix the sheet metal at the optimal clamping position; dividing the work areas of multiple stations based on the screw hole center coordinate regions; planning fastening paths for all screw holes within the work area of ​​each station; guiding the movement of an electric screwdriver bit equipped with a binocular stereo camera based on the planned screw hole fastening paths, and acquiring local images of each screw hole within its respective movement area; extracting the contour of each screw hole's local image to obtain a depth image contour map of each screw hole; locating the screw hole center based on the depth image contour map of each screw hole to obtain the image center coordinates of each screw hole; and performing spatial coordinate transformation based on the image center coordinates of each screw hole so that the electric screwdriver bit fastens the screw hole according to its spatial center coordinates. Perform screw hole fastening efficiently and accurately.

[0174] Based on the same technical concept, embodiments of this application also provide a screw hole locking system, such as... Figure 7 As shown, the system includes:

[0175] The screw hole coarse positioning module 701 is used to identify screw holes based on the global image acquired by the global camera and obtain the position features of the screw hole area;

[0176] Edge detection module 702 is used to perform edge detection of the board within the location features of the screw hole area and filter the screw hole center coordinate area within the edge of the board.

[0177] The clamping module 703 is used to form a network diagram based on the coordinate region of the screw hole center, determine the optimal clamping position based on the network diagram and a preset network segmentation algorithm, and control the clamping mechanism to clamp and fix the plate at the optimal clamping position.

[0178] The multi-station division module 704 is used to divide the working areas of each of the multiple stations based on the coordinate region of the screw hole center.

[0179] The fastening path planning module 705 is used to plan the fastening path for all screw holes in the working area of ​​each workstation.

[0180] The local image acquisition module 706 is used to guide the electric screwdriver bit carrying the binocular stereo camera to move based on the planned screw hole fastening path, and to acquire local images of each screw hole in its respective moving area.

[0181] The contour extraction module 707 is used to extract the contour of the local image of each screw hole to obtain the depth image contour map of each screw hole;

[0182] The screw hole center positioning module 708 is used to locate the center of the screw hole according to the depth image contour map of each screw hole, and obtain the image center coordinates of each screw hole;

[0183] The locking module 709 is used to transform the spatial coordinates according to the image center coordinates of each screw hole, so that the electric screwdriver bit can lock according to the spatial center coordinates of the screw hole.

[0184] This application also provides an electronic device corresponding to the method provided in the foregoing embodiments. Please refer to... Figure 8 The diagram illustrates an electronic device provided by some embodiments of this application. The electronic device 20 may include: a processor 200, a memory 201, a bus 202, and a communication interface 203, wherein the processor 200, the communication interface 203, and the memory 201 are connected via the bus 202; the memory 201 stores a computer program that can run on the processor 200, and when the processor 200 runs the computer program, it executes the method provided by any of the foregoing embodiments of this application.

[0185] The memory 201 may include high-speed random access memory (RAM) or non-volatile memory, such as at least one disk storage device. Communication between this system network element and at least one other network element is achieved through at least one physical port 203 (which can be wired or wireless), such as the Internet, wide area network, local area network, or metropolitan area network.

[0186] Bus 202 can be an ISA bus, PCI bus, or EISA bus, etc. The bus can be divided into an address bus, a data bus, a control bus, etc. The memory 201 is used to store programs. After receiving an execution instruction, the processor 200 executes the program. The method disclosed in any of the foregoing embodiments of this application can be applied to the processor 200, or implemented by the processor 200.

[0187] The processor 200 may be an integrated circuit chip with signal processing capabilities. In implementation, each step of the above method can be completed by the integrated logic circuitry in the hardware of the processor 200 or by instructions in software form. The processor 200 may be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it may also be a digital signal processor (DSP), an application-specific integrated circuit (ASIC), an off-the-shelf programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components. It can implement or execute the methods, steps, and logic block diagrams disclosed in the embodiments of this application. The general-purpose processor may be a microprocessor or any conventional processor. The steps of the methods disclosed in the embodiments of this application can be directly embodied in the execution of a hardware decoding processor, or executed by a combination of hardware and software modules in the decoding processor. The software modules may reside in random access memory, flash memory, read-only memory, programmable read-only memory, electrically erasable programmable memory, registers, or other mature storage media in the art. The storage medium is located in memory 201. The processor 200 reads the information in memory 201 and, in conjunction with its hardware, completes the steps of the above method.

[0188] The electronic devices and methods provided in the embodiments of this application are based on the same inventive concept and have the same beneficial effects as the methods they employ, operate, or implement.

[0189] This application also provides a computer-readable storage medium corresponding to the method provided in the foregoing embodiments. Please refer to... Figure 9 The computer-readable storage medium shown is an optical disc 30, on which a computer program (i.e., a program product) is stored, which, when run by a processor, executes the methods provided in any of the foregoing embodiments.

[0190] It should be noted that examples of the computer-readable storage medium may also include, but are not limited to, phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other optical and magnetic storage media, which will not be elaborated here.

[0191] The computer-readable storage medium provided in the above embodiments of this application and the method provided in the embodiments of this application are based on the same inventive concept and have the same beneficial effects as the methods adopted, run or implemented by the applications stored therein.

[0192] It should be noted that:

[0193] The algorithms and displays provided herein are not inherently related to any particular computer, virtual device, or other equipment. Various general-purpose devices can also be used in conjunction with the teachings herein. The required structure for constructing such devices is apparent from the above description. Furthermore, this application is not directed to any particular programming language. It should be understood that the content of this application described herein can be implemented using various programming languages, and the above description of specific languages ​​is for the purpose of disclosing the best mode of implementation of this application.

[0194] Numerous specific details are set forth in the specification provided herein. However, it will be understood that embodiments of this application may be practiced without these specific details. In some instances, well-known methods, structures, and techniques have not been shown in detail so as not to obscure the understanding of this specification.

[0195] Similarly, it should be understood that, in order to simplify this application and aid in understanding one or more of the various inventive aspects, in the above description of exemplary embodiments of this application, various features of this application are sometimes grouped together in a single embodiment, figure, or description thereof. However, this method of disclosure should not be construed as reflecting an intention that the claimed application requires more features than are expressly recited in each claim. Rather, as reflected in the following claims, inventive aspects lie in fewer than all features of a single foregoing disclosed embodiment. Therefore, the claims following the detailed description are hereby expressly incorporated into that detailed description, wherein each claim itself is a separate embodiment of this application.

[0196] Those skilled in the art will understand that modules in the device of the embodiments can be adaptively changed and placed in one or more devices different from that embodiment. Modules, units, or components in the embodiments can be combined into a single module, unit, or component, and further, they can be divided into multiple sub-modules, sub-units, or sub-components. Except where at least some of such features and / or processes or units are mutually exclusive, any combination can be used to combine all features disclosed in this specification (including the accompanying claims, abstract, and drawings) and all processes or units of any method or device so disclosed. Unless expressly stated otherwise, each feature disclosed in this specification (including the accompanying claims, abstract, and drawings) may be replaced by an alternative feature that serves the same, equivalent, or similar purpose.

[0197] Furthermore, those skilled in the art will understand that although some embodiments described herein include certain features but not others included in other embodiments, combinations of features from different embodiments are intended to be within the scope of this application and form different embodiments. For example, in the following claims, any of the claimed embodiments can be used in any combination.

[0198] The various component embodiments of this application can be implemented in hardware, or as software modules running on one or more processors, or a combination thereof. Those skilled in the art will understand that microprocessors or digital signal processors (DSPs) can be used in practice to implement some or all of the functions of some or all of the components in the virtual machine creation apparatus according to embodiments of this application. This application can also be implemented as a device or apparatus program (e.g., a computer program and computer program product) for performing part or all of the methods described herein. Such an implementation of this application can be stored on a computer-readable medium, or can be in the form of one or more signals. Such signals can be downloaded from an Internet website, provided on a carrier signal, or provided in any other form.

[0199] It should be noted that the above embodiments are illustrative of this application and not restrictive, and that those skilled in the art can devise alternative embodiments without departing from the scope of the appended claims. In the claims, any reference signs placed between parentheses should not be construed as limiting the claims. The word "comprising" does not exclude the presence of elements or steps not listed in the claims. The word "a" or "an" preceding an element does not exclude the presence of a plurality of such elements. This application can be implemented by means of hardware comprising several different elements and by means of a suitably programmed computer. In the unit claims enumerating several means, several of these means may be embodied by the same item of hardware. The use of the words first, second, and third, etc., does not indicate any order. These words can be interpreted as names.

[0200] The above description is merely a preferred embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the technical scope disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A screw hole fastening method, characterized in that, The method includes: Screw hole identification is performed based on global images acquired by the global camera to obtain the location features of the screw hole region; Perform edge detection of the plate material within the location features of the screw hole area, and filter the screw hole center coordinate region within the edge of the plate material. A network diagram is formed based on the coordinate region of the screw hole center, and the optimal pressing position is determined based on the network diagram and the preset network segmentation algorithm. The pressing mechanism is then controlled to press and fix the plate at the optimal pressing position. The working areas of each station are divided based on the coordinate region of the screw hole center. Plan the fastening path for all screw holes within the working area of ​​each workstation; The screwdriver bits carrying binocular stereo cameras are guided to move based on the planned screw hole fastening paths, and local images of each screw hole in their respective moving areas are acquired. Contour extraction is performed on the local image of each screw hole to obtain the depth image contour map of each screw hole; The center of each screw hole is located based on the depth image contour map, and the coordinates of the center of each screw hole are obtained. The spatial coordinates are transformed based on the center coordinates of the image for each screw hole, so that the electric screwdriver bit can be locked according to the spatial center coordinates of the screw hole; The step of determining the optimal compaction position based on the network graph and a preset network segmentation algorithm includes: dividing the network graph into multiple sub-networks; determining the optimal compaction position based on the median line between the boundary points of adjacent sub-networks; dividing the network graph into multiple sub-networks includes: determining the number of sub-networks to be divided; sorting the edges in the network graph according to their weights from largest to smallest; taking the two nodes corresponding to the edge with the largest weight after sorting as seed nodes, with the number of seed nodes equal to the number of sub-networks; finding the node with the smallest weight when connected to a seed node among the remaining unassigned nodes, and if the number of nodes in the sub-network corresponding to the seed node is less than a preset threshold, then assigning the node to the sub-network corresponding to the seed node and marking the node as an assigned node; repeating the node assignment step until all remaining nodes are marked as assigned nodes, thus obtaining the divided multiple sub-networks.

2. The method as described in claim 1, characterized in that, Before performing screw hole identification based on the global image acquired by the global camera, the method further includes: The minimum bounding matrix of the board edge is obtained from the global image acquired by the global camera; Detect the number of contours of the minimum outer matrix of the plate edge and determine whether the plates are connected; If no boards are connected, proceed to the next step; if boards are connected, send an alarm message.

3. The method as described in claim 1, characterized in that, The division of the working areas of each of the multiple stations based on the coordinate region of the screw hole center includes: Transform the screw holes into nodes in the network, the paths between the centers of the screw holes into edges in the network, and the distance between the centers of the screw holes into the weights of the edges, thus obtaining a set of nodes; Based on the weight matrix, sort the edges in the nodes from largest to smallest weight; select the node with the largest weight as the seed node for the working range of each workstation. Find the node with the minimum weight among the unassigned nodes and assign it to the work area of ​​the seed node, marking it as an assigned node, until all nodes have been marked as assigned nodes, thus obtaining the work areas of each of the multiple workstations after division.

4. The method as described in claim 1, characterized in that, The step of locating the center of each screw hole based on the depth image contour map to obtain the image center coordinates of each screw hole includes: A discrete parameter space is established between the maximum and minimum values ​​of the center coordinates and radius of the contour map of the depth image of each screw hole. The transformation is performed on each point in the candidate region of the target detection in the parameter space, and the center coordinates and radius values ​​of all points are calculated respectively to obtain the number of screw holes and the image center coordinates of each screw hole.

5. The method as described in claim 1, characterized in that, The step of performing edge detection of the board material within the location features of the screw hole area and filtering the screw hole center coordinate region within the edge of the board material includes: The positional features of the screw hole area are convolved along the X-axis and Y-axis using X-axis and Y-axis templates respectively. The center of the template is aligned with a pixel in the image, and the points around that pixel are multiplied with the coefficients on the template. Then, gradient calculation is performed to obtain the coordinates of the edge area of ​​the board material. Draw horizontal rays from the coordinates of each screw hole, and filter the screw holes within the edge of the board according to the number of times the screw hole rays intersect with the edge of the board to form the coordinate region of the screw hole center.

6. The method as described in claim 1, characterized in that, The step of identifying screw holes based on global images acquired by a global camera to obtain the location features of the screw hole region includes: Perform connected component analysis on the global image to extract the chain code table and linear list of each region; Calculate the length, perimeter, area, and center of the region boundary profile using the chain code table and the linear table, respectively. Based on the length, perimeter, area, and center of the region's boundary contour, the screw hole region is located to obtain its positional characteristics.

7. The method as described in claim 1, characterized in that, The step of planning the fastening path for all screw holes within the working area of ​​each workstation includes: The ant colony algorithm is used to calculate the coordinates of the center of each screw hole in the working area of ​​the electric screwdriver bit and return the shortest distance to the original location, which is used as the screw hole locking path for each screw hole.

8. The method as described in claim 1, characterized in that, The step of forming a network diagram based on the center coordinates of the image of each screw hole includes: The coordinates of the center of the image for each screw hole are used as nodes in the network graph; The lines connecting the screw holes are used as edges in the network graph; The distance between the screw holes is used as the weight of the edge in the network graph.

9. The method as described in claim 1, characterized in that, Determining the optimal compaction position based on the median line between adjacent sub-network boundary points includes: Sort the nodes in each subnetwork in ascending order according to their X-axis coordinates. The first node after sorting is taken as the left critical node of the subnetwork, and the last node after sorting is taken as the right critical node of the subnetwork. Based on the median line between the right critical node of the previous sub-network and the left critical node of the next sub-network in the ascending order of the X-axis coordinates, the optimal clamping position of each clamping bar is obtained.

10. A screw hole locking system, characterized in that, The system includes: The screw hole coarse positioning module is used to identify screw holes based on the global image acquired by the global camera and obtain the positional features of the screw hole area; The edge detection module is used to perform edge detection of the board material within the location features of the screw hole area and to filter the screw hole center coordinate area within the edge of the board material. A clamping module is used to form a network diagram based on the coordinate region of the screw hole center, determine the optimal clamping position based on the network diagram and a preset network segmentation algorithm, and control the clamping mechanism to clamp and fix the plate at the optimal clamping position. Determining the optimal clamping position based on the network diagram and the preset network segmentation algorithm includes: dividing the network diagram into multiple sub-networks; determining the optimal clamping position based on the median line between the boundary points of adjacent sub-networks; dividing the network diagram into multiple sub-networks includes: determining the number of sub-networks to be divided; sorting the edges in the network diagram according to their weights from largest to smallest; taking the two nodes corresponding to the edge with the largest weight after sorting as seed nodes, with the number of seed nodes equal to the number of sub-networks; finding the node with the smallest weight when connected to a seed node among the remaining unassigned nodes, and if the number of nodes in the sub-network corresponding to the seed node is less than a preset threshold, then assigning the node to the sub-network corresponding to the seed node and marking it as an assigned node; repeating the node assignment step until all remaining nodes are marked as assigned nodes, resulting in multiple sub-networks. A multi-station partitioning module is used to divide the working areas of each of the multiple stations based on the coordinate region of the screw hole center. The fastening path planning module is used to plan the fastening path for all screw holes in the working area of ​​each workstation. The local image acquisition module is used to guide the electric screwdriver bit carrying the binocular stereo camera to move based on the planned screw hole fastening path, and to acquire local images of each screw hole in its respective moving area; The contour extraction module is used to extract the contour of the local image of each screw hole to obtain the depth image contour map of each screw hole; The screw hole center positioning module is used to locate the center of each screw hole based on the depth image contour map, and obtain the image center coordinates of each screw hole; The locking module is used to transform the spatial coordinates according to the center coordinates of the image of each screw hole, so that the electric screwdriver bit can lock according to the spatial center coordinates of the screw hole.

11. An electronic device, comprising: A memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that the processor executes the computer program to implement the method as claimed in any one of claims 1-9.

12. A computer-readable storage medium, characterized in that, It stores computer-readable instructions that can be executed by a processor to implement the method as described in any one of claims 1-9.

Images