An intelligent control method and system for a material taking device for precision hardware parts

Abstract

The present invention discloses an intelligent control method and system for a material taking device for precision hardware parts, which relates to the field of industrial automation control. The method includes: when the weight sensor in the preset material taking area shows an increase in weight, six-view images of the preset material taking area are collected by an image collector to obtain a material taking area image; the hardware parts in the material taking area image are identified to obtain a hardware parts model identification result and a hardware parts model recognition result; the material taking path is optimized according to the hardware parts model recognition result to obtain a recommended material taking path; and the material taking control is performed according to the hardware parts positioning identification area and the recommended material taking path. The technical problem of poor versatility in the control of the existing hardware parts material taking device, which restricts the improvement of production efficiency, is solved, and the technical effects of high versatility and improved production efficiency are achieved.

Description

Technical Field

[0001] This application relates to the field of industrial automation control, and particularly to an intelligent control method and system for a material taking device for precision hardware parts. Background Art

[0002] In metal processing and manufacturing, the material taking of hardware parts is a crucial link. In traditional hardware part material taking methods, operators need to rely on experience and the naked eye to judge the model of the hardware parts, and then manually adjust the parameters and configurations of the material taking device according to the identified hardware part model to ensure that the hardware parts can be accurately grasped. After the configuration is completed, the material taking device starts to perform the material taking operation, taking out the hardware parts from the preset position and placing them at the designated position. This method has poor versatility, and each hardware part model requires a separate configuration of the material taking device, resulting in a long production preparation time and restricting the improvement of production efficiency.

[0003] In the current related technologies, there are technical problems in the control of the material taking device for hardware parts, such as poor versatility and restricted improvement of production efficiency. Summary of the Invention

[0004] This application provides an intelligent control method and system for a material taking device for precision hardware parts. When the weight sensor in the preset material taking area shows an increase in weight, a six-view image collection of the preset material taking area is performed through an image collector to obtain an image of the material taking area. Image recognition technology is used to process the image to identify the model and model of the hardware parts, and determine the positioning identification area of the hardware parts. According to the identified hardware part model result, an optimization calculation of the material taking path is performed to obtain a recommended material taking path. According to the hardware part positioning identification area and the recommended material taking path, the material taking device is controlled to perform the material taking operation and other technical means, which can be applicable to multiple hardware part models, without the need to separately configure a material taking device for each model, achieving the technical effects of high versatility and improved production efficiency.

[0005] This application provides an intelligent control method for a material taking device for precision hardware parts, including:

[0006] When the weight sensor in the preset material taking area shows an increase in weight, a six-view image collection of the preset material taking area is performed through an image collector to obtain an image of the material taking area, where the weight sensor is mounted on the surface of the preset material taking area, and the preset material taking area is a transparent area; hardware part recognition is performed on the image of the material taking area to obtain a hardware part model recognition result and a hardware part model recognition result, where the hardware part model recognition result has a hardware part positioning identification area; material taking path optimization is performed according to the hardware part model recognition result to obtain a recommended material taking path; and material taking control is performed according to the hardware part positioning identification area and the recommended material taking path.

[0007] In a possible implementation, identify the hardware parts in the image of the picking area to obtain the identification results of the hardware part model and the identification results of the hardware part model, and perform the following processing:

[0008] Communicate with the weight sensor to obtain the increased weight information; obtain the first hardware part model to be verified that meets the increased weight information, where the first hardware part model to be verified has a plurality of picking area images with a plurality of placement postures; for the image of the picking area, traverse and compare the plurality of picking area images to be verified to obtain a plurality of image similarities; when any one of the plurality of image similarities is greater than or equal to the image similarity threshold, set the first hardware part model to be verified as the identification result of the hardware part model; at the same time, extract the identification result of the hardware part model according to the placement posture of the image similarity that is greater than or equal to the image similarity threshold.

[0009] In a possible implementation, for the image of the picking area, traverse and compare the plurality of picking area images to be verified to obtain a plurality of image similarities, and perform the following processing:

[0010] Extract the first picking area image to be verified from the plurality of picking area images to be verified, where the first picking area image to be verified has a front view to be verified, a rear view to be verified, a first side view to be verified, a second side view to be verified, a top view to be verified, and a bottom view to be verified; the image of the picking area includes a front view of the picking area, a rear view of the picking area, a first side view of the picking area, a second side view of the picking area, a top view of the picking area, and a bottom view of the picking area; compare the front view to be verified, the rear view to be verified, the first side view to be verified, the second side view to be verified, the top view to be verified, and the bottom view to be verified with the front view of the picking area, the rear view of the picking area, the first side view of the picking area, the second side view of the picking area, the top view of the picking area, and the bottom view of the picking area to obtain the first image similarity; add the first image similarity to the plurality of image similarities.

[0011] In a possible implementation, compare the front view to be verified, the rear view to be verified, the first side view to be verified, the second side view to be verified, the top view to be verified, and the bottom view to be verified with the front view of the picking area, the rear view of the picking area, the first side view of the picking area, the second side view of the picking area, the top view of the picking area, and the bottom view of the picking area to obtain the first image similarity, and perform the following processing:

[0012] Compare the to-be-verified front view and the front view of the material taking area to obtain the front view similarity; compare the to-be-verified rear view and the rear view of the material taking area to obtain the rear view similarity; compare the to-be-verified first side view and the first side view of the material taking area to obtain the first side view similarity; compare the to-be-verified second side view and the second side view of the material taking area to obtain the second side view similarity; compare the to-be-verified top view and the top view of the material taking area to obtain the top view similarity; compare the to-be-verified bottom view and the bottom view of the material taking area to obtain the bottom view similarity; take the minimum value of the front view similarity, the rear view similarity, the first side view similarity, the second side view similarity, the top view similarity and the bottom view similarity, and set it as the first image similarity.

[0013] In a possible implementation, compare the to-be-verified front view and the front view of the material taking area to obtain the front view similarity, and perform the following processing:

[0014] Obtain an image comparison network, where the image comparison network includes a first feature extraction channel, a second feature extraction channel and a feature comparison channel. The model structures and model parameters of the first feature extraction channel and the second feature extraction channel are exactly the same. The feature comparison channel is used to evaluate the proportion of feature descriptors with a deviation less than or equal to the deviation threshold; input the to-be-verified front view into the first feature extraction channel to obtain a first feature descriptor matrix; input the front view of the material taking area into the second feature extraction channel to obtain a second feature descriptor matrix; input the first feature descriptor matrix and the second feature descriptor matrix into the feature comparison channel to obtain the front view similarity.

[0015] In a possible implementation, optimize the material taking path according to the hardware part model recognition result to obtain a recommended material taking path, and perform the following processing:

[0016] According to the hardware part model recognition result, extract the passing space diameter threshold, where the passing space diameter threshold is the distance between the two farthest points distributed on the hardware part model recognition result; obtain the material taking target position and the robot arm movement constraint area; based on the passing space diameter threshold, perform the shortest path planning in the robot arm movement constraint area with the preset material taking area as the starting point and the material taking target position as the ending point to obtain the recommended material taking path.

[0017] In a possible implementation, with the preset material taking area as the starting point and the material taking target position as the ending point, perform the shortest path planning in the robot arm movement constraint area based on the passing space diameter threshold to obtain the recommended material taking path, and perform the following processing:

[0018] According to the passing space diameter threshold, the passing area where the movement constraint area of the robotic arm is less than or equal to the passing space diameter threshold is set as a virtual obstacle area; an entity obstacle area of the movement constraint area of the robotic arm is obtained; according to the virtual obstacle area and the entity obstacle area, with the preset material taking area as the starting point and the material taking target position as the ending point, the shortest path planning is performed in the movement constraint area of the robotic arm to obtain the recommended material taking path.

[0019] The present application further provides an intelligent control system for a material taking device for precision hardware parts, including:

[0020] A six-view acquisition module, which is used to perform six-view acquisition on a preset material taking area through an image collector to obtain a material taking area image when a weight sensor in the preset material taking area shows an increase in weight, wherein the weight sensor is mounted on the surface of the preset material taking area, and the preset material taking area is a transparent area; a hardware part recognition module, which is used to recognize hardware parts in the material taking area image to obtain a hardware part model recognition result and a hardware part model recognition result, wherein the hardware part model recognition result has a hardware part positioning identification area; a material taking path optimization module, which is used to optimize the material taking path according to the hardware part model recognition result to obtain a recommended material taking path; a material taking control module, which is used to perform material taking control according to the hardware part positioning identification area and the recommended material taking path.

[0021] It is intended to propose an intelligent control method and system for a material taking device for precision hardware parts through the present application. First, when a weight sensor in a preset material taking area shows an increase in weight, six-view acquisition is performed on the preset material taking area through an image collector to obtain a material taking area image, wherein the weight sensor is mounted on the surface of the preset material taking area, and the preset material taking area is a transparent area. Then, hardware parts are recognized in the material taking area image to obtain a hardware part model recognition result and a hardware part model recognition result, wherein the hardware part model recognition result has a hardware part positioning identification area. Then, the material taking path is optimized according to the hardware part model recognition result to obtain a recommended material taking path. Finally, material taking control is performed according to the hardware part positioning identification area and the recommended material taking path, achieving the technical effects of high versatility and improved production efficiency. Description of the Drawings

[0022] To more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings of the embodiments of the present invention will be briefly introduced below. Flowcharts are used in this application to illustrate the operations performed by the systems according to the embodiments of the present application. It should be understood that the operations in the front or below do not necessarily need to be precisely executed in sequence. On the contrary, as needed, various steps can be executed in reverse order or simultaneously. At the same time, other operations can also be added to these processes, or one or several steps can be removed from these processes.

[0023] Figure 1 It is a schematic flowchart of an intelligent control method for a material taking device for precision hardware parts provided by an embodiment of the present application.

[0024] Figure 2 It is a schematic structural diagram of an intelligent control system for a material taking device for precision hardware parts provided by an embodiment of the present application.

[0025] Explanation of reference numerals: six-view acquisition module 10, hardware part identification module 20, material taking path optimization module 30, material taking control module 40. Detailed implementation manners

[0026] The above description is only an overview of the technical solutions of the present application. In order to be able to more clearly understand the technical means of the present application, it can be implemented according to the content of the description. And in order to make the above and other purposes, features and advantages of the present application more obvious and understandable, the following specifically gives the detailed implementation manners of the present application.

[0027] In order to make the purpose, technical solutions and advantages of the present application clearer, the present application will be further described in detail below in conjunction with the accompanying drawings. The described embodiments should not be regarded as limitations of the present application. All other embodiments obtained by those of ordinary skill in the art without creative efforts fall within the scope of protection of the present application.

[0028] In the following description, "some embodiments" are involved, which describe a subset of all possible embodiments. However, it can be understood that "some embodiments" can be the same subset or different subsets of all possible embodiments, and can be combined with each other without conflict. The terms "first\second" involved are only used to distinguish similar objects and do not represent a specific order for the objects. The terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusion. For example, a process, method, system, product or server including a series of steps or units does not necessarily have to be limited to those steps or units clearly listed, but may include other steps or modules not clearly listed or inherent to these processes, methods, products or devices. Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by those skilled in the technical field to which this application belongs. The terms used herein are only for the purpose of describing the embodiments of the present application.

[0029] The embodiment of the present application provides an intelligent control method for a material taking device for precision hardware parts, as Figure 1 shown, the method includes:

[0030] Step S100, when the weight sensor in the preset material taking area shows an increase in weight, the image collector performs six-view acquisition on the preset material taking area to obtain a material taking area image. Among them, the weight sensor is mounted on the surface of the preset material taking area, and the preset material taking area is a transparent area. Specifically, the preset material taking area refers to an area that is preset for placing hardware parts and for the material taking device to take materials, and this area is made of transparent material so that the image collector can capture the internal situation. When the weight sensor (a sensor capable of detecting the weight of an object, mounted on the surface of the preset material taking area for sensing the placement of hardware parts) on the preset material taking area detects an increase in weight, it indicates that a hardware part has been placed in this area. At this time, the system triggers the image collector (a device for taking images of the preset material taking area, capable of obtaining six-view images) to perform six-view acquisition on the preset material taking area, that is, to take images of the material taking area from six angles: front, back, left, right, up, and down.

[0031] Step S200, identify the hardware parts in the material taking area image to obtain a hardware part model identification result and a hardware part model identification result. Among them, the hardware part model identification result has a hardware part positioning identification area. Specifically, identify the hardware parts in the obtained material taking area image through image processing technology, including identifying the model of the hardware part (that is, the type or kind of the hardware part) and the model of the hardware part (that is, specific shape, size and other information of the hardware part), and mark the positioning area of the hardware part in the image.

[0032] In a possible implementation, the hardware part is identified from the image of the material taking area to obtain the identification result of the hardware part model and the identification result of the hardware part model. Step S200 further includes step S210 of communicating with the weight sensor to obtain the increased weight information. Specifically, when the weight sensor in the preset material taking area detects an increase in weight, the intelligent control system communicates with the weight sensor to read and record the current increased weight information, which is a specific weight value or a range of weight changes, and is used to judge the possible type of the placed hardware part. Step S220, obtaining a first hardware part model to be verified that meets the increased weight information, where the first hardware part model to be verified has a number of images of the material taking area to be verified with a number of placement postures. Specifically, the intelligent control system selects from the preset hardware part database the hardware part models that conform to this weight characteristic according to the obtained increased weight information. These models are the "first hardware part models to be verified", and each model contains a number of images of the material taking area to be verified with different placement postures. Among them, the hardware part database is a database that stores information such as the models, weights, dimensions, and images of various hardware parts.

[0033] Step S230, for the image of the material taking area, traverse and compare the number of images of the material taking area to be verified to obtain a number of image similarities. Specifically, the intelligent control system compares the image of the material taking area (i.e., the actually captured image of the hardware part) with the images of the material taking area to be verified of each first hardware part model to be verified one by one. Through algorithms such as image feature extraction and feature matching, a series of image similarity values are finally obtained, which are used to evaluate the similarity between the two. Among them, the value range of the image similarity is between 0 and 1, and the larger the value, the more similar. Step S240, when any one of the number of image similarities is greater than or equal to the image similarity threshold, set the first hardware part model to be verified as the identification result of the hardware part model. Specifically, the intelligent control system sets an image similarity threshold to judge which first hardware part model the image of the material taking area is most similar to. If the image similarity of a certain model is greater than or equal to this threshold, then this model is considered to be the correct identification result of the hardware part model. Step S250, at the same time, according to the placement posture of the image similarity that is greater than or equal to the image similarity threshold, extract the identification result of the hardware part model. Specifically, after determining the hardware part model, the intelligent control system further extracts the corresponding identification result of the hardware part model according to the placement posture with the highest image similarity (i.e., the placement posture of the image of the material taking area to be verified that is most similar to the image of the material taking area, including direction, angle, etc.). This result includes information such as the specific shape, dimensions, and positioning identification area of the hardware part. This implementation method can more accurately identify the model and model of the hardware part by traversing and comparing multiple images of the material taking area to be verified, achieving the technical effect of improving the accuracy and efficiency of hardware part identification.

[0034] In a possible implementation, for the image of the picking area, traverse the several picking area images to be verified for comparison, and obtain several image similarities. Step S230 further includes step S231, extracting a first picking area image to be verified from the several picking area images to be verified, where the first picking area image to be verified has a front view to be verified, a rear view to be verified, a first side view to be verified, a second side view to be verified, a top view to be verified, and a bottom view to be verified. Specifically, the intelligent control system extracts each picking area image to be verified from multiple picking area images with different placement postures pre-stored and associated with the first type of hardware to be verified. These picking area images to be verified comprehensively cover various possible placement postures of the hardware, including but not limited to the front view, rear view, first side view, second side view, top view, and bottom view. Among them, the first picking area image to be verified is the picking area image to be verified that is currently being extracted for comparison during the traversal process. Step S232, the image of the picking area includes a front view of the picking area, a rear view of the picking area, a first side view of the picking area, a second side view of the picking area, a top view of the picking area, and a bottom view of the picking area. According to the front view to be verified, the rear view to be verified, the first side view to be verified, the second side view to be verified, the top view to be verified, and the bottom view to be verified, compare with the front view of the picking area, the rear view of the picking area, the first side view of the picking area, the second side view of the picking area, the top view of the picking area, and the bottom view of the picking area, and obtain a first image similarity. Specifically, compare the extracted first picking area image to be verified (including the front view, rear view, first side view, second side view, top view, and bottom view) with the actually collected image of the picking area (also including these views) one by one, and quantitatively evaluate the similarity between the two. Finally, for each pair of views, an image similarity value is obtained, and these values together constitute the first image similarity (the first image similarity is a set containing multiple view similarity values, or can also be a single value that synthesizes these view similarity values).

[0035] Step S233, add the first image similarity into the several image similarities. Specifically, after completing the comparison with the first image of the material taking area to be verified, the intelligent control system adds the obtained first image similarity to the image similarity set used to store all comparison results. This set is continuously updated as the traversal process progresses and finally contains the similarity information between all images of the material taking areas to be verified and the image of the material taking area. This implementation method comprehensively evaluates information such as the shape, size, and placement posture of the hardware by collecting and comparing images of the hardware from multiple perspectives (front view, rear view, side view, top view, and bottom view), effectively reducing recognition errors caused by perspective limitations or image occlusion. By comparing each image of the material taking area to be verified one by one, it ensures that the most matching model among multiple possible hardware models is accurately found for the actual image of the material taking area, achieving the technical effect of improving the accuracy and reliability of hardware recognition.

[0036] In a possible implementation, based on the to-be-verified front view, the to-be-verified rear view, the to-be-verified first side view, the to-be-verified second side view, the to-be-verified top view, and the to-be-verified bottom view, compare them with the pick-up area front view, the pick-up area rear view, the pick-up area first side view, the pick-up area second side view, the pick-up area top view, and the pick-up area bottom view to obtain the first image similarity. Step S232 further includes step S2321, compare the to-be-verified front view and the pick-up area front view to obtain the front view similarity, compare the to-be-verified rear view and the pick-up area rear view to obtain the rear view similarity, compare the to-be-verified first side view and the pick-up area first side view to obtain the first side view similarity, compare the to-be-verified second side view and the pick-up area second side view to obtain the second side view similarity, compare the to-be-verified top view and the pick-up area top view to obtain the top view similarity, and compare the to-be-verified bottom view and the pick-up area bottom view to obtain the bottom view similarity. Specifically, the intelligent control system first compares the to-be-verified front view in the to-be-verified pick-up area image with the pick-up area front view in the actual pick-up area image, and evaluates the similarity degree of the two images by calculating the distance or similarity between feature points, so as to obtain a front view similarity value. Immediately afterwards, compare the to-be-verified rear view with the pick-up area rear view, and similarly obtain a rear view similarity value through feature extraction and matching algorithms. Similarly, the first side view and the second side view are also compared respectively to obtain the first side view similarity and the second side view similarity. Finally, compare the to-be-verified top view with the pick-up area top view, and the to-be-verified bottom view with the pick-up area bottom view, and obtain the top view similarity and the bottom view similarity respectively. Step S2322, take the minimum value of the front view similarity, the rear view similarity, the first side view similarity, the second side view similarity, the top view similarity, and the bottom view similarity, and set it as the first image similarity. Specifically, after completing the comparison of all views, the intelligent control system takes the minimum value of these view similarities as the first image similarity. Because in practical applications, even if the overall shape of the hardware part is similar, small differences in a certain local view (such as edges, corners, etc.) may lead to a significant reduction in the overall similarity. Therefore, taking the minimum value can ensure that in subsequent judgments, significant differences in other views will not be ignored due to the high similarity of a certain view. This implementation method can ensure that the most matching model with the actual pick-up area image is accurately found among multiple possible hardware part models by comparing each view one by one and taking the minimum value as the first image similarity, achieving the technical effect of improving the accuracy and reliability of hardware part recognition.

[0037] In a possible implementation, the to-be-verified front view and the front view of the picking area are compared to obtain the front view similarity. Step S2321 further includes step S23211 of obtaining an image comparison network, where the image comparison network includes a first feature extraction channel, a second feature extraction channel, and a feature comparison channel. The model structures and model parameters of the first feature extraction channel and the second feature extraction channel are exactly the same. The feature comparison channel is used to evaluate the proportion of feature descriptors with a deviation less than or equal to the deviation threshold. Specifically, a neural network for image comparison is constructed, which consists of three main parts: a first feature extraction channel, a second feature extraction channel, and a feature comparison channel. The first feature extraction channel and the second feature extraction channel have exactly the same model structure and model parameters, that is, they can extract the same feature descriptors (mathematical representations of image features) from the input images. The feature comparison channel is used to evaluate the deviation between two feature descriptors and calculate the proportion of deviations less than or equal to the deviation threshold (a preset threshold used to determine whether the difference between two feature descriptors is small enough to consider the two images similar), and this proportion is used as a measure of image similarity. Step S23212: Input the to-be-verified front view into the first feature extraction channel to obtain a first feature descriptor matrix, and input the front view of the picking area into the second feature extraction channel to obtain a second feature descriptor matrix. Specifically, input the to-be-verified front view into the first feature extraction channel, and through the processing of this channel, obtain a first feature descriptor matrix containing image features. At the same time, input the front view of the picking area into the second feature extraction channel to obtain a second feature descriptor matrix containing image features. These two feature descriptor matrices are used for feature comparison. Step S23213: Input the first feature descriptor matrix and the second feature descriptor matrix into the feature comparison channel to obtain the front view similarity. Specifically, input the first feature descriptor matrix and the second feature descriptor matrix into the feature comparison channel. The feature comparison channel calculates the deviation between these two feature descriptor matrices and evaluates the proportion of deviations less than or equal to the deviation threshold, and uses this proportion as a measure of the front view similarity to obtain the front view similarity. This implementation method can accurately evaluate the similarity between two images by constructing an image comparison network and using the feature extraction channel and the feature comparison channel to extract and compare image features. Since the feature extraction channels have the same model structure and parameters, it can ensure the consistency and comparability of the feature descriptors extracted from the two images. By calculating the deviation between the feature descriptors and evaluating the proportion of deviations less than or equal to the threshold, a robust and reliable similarity measure can be obtained, thus achieving the technical effect of improving the accuracy and efficiency of image similarity recognition.

[0038] Step S300: Optimize the material taking path based on the recognition result of the hardware part model to obtain a recommended material taking path. Specifically, based on the recognition result of the hardware part model, the optimization of the material taking path is carried out, including considering factors such as the size and shape of the hardware part and the movement constraints of the robotic arm, and planning an optimal path from the preset material taking area to the material taking target position, that is, the recommended material taking path.

[0039] In a possible implementation manner, when optimizing the material taking path based on the recognition result of the hardware part model to obtain a recommended material taking path, step S300 further includes step S310: Extract the passing space diameter threshold according to the recognition result of the hardware part model, where the passing space diameter threshold is the distance between the two farthest points distributed on the recognition result of the hardware part model. Specifically, analyze the recognition result of the hardware part model to obtain its three-dimensional model data. In the three-dimensional model of the hardware part, find the two farthest points (usually the vertices or edge points of the hardware part), and calculate the straight-line distance between these two points, which is defined as the passing space diameter threshold. The passing space diameter threshold is a key parameter that reflects the maximum width or diameter occupied by the hardware part in space and is used to ensure that the robotic arm has enough space to pass during path planning. Step S320: Obtain the material taking target position and the movement constraint area of the robotic arm. Specifically, determine the target position of material taking, that is, the input position of the next process or equipment. At the same time, obtain the movement constraint area of the robotic arm, which defines the space range where the robotic arm can move, and this range is restricted by factors such as the size of the workbench and the position of obstacles. Step S330: Starting from the preset material taking area and ending at the material taking target position, perform the shortest path planning in the movement constraint area of the robotic arm based on the passing space diameter threshold to obtain the recommended material taking path. Specifically, starting from the preset material taking area and ending at the material taking target position, perform path planning based on the passing space diameter threshold and the movement constraint area of the robotic arm. The path planning algorithm needs to consider that the robotic arm needs to avoid obstacles during movement and ensure that there is enough space to pass through the hardware part. By calculating multiple possible paths and selecting the shortest path that meets the constraint conditions as the recommended material taking path. This implementation method ensures that the robotic arm has enough space to pass through the hardware part during movement, avoiding collisions or jams by extracting the passing space diameter threshold of the hardware part. At the same time, considering the movement constraint area of the robotic arm, it ensures that the path generated by the path planning algorithm is feasible and meets the requirements of the actual working environment, ensuring that the robotic arm can move safely and efficiently during material taking. By selecting the shortest path as the recommended material taking path, the movement efficiency of the robotic arm is further improved, the material taking time is reduced, and the efficiency of the entire production line is improved.

[0040] In a possible implementation, starting from the preset material taking area and ending at the material taking target position, based on the passing space diameter threshold, the shortest path planning is carried out in the moving constraint area of the robotic arm to obtain the material taking recommended path. Step S330 further includes step S331 of setting the passing area in the moving constraint area of the robotic arm that is less than or equal to the passing space diameter threshold as a virtual obstacle area according to the passing space diameter threshold. Specifically, traverse the moving constraint area of the robotic arm, check each sub-area therein (which can be small grids or area divisions), and determine whether the width or diameter of these sub-areas is less than or equal to the passing space diameter threshold of the hardware. If the width or diameter of a certain sub-area is less than or equal to the passing space diameter threshold of the hardware, then this sub-area is regarded as a potential obstacle area, that is, a virtual obstacle area. These areas will be regarded as non-passable areas in path planning. Step S332 is to obtain the physical obstacle area of the moving constraint area of the robotic arm. Specifically, obtain the actual obstacle areas within the moving constraint area of the robotic arm, and these obstacle areas include fixed equipment on the workbench, other mechanical components, already placed hardware, etc. The information of the physical obstacle area is obtained through pre-surveying or real-time sensor data. Step S333 is to carry out the shortest path planning in the moving constraint area of the robotic arm with the preset material taking area as the starting point and the material taking target position as the ending point according to the virtual obstacle area and the physical obstacle area to obtain the material taking recommended path. Specifically, take the preset material taking area as the starting point and the material taking target position as the ending point, and at the same time consider the virtual obstacle area and the physical obstacle area to carry out the shortest path planning within the moving constraint area of the robotic arm. The path planning algorithm attempts to find a path from the starting point to the ending point. This path needs to avoid all obstacle areas (including virtual obstacle areas and physical obstacle areas) and be as short as possible in length. Finally, the system outputs the path that meets the conditions as the material taking recommended path. In this implementation, by setting the virtual obstacle area, the system can more accurately evaluate the possible space limitations encountered by the robotic arm during movement, thereby avoiding collisions or jams caused by insufficient space. At the same time, considering the physical obstacle area and the virtual obstacle area for the shortest path planning ensures that the robotic arm can avoid all obstacle areas during movement and reach the target position in the shortest time, thus improving the safety and efficiency of the robotic arm during the material taking process.

[0041] Step S400: Perform picking control based on the hardware part positioning identification area and the recommended picking path. Specifically, perform picking control according to the hardware part positioning identification area and the recommended picking path, including controlling the robotic arm to move to above the hardware part positioning identification area along the recommended path, then accurately grasping the hardware part and moving it to the specified picking target position. In the embodiment of the present application, when the weight sensor in the preset picking area shows an increase in weight, six-view images of the preset picking area are collected through an image collector to obtain an image of the picking area. The image is processed using image recognition technology to identify the model and type of the hardware part, and the positioning identification area of the hardware part is determined. According to the identified result of the hardware part model, the picking path is optimized to obtain a recommended picking path. Based on the hardware part positioning identification area and the recommended picking path, the picking device is controlled to perform the picking operation and other technical means, which can be applicable to various hardware part models without separately configuring a picking device for each model, achieving the technical effects of high versatility and improved production efficiency.

[0042] In the above, with reference to Figure 1 A method for intelligent control of a picking device for precision hardware parts according to an embodiment of the present invention was described in detail. Next, a system for intelligent control of a picking device for precision hardware parts according to an embodiment of the present invention will be described with reference to Figure 2 Describe a system for intelligent control of a picking device for precision hardware parts according to an embodiment of the present invention.

[0043] A system for intelligent control of a picking device for precision hardware parts according to an embodiment of the present invention is used to solve the technical problems of poor versatility in the control of existing hardware part picking devices and the limitation of production efficiency improvement, achieving the technical effects of high versatility and improved production efficiency. A system for intelligent control of a picking device for precision hardware parts includes: a six-view image acquisition module 10, a hardware part identification module 20, a picking path optimization module 30, and a picking control module 40.

[0044] The six-view image acquisition module 10 is used to collect six-view images of the preset picking area through an image collector when the weight sensor in the preset picking area shows an increase in weight, to obtain an image of the picking area. Among them, the weight sensor is mounted on the surface of the preset picking area, and the preset picking area is a transparent area; the hardware part identification module 20 is used to identify the hardware part in the image of the picking area to obtain the identification result of the hardware part model and the identification result of the hardware part type. Among them, the identification result of the hardware part model has a positioning identification area of the hardware part; the picking path optimization module 30 is used to optimize the picking path according to the identification result of the hardware part model to obtain a recommended picking path; the picking control module 40 is used to perform picking control according to the positioning identification area of the hardware part and the recommended picking path.

[0045] Next, the specific configuration of the hardware component recognition module 20 will be described in detail. As described above, the hardware component in the image of the picking area is recognized to obtain the recognition result of the hardware component model and the recognition result of the hardware component model. The hardware component recognition module 20 may further include: an increased weight information acquisition unit for communicating with a weight sensor to obtain increased weight information; a first hardware component model to be verified acquisition unit for obtaining a first hardware component model to be verified that meets the increased weight information, where the first hardware component model to be verified has a plurality of picking area images to be verified with a plurality of placement postures; an image comparison unit for comparing the picking area image with the plurality of picking area images to be verified to obtain a plurality of image similarities; a hardware component model recognition result acquisition unit for setting the first hardware component model to be verified as the hardware component model recognition result when any one of the plurality of image similarities is greater than or equal to the image similarity threshold; and a hardware component model recognition result extraction unit for extracting the hardware component model recognition result according to the placement posture of the image similarity that is greater than or equal to the image similarity threshold at the same time.

[0046] Among them, for the picking area image, the plurality of picking area images to be verified are traversed and compared to obtain a plurality of image similarities. The image comparison unit may further include: a first picking area image to be verified extraction subunit for extracting a first picking area image to be verified from the plurality of picking area images to be verified, where the first picking area image to be verified has a front view to be verified, a rear view to be verified, a first side view to be verified, a second side view to be verified, a top view to be verified, and a bottom view to be verified; a multi-view comparison subunit for the picking area image including a picking area front view, a picking area rear view, a picking area first side view, a picking area second side view, a picking area top view, and a picking area bottom view, and comparing the front view to be verified, the rear view to be verified, the first side view to be verified, the second side view to be verified, the top view to be verified, and the bottom view to be verified with the picking area front view, the picking area rear view, the picking area first side view, the picking area second side view, the picking area top view, and the picking area bottom view to obtain a first image similarity; and an image similarity acquisition subunit for adding the first image similarity to the plurality of image similarities.

[0047] Among them, according to the to-be-verified front view, the to-be-verified rear view, the to-be-verified first side view, the to-be-verified second side view, the to-be-verified top view, and the to-be-verified bottom view, compare with the picking area front view, the picking area rear view, the picking area first side view, the picking area second side view, the picking area top view, and the picking area bottom view to obtain the first image similarity. The multi-view comparison sub-unit may further include: Each view similarity acquisition micro-unit is used to compare the to-be-verified front view and the picking area front view to obtain the front view similarity, compare the to-be-verified rear view and the picking area rear view to obtain the rear view similarity, compare the to-be-verified first side view and the picking area first side view to obtain the first side view similarity, compare the to-be-verified second side view and the picking area second side view to obtain the second side view similarity, compare the to-be-verified top view and the picking area top view to obtain the top view similarity, and compare the to-be-verified bottom view and the picking area bottom view to obtain the bottom view similarity; The similarity minimum value extraction micro-unit is used to take the minimum value of the front view similarity, the rear view similarity, the first side view similarity, the second side view similarity, the top view similarity, and the bottom view similarity, and set it as the first image similarity.

[0048] Among them, compare the to-be-verified front view and the picking area front view to obtain the front view similarity. The each view similarity acquisition micro-unit may further include: The image comparison network acquisition sub-micro-unit is used to obtain an image comparison network, where the image comparison network includes a first feature extraction channel, a second feature extraction channel, and a feature comparison channel. The model structures and model parameters of the first feature extraction channel and the second feature extraction channel are exactly the same. The feature comparison channel is used to evaluate the proportion of the feature descriptors with the deviation less than or equal to the deviation threshold; The feature descriptor matrix acquisition sub-micro-unit is used to input the to-be-verified front view into the first feature extraction channel to obtain a first feature descriptor matrix, and input the picking area front view into the second feature extraction channel to obtain a second feature descriptor matrix; The front view similarity acquisition sub-micro-unit is used to input the first feature descriptor matrix and the second feature descriptor matrix into the feature comparison channel to obtain the front view similarity.

[0049] Next, the specific configuration of the material taking path optimization module 30 will be described in detail. As described above, according to the recognition result of the hardware part model, the material taking path is optimized to obtain the recommended material taking path. The material taking path optimization module 30 may further include: a passing space diameter threshold extraction unit for extracting a passing space diameter threshold according to the recognition result of the hardware part model, where the passing space diameter threshold is the distance between the two farthest points distributed on the recognition result of the hardware part model; a constraint information acquisition unit for obtaining the material taking target position and the robot arm movement constraint area; a shortest path planning unit for taking the preset material taking area as the starting point and the material taking target position as the ending point, and performing shortest path planning in the robot arm movement constraint area based on the passing space diameter threshold to obtain the recommended material taking path.

[0050] Among them, taking the preset material taking area as the starting point and the material taking target position as the ending point, and performing shortest path planning in the robot arm movement constraint area based on the passing space diameter threshold to obtain the recommended material taking path, the shortest path planning unit may further include: a virtual obstacle area setting sub-unit for setting the passing area in the robot arm movement constraint area that is less than or equal to the passing space diameter threshold as a virtual obstacle area according to the passing space diameter threshold; a physical obstacle area acquisition sub-unit for obtaining the physical obstacle area of the robot arm movement constraint area; a material taking recommended path acquisition sub-unit for performing shortest path planning in the robot arm movement constraint area with the virtual obstacle area and the physical obstacle area, taking the preset material taking area as the starting point and the material taking target position as the ending point, to obtain the recommended material taking path.

[0051] The intelligent control system of a material taking device for precision hardware parts provided by the embodiment of the present invention can execute the intelligent control method of a material taking device for precision hardware parts provided by any embodiment of the present invention, and has the corresponding functional modules and beneficial effects of the execution method.

[0052] Although the present application makes various references to certain modules in the system according to the embodiments of the present application, however, any number of different modules can be used and run on the user terminal and / or the server. The included various units and modules are only divided according to the functional logic, but are not limited to the above division, as long as the corresponding functions can be realized; in addition, the specific names of the functional units are only for the convenience of mutual distinction and do not limit the protection scope of the present invention.

[0053] The above specific embodiments do not constitute a limitation on the protection scope of this application. Those skilled in the art should understand that various modifications, combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principle of this application shall be included within the protection scope of this application. In some cases, the actions or steps recorded in this application can be executed in a sequence different from that in the embodiments and still achieve the desired results. Additionally, the processes depicted in the drawings do not necessarily require the specific order or consecutive order shown to achieve the desired results. In certain embodiments, multitasking and parallel processing are also possible or may be advantageous.

Claims

1. An intelligent control method for a material taking device for precision hardware parts, characterized in that, Including: When the weight sensor in the preset material taking area shows an increase in weight, the preset material taking area is collected in six views by an image collector to obtain a material taking area image. Among them, the weight sensor is mounted on the surface of the preset material taking area, and the preset material taking area is a transparent area; Perform hardware part identification on the material taking area image to obtain a hardware part model identification result and a hardware part model identification result, including: Communicate with the weight sensor to obtain the increased weight information; Obtain the first hardware part model to be verified that meets the increased weight information, where the first hardware part model to be verified has several material taking area images with several placement postures; For the material taking area image, traverse and compare the several material taking area images to be verified to obtain several image similarities; When any one of the several image similarities is greater than or equal to the image similarity threshold, set the first hardware part model to be verified as the hardware part model identification result; At the same time, according to the placement posture of the hardware part in the image where the image similarity is greater than or equal to the threshold, extract the hardware part model identification result; Among them, the hardware part model identification result has a hardware part positioning identification area; Optimize the material taking path according to the hardware part model identification result to obtain a recommended material taking path; Perform material taking control according to the hardware part positioning identification area and the recommended material taking path.

2. The intelligent control method of a material taking device for precision hardware parts according to claim 1, wherein, For the material taking area image, traverse and compare the several material taking area images to be verified to obtain several image similarities, including: Extract the first material taking area image to be verified from the several material taking area images to be verified, where the first material taking area image to be verified has a front view to be verified, a rear view to be verified, a first side view to be verified, a second side view to be verified, a top view to be verified, and a bottom view to be verified; The material taking area image includes a front view of the material taking area, a rear view of the material taking area, a first side view of the material taking area, a second side view of the material taking area, a top view of the material taking area, and a bottom view of the material taking area; According to the front view to be verified, the rear view to be verified, the first side view to be verified, the second side view to be verified, the top view to be verified, and the bottom view to be verified, compare with the front view of the material taking area, the rear view of the material taking area, the first side view of the material taking area, the second side view of the material taking area, the top view of the material taking area, and the bottom view of the material taking area to obtain a first image similarity; Add the first image similarity to the several image similarities.

3. The intelligent control method of a material taking device for precision hardware parts according to claim 2, characterized in that, According to the front view to be verified, the rear view to be verified, the first side view to be verified, the second side view to be verified, the top view to be verified, and the bottom view to be verified, compare with the front view of the material taking area, the rear view of the material taking area, the first side view of the material taking area, the second side view of the material taking area, the top view of the material taking area, and the bottom view of the material taking area to obtain a first image similarity, including: Compare the front view to be verified and the front view of the material taking area to obtain a front view similarity; Compare the post-view to be verified and the post-view of the material taking area to obtain the post-view similarity; Compare the first side view to be verified and the first side view of the material taking area to obtain the first side view similarity; Compare the second side view to be verified and the second side view of the material taking area to obtain the second side view similarity; Compare the top view to be verified and the top view of the material taking area to obtain the top view similarity; Compare the bottom view to be verified and the bottom view of the material taking area to obtain the bottom view similarity; Take the minimum value of the front view similarity, the post-view similarity, the first side view similarity, the second side view similarity, the top view similarity and the bottom view similarity, and set it as the first image similarity.

4. The intelligent control method of a material taking device for precision hardware parts according to claim 3, characterized in that, Compare the front view to be verified and the front view of the material taking area to obtain the front view similarity, including: Obtain an image comparison network, where the image comparison network includes a first feature extraction channel, a second feature extraction channel and a feature comparison channel. The model structures and model parameters of the first feature extraction channel and the second feature extraction channel are exactly the same. The feature comparison channel is used to evaluate the proportion of feature descriptors with a deviation less than or equal to the deviation threshold; Input the front view to be verified into the first feature extraction channel to obtain a first feature descriptor matrix; Input the front view of the material taking area into the second feature extraction channel to obtain a second feature descriptor matrix; Input the first feature descriptor matrix and the second feature descriptor matrix into the feature comparison channel to obtain the front view similarity.

5. The intelligent control method of a material taking device for precision hardware parts according to claim 1, characterized in that, Optimize the material taking path according to the recognition result of the hardware part model to obtain a recommended material taking path, including: Extract the passing space diameter threshold according to the recognition result of the hardware part model, where the passing space diameter threshold is the distance between the two farthest points distributed on the recognition result of the hardware part model; Obtain the material taking target position and the robot arm movement constraint area; Taking the preset material taking area as the starting point and the material taking target position as the ending point, perform the shortest path planning in the robot arm movement constraint area based on the passing space diameter threshold to obtain the recommended material taking path.

6. The intelligent control method of a material taking device for precision hardware parts according to claim 5, wherein, Taking the preset material taking area as the starting point and the material taking target position as the ending point, perform the shortest path planning in the robot arm movement constraint area based on the passing space diameter threshold to obtain the recommended material taking path, including: According to the passing space diameter threshold, set the passing area in the robot arm movement constraint area that is less than or equal to the passing space diameter threshold as the virtual obstacle area; Obtain the physical obstacle area of the robot arm movement constraint area; According to the virtual obstacle area and the physical obstacle area, taking the preset material taking area as the starting point and the material taking target position as the ending point, perform the shortest path planning in the robot arm movement constraint area to obtain the recommended material taking path.

7. An intelligent control system for a material taking device of precision hardware parts, characterized in that, The system is used to implement the intelligent control method for the material taking device for precision hardware parts according to any one of claims 1-6. The system includes: Six-view acquisition module, which is used to perform six-view acquisition on a preset material-taking area through an image collector to obtain a material-taking area image when a weight sensor in the preset material-taking area shows an increase in weight. Among them, the weight sensor is mounted on the surface of the preset material-taking area, and the preset material-taking area is a transparent area; Hardware part recognition module, which is used to recognize hardware parts in the material-taking area image to obtain a hardware part model recognition result and a hardware part model recognition result. Among them, the hardware part model recognition result has a hardware part positioning identification area; Material-taking path optimization module, which is used to optimize the material-taking path according to the hardware part model recognition result to obtain a recommended material-taking path; Material-taking control module, which is used to perform material-taking control according to the hardware part positioning identification area and the recommended material-taking path.

Images