Intelligent control method and system for material taking device for precision hardware

Abstract

The invention discloses an intelligent control method and system for a material taking device for precision hardware, and relates to the related field of industrial automation control, and the method comprises the steps: carrying out the six-view collection of a preset material taking region through an image collector when a weight sensor in the preset material taking region displays that the weight is increased, and obtaining a material taking region image; hardware recognition is carried out on the material taking area image, and a hardware model recognition result and a hardware model recognition result are obtained; carrying out material taking path optimization according to a hardware model identification result to obtain a material taking recommendation path; and material taking control is carried out according to the hardware positioning identification area and the material taking recommendation path. The technical problems that an existing hardware taking device is poor in control universality, and production efficiency improvement is limited are solved, and the technical effects that the high universality is achieved, and the production efficiency is improved are achieved.

Description

Technical Field

[0001] The present application relates to the field of industrial automation control, and in particular to an intelligent control method and system for a material taking device for precision hardware. Background Art

[0002] In the metal processing and manufacturing industry, the retrieving of hardware is a crucial link. In the traditional method of retrieving hardware, the operator needs to rely on experience and the naked eye to judge the model of the hardware, and then manually adjust the parameters and configuration of the retrieving device according to the identified hardware model to ensure that the hardware can be accurately grasped. After the configuration is completed, the retrieving device begins to perform the retrieving operation, taking the hardware from the preset position and placing it in the specified position. This method has poor versatility, and each hardware model needs to be configured with a separate retrieving device, resulting in a long production preparation time and limiting the improvement of production efficiency.

[0003] In the current related technologies, the control of the hardware material taking device has the technical problem of poor versatility and limiting the improvement of production efficiency. Summary of the invention

[0004] The present application provides an intelligent control method and system for a material picking device for precision hardware. When the weight sensor of a preset material picking area shows an increase in weight, six views of the preset material picking area are collected by an image collector to obtain an image of the material picking area. The image is processed using image recognition technology to identify the type and model of the hardware and determine the positioning identification area of ​​the hardware. According to the results of the identified hardware model, the material picking path is optimized and calculated to obtain a recommended material picking path. According to the hardware positioning identification area and the recommended material picking path, the material picking device is controlled to perform material picking operations. Such technical means can be applied to a variety of hardware models, and there is no need to configure a material picking device for each model separately, thereby achieving a technical effect of high versatility and improved production efficiency.

[0005] The present application provides an intelligent control method for a material taking device for precision hardware, comprising: When the weight sensor of the preset material picking area shows an increase in weight, six views of the preset material picking area are collected by the image collector to obtain an image of the material picking area, wherein the weight sensor is mounted on the surface of the preset material picking area and the preset material picking area is a transparent area; hardware identification is performed on the material picking area image to obtain hardware model identification results and hardware model identification results, wherein the hardware model identification result has a hardware positioning identification area; the material picking path is optimized according to the hardware model identification result to obtain a recommended material picking path; material picking control is performed according to the hardware positioning identification area and the recommended material picking path.

[0006] In a possible implementation, the material picking area image is subjected to hardware identification, a hardware model identification result and a hardware model identification result are obtained, and the following processing is performed: Communicate with the weight sensor to obtain added weight information; obtain a first hardware model to be verified that meets the added weight information, wherein the first hardware model to be verified has a plurality of material picking area images to be verified with a plurality of placement postures; for the material picking area image, traverse the plurality of material picking area images to be verified for comparison to obtain a plurality of image similarities; when any one of the plurality of image similarities is greater than or equal to an image similarity threshold, set the first hardware model to be verified as the hardware model recognition result; and at the same time, extract the hardware model recognition result according to the placement posture with an image similarity greater than or equal to the image similarity threshold.

[0007] In a possible implementation, the material picking area image is compared with the plurality of material picking area images to be verified, and a plurality of image similarities are obtained, and the following processing is performed: A first material picking area image to be verified is extracted from the plurality of material picking area images to be verified, wherein the first material picking area image to be verified has a front view to be verified, a rear view to be verified, a first side view to be verified, a second side view to be verified, a top view to be verified and a bottom view to be verified; the material picking area image includes a front view of the material picking area, a rear view of the material picking area, a first side view of the material picking area, a second side view of the material picking area, a top view of the material picking area and a bottom view of the material picking area; according to the front view to be verified, the rear view to be verified, the first side view to be verified, the second side view to be verified, the top view to be verified and the bottom view to be verified, the material picking area front view, the rear view of the material picking area, the first side view of the material picking area, the second side view of the material picking area, the top view of the material picking area and the bottom view of the material picking area are compared to obtain a first image similarity; and the first image similarity is added to the plurality of image similarities.

[0008] In a possible implementation, the front view to be verified, the rear view to be verified, the first side view to be verified, the second side view to be verified, the top view to be verified and the bottom view to be verified are compared with the front view of the material picking area, the rear view of the material picking area, the first side view of the material picking area, the second side view of the material picking area, the top view of the material picking area and the bottom view of the material picking area to obtain a first image similarity, and the following processing is performed: Compare the front view to be verified with the front view of the material picking area to obtain the similarity of the front view; compare the rear view to be verified with the rear view of the material picking area to obtain the similarity of the rear view; compare the first side view to be verified with the first side view of the material picking area to obtain the similarity of the first side view; compare the second side view to be verified with the second side view of the material picking area to obtain the similarity of the second side view; compare the top view to be verified with the top view of the material picking area to obtain the similarity of the top view; compare the bottom view to be verified with the bottom view of the material picking area to obtain the similarity of the bottom view; take the minimum value of the front view similarity, the rear view similarity, the first side view similarity, the second side view similarity, the top view similarity and the bottom view similarity, and set it as the first image similarity.

[0009] In a possible implementation, the front view to be verified is compared with the front view of the material taking area to obtain the front view similarity, and the following processing is performed: An image comparison network is obtained, wherein the image comparison network includes a first feature extraction channel, a second feature extraction channel and a feature comparison channel, the model structure and model parameters of the first feature extraction channel and the second feature extraction channel are exactly the same, and the feature comparison channel is used to evaluate the proportion of feature descriptor deviations that are less than or equal to a deviation threshold; the front view to be verified is input into the first feature extraction channel to obtain a first feature descriptor matrix; the front view of the material collection area is input into the second feature extraction channel to obtain a second feature descriptor matrix; the first feature descriptor matrix and the second feature descriptor matrix are input into the feature comparison channel to obtain the front view similarity.

[0010] In a possible implementation, the material picking path is optimized according to the hardware model recognition result to obtain a recommended material picking path, and the following processing is performed: According to the hardware model recognition result, extract the passing space diameter threshold, wherein the passing space diameter threshold is the distance between the two farthest points on the hardware model recognition result; obtain the material picking target position and the robot arm movement constraint area; take the preset material picking area as the starting point and the material picking target position as the end point, perform the shortest path planning in the robot arm movement constraint area based on the passing space diameter threshold, and obtain the recommended material picking path.

[0011] In a possible implementation, taking the preset material picking area as the starting point and the material picking target position as the end point, the shortest path planning is performed in the robot arm movement constraint area based on the passing space diameter threshold, the material picking recommended path is obtained, and the following processing is performed: According to the passing space diameter threshold, the passage area in which the robot arm movement constraint area is less than or equal to the passing space diameter threshold is set as a virtual obstacle area; the physical obstacle area in the robot arm movement constraint area is obtained; according to the virtual obstacle area and the physical obstacle area, with the preset material picking area as the starting point and the material picking target position as the end point, the shortest path planning is performed in the robot arm movement constraint area to obtain the recommended material picking path.

[0012] The present application also provides an intelligent control system for a material taking device for precision hardware, comprising: A six-view acquisition module, which is used to perform six-view acquisition on the preset material picking area through an image acquisition device to obtain an image of the material picking area when the weight sensor of the preset material picking area shows an increase in weight, wherein the weight sensor is mounted on the surface of the preset material picking area, and the preset material picking area is a transparent area; a hardware identification module, which is used to perform hardware identification on the material picking area image to obtain a hardware model identification result and a hardware model identification result, wherein the hardware model identification result has a hardware positioning identification area; a material picking path optimization module, which is used to optimize the material picking path according to the hardware model identification result to obtain a recommended material picking path; a material picking control module, which is used to control material picking according to the hardware positioning identification area and the recommended material picking path.

[0013] The present application proposes an intelligent control method and system for a material picking device for precision hardware. First, when the weight sensor of a preset material picking area shows an increase in weight, six views of the preset material picking area are collected by an image collector to obtain an image of the material picking area, wherein the weight sensor is mounted on the surface of the preset material picking area, and the preset material picking area is a transparent area. Then, hardware recognition is performed on the image of the material picking area to obtain hardware model recognition results and hardware model recognition results, wherein the hardware model recognition result has a hardware positioning identification area. Then, the material picking path is optimized according to the hardware model recognition result to obtain a recommended material picking path. Finally, material picking is controlled according to the hardware positioning identification area and the recommended material picking path, thereby achieving a technical effect of high versatility and improved production efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0014] In order to more clearly illustrate the technical solution of the embodiment of the present invention, the accompanying drawings of the embodiment of the present invention will be briefly introduced below. A flow chart is used in the present application to illustrate the operations performed by the system according to the embodiment of the present application. It should be understood that the preceding or following operations are not necessarily performed accurately in order. On the contrary, various steps can be processed in reverse order or simultaneously as needed. At the same time, other operations can also be added to these processes, or one or more operations can be removed from these processes.

[0015] Figure 1 A schematic flow chart of an intelligent control method for a material taking device for precision hardware provided in an embodiment of the present application.

[0016] Figure 2 A schematic structural diagram of an intelligent control system for a material taking device for precision hardware provided in an embodiment of the present application.

[0017] Description of the reference numerals: six-view acquisition module 10 , hardware identification module 20 , material picking path optimization module 30 , material picking control module 40 . DETAILED DESCRIPTION

[0018] The above description is only an overview of the technical solution of the present application. In order to more clearly understand the technical means of the present application, it can be implemented in accordance with the contents of the specification. In order to make the above and other purposes, features and advantages of the present application more obvious and easy to understand, the specific implementation methods of the present application are listed below.

[0019] In order to make the purpose, technical solutions and advantages of the present application clearer, the present application will be further described in detail below in conjunction with the accompanying drawings. The described embodiments should not be regarded as limiting the present application. All other embodiments obtained by ordinary technicians in the field without making creative work are within the scope of protection of this application.

[0020] In the following description, reference is made to "some embodiments", which describe a subset of all possible embodiments, but it is understood that "some embodiments" may be the same subset or different subsets of all possible embodiments, and may be combined with each other without conflict, and the terms "first\second" involved are merely to distinguish similar objects and do not represent a specific ordering of objects. The terms "including" and "having" and any variations are intended to cover non-exclusive inclusions, for example, a process, method, system, product, or server that includes a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or modules that are not clearly listed or inherent to these processes, methods, products, or devices. Unless otherwise defined, all technical and scientific terms used herein have the same meaning as those generally understood by technicians in the technical field of this application. The terms used herein are for the purpose of describing the embodiments of the present application only.

[0021] The embodiment of the present application provides an intelligent control method for a material taking device for precision hardware, such as Figure 1 As shown, the method includes: Step S100, when the weight sensor of the preset material picking area shows that the weight has increased, the preset material picking area is collected by the image collector in six views to obtain the image of the material picking area, wherein the weight sensor is mounted on the surface of the preset material picking area, and the preset material picking area is a transparent area. Specifically, the preset material picking area refers to an area that is pre-set for placing hardware and for the material picking device to pick up materials, and the area is made of transparent material so that the image collector can capture its internal situation. When the weight sensor on the preset material picking area (a sensor that can detect the weight of an object, mounted on the surface of the preset material picking area, and used to sense the placement of hardware) detects an increase in weight, it indicates that hardware has been placed in the area. At this time, the system triggers the image collector (a device for capturing images of the preset material picking area, capable of acquiring six-view images) to capture six views of the preset material picking area, that is, to capture images of the material picking area from six angles, front, back, left, right, up, and down.

[0022] Step S200, perform hardware identification on the material collection area image to obtain hardware model identification results and hardware model identification results, wherein the hardware model identification result has a hardware positioning identification area. Specifically, the obtained material collection area image is subjected to hardware identification through image processing technology, including identifying the model of the hardware (i.e., the type or type of the hardware) and the model of the hardware (i.e., the specific shape, size, and other information of the hardware), and marking the positioning area of ​​the hardware in the image.

[0023] In a possible implementation, the image of the material collection area is subjected to hardware identification, and the hardware model identification result and the hardware model identification result are obtained. Step S200 further includes step S210, communicating with the weight sensor to obtain the weight increase information. Specifically, when the weight sensor of the preset material collection area detects the weight increase, the intelligent control system communicates with the weight sensor to read and record the current weight increase information, which is a specific weight value or a range of weight change, and is used to determine the type of hardware that may be placed. Step S220, obtains the first hardware model to be verified that meets the weight increase information, wherein the first hardware model to be verified has several material collection area images to be verified with several placement postures. Specifically, the intelligent control system selects the hardware models that meet this weight feature from the preset hardware database according to the obtained weight increase information, and these models are the "first hardware models to be verified", and each model contains multiple material collection area images to be verified with different placement postures. Among them, the hardware database is a database that stores information such as the model, weight, size, image, etc. of multiple hardware.

[0024] Step S230, for the material picking area image, traverse the several material picking area images to be verified for comparison, and obtain several image similarities. Specifically, the intelligent control system compares the material picking area image (i.e., the hardware image actually taken) with the material picking area image to be verified of each first hardware model to be verified one by one, and finally obtains a series of image similarity values ​​through image feature extraction, feature matching and other algorithms, which are used to evaluate the similarity between the two, wherein the image similarity value range is between 0 and 1, and the larger the value, the more similar it is. Step S240, when any one of the several image similarities is greater than or equal to the image similarity threshold, the first hardware model to be verified is set as the hardware model recognition result. Specifically, the intelligent control system sets an image similarity threshold to determine which image of the first hardware model to be verified the material picking area image is most similar to. If the image similarity of a certain model is greater than or equal to this threshold, then the model is considered to be the correct hardware model recognition result. Step S250, extract the hardware model recognition result according to the placement posture of the image similarity greater than or equal to the image similarity threshold. Specifically, after determining the model of the hardware, the intelligent control system further extracts the corresponding hardware model recognition result according to the placement posture with the highest image similarity (i.e., the placement posture of the material collection area image to be verified that is most similar to the material collection area image, including direction, angle, etc.), and this result contains information such as the specific shape, size, and positioning identification area of ​​the hardware. This implementation method can more accurately identify the model and model of the hardware by traversing and comparing multiple material collection area images to be verified, thereby achieving the technical effect of improving the accuracy and efficiency of hardware recognition.

[0025] In a possible implementation, the material picking area image is traversed through the several material picking area images to be verified for comparison to obtain several image similarities, and step S230 further includes step S231, extracting the first material picking area image to be verified from the several material picking area images to be verified, wherein the first material picking area image to be verified has a front view to be verified, a rear view to be verified, a first side view to be verified, a second side view to be verified, a top view to be verified, and a bottom view to be verified. Specifically, the intelligent control system extracts each material picking area image to be verified one by one from the pre-stored material picking area images to be verified of multiple placement postures associated with the first hardware model to be verified. These material picking area images to be verified fully cover various possible placement postures of hardware, including but not limited to the front view, rear view, first side view, second side view, top view, and bottom view. Among them, the first material picking area image to be verified is the material picking area image to be verified that is currently being extracted for comparison during the traversal process. Step S232, the material picking area image includes a front view of the material picking area, a rear view of the material picking area, a first side view of the material picking area, a second side view of the material picking area, a top view of the material picking area, and a bottom view of the material picking area. According to the front view to be verified, the rear view to be verified, the first side view to be verified, the second side view to be verified, the top view to be verified, and the bottom view to be verified, the front view of the material picking area, the rear view of the material picking area, the first side view of the material picking area, the second side view of the material picking area, the top view of the material picking area, and the bottom view of the material picking area are compared to obtain the first image similarity. Specifically, the extracted first material picking area image to be verified (including the front view, the rear view, the first side view, the second side view, the top view, and the bottom view) is compared one by one with the actually collected material picking area image (also including these views), and the similarity between the two is quantitatively evaluated. Finally, for each pair of views, an image similarity value is obtained, and these values ​​together constitute the first image similarity (the first image similarity is a set containing multiple view similarity values, or it can be a single value that integrates these view similarity values).

[0026] Step S233, adding the first image similarity to the several image similarities. Specifically, after completing the comparison with the first material picking area image to be verified, the intelligent control system adds the obtained first image similarity to the image similarity set used to store all comparison results. This set is continuously updated as the traversal process proceeds, and ultimately contains the similarity information between all the material picking area images to be verified and the material picking area images. This implementation method comprehensively evaluates the shape, size, placement posture and other information of the hardware by collecting and comparing images of the hardware from multiple perspectives (front view, rear view, side view, top view and bottom view), effectively reducing the recognition errors caused by viewing angle limitations or image occlusion, and by comparing each material picking area image to be verified one by one, ensuring that the model that best matches the actual material picking area image is accurately found among multiple possible hardware models, thereby achieving the technical effect of improving the accuracy and reliability of hardware recognition.

[0027] In a possible implementation, the front view to be verified, the rear view to be verified, the first side view to be verified, the second side view to be verified, the top view to be verified and the bottom view to be verified are compared with the front view of the material picking area, the rear view of the material picking area, the first side view of the material picking area, the second side view of the material picking area, the top view of the material picking area and the bottom view of the material picking area to obtain a first image similarity. Step S232 further includes step S2321, comparing the front view to be verified with the front view of the material picking area to obtain a front view similarity, comparing the rear view to be verified with the rear view of the material picking area to obtain a rear view similarity, comparing the first side view to be verified with the first side view of the material picking area to obtain a first side view similarity, comparing the second side view to be verified with the second side view of the material picking area to obtain a second side view similarity, comparing the top view to be verified with the top view of the material picking area to obtain a top view similarity, and comparing the bottom view to be verified with the bottom view of the material picking area to obtain a bottom view similarity. Specifically, the intelligent control system first compares the front view to be verified in the image of the material picking area to be verified with the front view of the material picking area in the image of the actual material picking area, and evaluates the similarity of the two images by calculating the distance or similarity between the feature points, thereby obtaining a front view similarity value. Next, the rear view to be verified is compared with the rear view of the material picking area, and a rear view similarity value is obtained by feature extraction and matching algorithms. Similarly, the first side view and the second side view are also compared respectively to obtain the first side view similarity and the second side view similarity. Finally, the top view to be verified is compared with the top view of the material picking area, and the bottom view to be verified is compared with the bottom view of the material picking area, and the top view similarity and the bottom view similarity are obtained respectively. Step S2322, taking the minimum value of the front view similarity, the back view similarity, the first side view similarity, the second side view similarity, the top view similarity and the bottom view similarity, and setting it as the first image similarity. Specifically, after completing the comparison of all views, the intelligent control system takes the minimum value of these view similarities as the first image similarity. Because in actual applications, even if the overall shape of the hardware is similar, a slight difference in a local view (such as edges, corners, etc.) may lead to a significant reduction in the overall similarity. Therefore, taking the minimum value can ensure that in subsequent judgments, significant differences in other views will not be ignored due to the high similarity of a certain view. This implementation method can ensure that the model that best matches the actual material collection area image is accurately found among multiple possible hardware models by comparing each view one by one and taking the minimum value as the first image similarity, thereby achieving the technical effect of improving the accuracy and reliability of hardware recognition.

[0028] In a possible implementation, the front view to be verified and the front view of the material collection area are compared to obtain the front view similarity, and step S2321 further includes step S23211, obtaining an image comparison network, wherein the image comparison network includes a first feature extraction channel, a second feature extraction channel and a feature comparison channel, the model structure and model parameters of the first feature extraction channel and the second feature extraction channel are exactly the same, and the feature comparison channel is used to evaluate the proportion of feature descriptor deviations less than or equal to the deviation threshold. Specifically, a neural network for image comparison is constructed, which consists of three main parts: a first feature extraction channel, a second feature extraction channel and a feature comparison channel. The first feature extraction channel and the second feature extraction channel have exactly the same model structure and model parameters, that is, they can extract the same feature descriptor (a mathematical representation of image features) from the input image. The feature comparison channel is used to evaluate the deviation between two feature descriptors, and calculate the proportion of deviations less than or equal to the deviation threshold (a preset threshold used to determine whether the difference between the two feature descriptors is small enough to consider the two images similar), and use this proportion as a measure of image similarity. Step S23212, input the front view to be verified into the first feature extraction channel to obtain the first feature description sub-matrix, and input the front view of the material extraction area into the second feature extraction channel to obtain the second feature description sub-matrix. Specifically, the front view to be verified is input into the first feature extraction channel, and a first feature description sub-matrix containing image features is obtained through the processing of the channel. At the same time, the front view of the material extraction area is input into the second feature extraction channel to obtain a second feature description sub-matrix containing image features. These two feature description sub-matrices are used for feature comparison. Step S23213, input the first feature description sub-matrix and the second feature description sub-matrix into the feature comparison channel to obtain the front view similarity. Specifically, the first feature description sub-matrix and the second feature description sub-matrix are input into the feature comparison channel. The feature comparison channel calculates the deviation between the two feature description sub-matrices, and evaluates the proportion of deviations less than or equal to the deviation threshold, and uses this proportion as a measure of the front view similarity to obtain the front view similarity. This implementation method can accurately evaluate the similarity between two images by constructing an image comparison network and using feature extraction channels and feature comparison channels to extract and compare image features. Since the feature extraction channels have the same model structure and parameters, it can ensure that the feature descriptors extracted from the two images are consistent and comparable. By calculating the deviation between feature descriptors and evaluating the proportion of deviations less than or equal to the threshold, a robust and reliable similarity metric can be obtained, thereby achieving the technical effect of improving the accuracy and efficiency of image similarity recognition.

[0029] Step S300, optimizing the material picking path according to the hardware model recognition result to obtain a recommended material picking path. Specifically, optimizing the material picking path according to the hardware model recognition result includes considering factors such as the size and shape of the hardware and the movement constraints of the robot arm, and planning an optimal path from the preset material picking area to the material picking target position, i.e., the recommended material picking path.

[0030] In a possible implementation, the material picking path is optimized according to the hardware model recognition result to obtain the recommended material picking path. Step S300 further includes step S310, extracting the through space diameter threshold according to the hardware model recognition result, wherein the through space diameter threshold is the distance between the two farthest points on the hardware model recognition result. Specifically, the hardware model recognition result is parsed to obtain its three-dimensional model data, and the two farthest points (usually the vertices or edge points of the hardware) are found in the three-dimensional model of the hardware, and the straight-line distance between the two points is calculated, and this distance is defined as the through space diameter threshold. The through space diameter threshold is a key parameter, which reflects the maximum width or diameter occupied by the hardware in the space, and is used to ensure that the robot arm has enough space to pass during path planning. Step S320, obtaining the material picking target position and the robot arm movement constraint area. Specifically, the material picking target position is determined, that is, the input position of the next process or equipment. At the same time, the robot arm movement constraint area is obtained, which defines the spatial range in which the robot arm can move, and the range is limited by factors such as the size of the workbench and the position of obstacles. Step S330, taking the preset material picking area as the starting point and the material picking target position as the end point, the shortest path planning is performed in the robot arm movement constraint area based on the passing space diameter threshold, and the recommended material picking path is obtained. Specifically, taking the preset material picking area as the starting point and the material picking target position as the end point, the path planning is performed based on the passing space diameter threshold and the robot arm movement constraint area. The path planning algorithm needs to take into account that the robot arm needs to avoid obstacles during the movement process, while ensuring that there is enough space to pass the hardware. By calculating multiple possible paths, the shortest path that meets the constraint conditions is selected as the recommended material picking path. This implementation method ensures that the robot arm has enough space to pass the hardware during the movement process to avoid collision or jamming by extracting the passing space diameter threshold of the hardware. At the same time, considering the movement constraint area of ​​the robot arm, it is ensured that the path generated by the path planning algorithm is feasible and meets the requirements of the actual working environment, and ensures that the robot arm can move safely and efficiently during the material picking process. By selecting the shortest path as the recommended material picking path, the movement efficiency of the robot arm is further improved, the material picking time is reduced, and the efficiency of the entire production line is improved.

[0031] In a possible implementation, the preset material picking area is taken as the starting point, the material picking target position is taken as the end point, and the shortest path planning is performed in the robot arm movement constraint area based on the through space diameter threshold to obtain the recommended material picking path. Step S330 further includes step S331, according to the through space diameter threshold, the passage area of ​​the robot arm movement constraint area that is less than or equal to the through space diameter threshold is set as a virtual obstacle area. Specifically, the movement constraint area of ​​the robot arm is traversed, and each sub-area therein (which can be a small grid or area division) is checked to determine whether the width or diameter of these sub-areas is less than or equal to the through space diameter threshold of the hardware. If the width or diameter of a sub-area is less than or equal to the through space diameter threshold of the hardware, then this sub-area is regarded as a potential obstacle area, that is, a virtual obstacle area. These areas will be regarded as inaccessible areas in path planning. Step S332, obtain the physical obstacle area of ​​the robot arm movement constraint area. Specifically, the actual obstacle area in the robot arm movement constraint area is obtained, and these obstacle areas include fixed equipment on the workbench, other mechanical parts, hardware that has been placed, etc. The information of the physical obstacle area is obtained through pre-mapping or real-time sensor data. Step S333, based on the virtual obstacle area and the physical obstacle area, with the preset material picking area as the starting point and the material picking target position as the end point, the shortest path planning is performed in the robot arm movement constraint area to obtain the recommended material picking path. Specifically, the preset material picking area is taken as the starting point, the material picking target position is taken as the end point, and the virtual obstacle area and the physical obstacle area are considered at the same time, and the shortest path planning is performed in the robot arm movement constraint area. The path planning algorithm attempts to find a path from the starting point to the end point. This path needs to avoid all obstacle areas (including virtual obstacle areas and physical obstacle areas) and the length is as short as possible. Finally, the system outputs a path that meets the conditions as a recommended material picking path. By setting a virtual obstacle area, this implementation method enables the system to more accurately evaluate the space limitations that the robot arm may encounter during the movement, thereby avoiding collisions or jams due to insufficient space. At the same time, the shortest path planning is carried out by considering the physical obstacle areas and virtual obstacle areas, ensuring that the robot arm can avoid all obstacle areas during movement and reach the target position in the shortest time, thereby improving the safety and efficiency of the robot arm in the material picking process.

[0032] Step S400, performing material picking control according to the hardware positioning identification area and the recommended material picking path. Specifically, performing material picking control according to the hardware positioning identification area and the recommended material picking path includes controlling the robot arm to move to the top of the hardware positioning identification area according to the recommended path, and then accurately grabbing the hardware and moving it to the designated material picking target position. The embodiment of the present application adopts the method of acquiring six views of the preset material picking area through an image collector when the weight sensor of the preset material picking area shows that the weight has increased, obtaining an image of the material picking area, processing the image using image recognition technology, identifying the model and model of the hardware, and determining the positioning identification area of ​​the hardware, optimizing the material picking path according to the identified hardware model result, obtaining the recommended material picking path, and controlling the material picking device to perform material picking operations according to the hardware positioning identification area and the recommended material picking path. The technical means can be applied to a variety of hardware models, and there is no need to configure a material picking device for each model separately, achieving a technical effect of high versatility and improved production efficiency.

[0033] In the above, refer to Figure 1 The present invention describes in detail an intelligent control method for a material taking device for precision hardware according to an embodiment of the present invention. Figure 2 An intelligent control system for a material taking device for precision hardware according to an embodiment of the present invention is described.

[0034] According to an embodiment of the present invention, an intelligent control system for a material taking device for precision hardware is used to solve the technical problems of poor versatility and limited production efficiency improvement in the existing hardware material taking device control, and achieve the technical effect of high versatility and improved production efficiency. An intelligent control system for a material taking device for precision hardware includes: a six-view acquisition module 10, a hardware identification module 20, a material taking path optimization module 30, and a material taking control module 40.

[0035] The six-view acquisition module 10 is used to perform six-view acquisition on the preset material picking area through an image acquisition device to obtain an image of the material picking area when the weight sensor of the preset material picking area shows an increase in weight, wherein the weight sensor is mounted on the surface of the preset material picking area, and the preset material picking area is a transparent area; the hardware recognition module 20 is used to perform hardware recognition on the material picking area image to obtain hardware model recognition results and hardware model recognition results, wherein the hardware model recognition result has a hardware positioning identification area; the material picking path optimization module 30 is used to optimize the material picking path according to the hardware model recognition result to obtain a recommended material picking path; the material picking control module 40 is used to control the material picking according to the hardware positioning identification area and the recommended material picking path.

[0036] The specific configuration of the hardware identification module 20 will be described in detail below. As described above, the hardware identification is performed on the material collection area image to obtain the hardware model identification result and the hardware model identification result. The hardware identification module 20 may further include: an added weight information acquisition unit for communicating with the weight sensor to obtain the added weight information; a first hardware model to be verified acquisition unit for obtaining a first hardware model to be verified that meets the added weight information, wherein the first hardware model to be verified has a plurality of material collection area images to be verified with a plurality of placement postures; an image comparison unit for comparing the material collection area image, traversing the plurality of material collection area images to be verified, and obtaining a plurality of image similarities; a hardware model identification result acquisition unit for setting the first hardware model to be verified as the hardware model identification result when any one of the plurality of image similarities is greater than or equal to the image similarity threshold; a hardware model identification result extraction unit for extracting the hardware model identification result according to the placement postures of the image similarities that are greater than or equal to the image similarity threshold at the same time.

[0037] Among them, for the material picking area image, traverse the several material picking area images to be verified for comparison, and obtain several image similarities, the image comparison unit can further include: a first material picking area image extraction subunit to be verified is used to extract a first material picking area image to be verified from the several material picking area images to be verified, wherein the first material picking area image to be verified has a front view to be verified, a rear view to be verified, a first side view to be verified, a second side view to be verified, a top view to be verified and a bottom view to be verified; a multi-view comparison subunit is used for the material picking area image including the front view of the material picking area, the rear view of the material picking area, the material picking area The first side view of the material picking area, the second side view of the material picking area, the top view of the material picking area and the bottom view of the material picking area are compared with the front view of the material picking area, the rear view of the material picking area, the first side view of the material picking area, the second side view of the material picking area, the top view of the material picking area and the bottom view of the material picking area according to the front view to be verified, the rear view to be verified, the first side view of the material picking area, the second side view of the material picking area, the top view of the material picking area and the bottom view of the material picking area to obtain a first image similarity; the image similarity acquisition subunit is used to add the first image similarity to the several image similarities.

[0038] wherein, according to the front view to be verified, the rear view to be verified, the first side view to be verified, the second side view to be verified, the top view to be verified and the bottom view to be verified, a comparison is performed with the front view of the material picking area, the rear view of the material picking area, the first side view of the material picking area, the second side view of the material picking area, the top view of the material picking area and the bottom view of the material picking area to obtain a first image similarity; the multi-view comparison subunit may further include: each view similarity acquisition micro-unit is used to compare the front view to be verified with the front view of the material picking area to obtain the front view similarity, compare the rear view to be verified with the rear view of the material picking area to obtain the rear view View similarity, comparing the first side view to be verified with the first side view of the material picking area to obtain the similarity of the first side view, comparing the second side view to be verified with the second side view of the material picking area to obtain the similarity of the second side view, comparing the top view to be verified with the top view of the material picking area to obtain the top view similarity, comparing the bottom view to be verified with the top view of the material picking area to obtain the bottom view similarity; the similarity minimum extraction micro unit is used to take the minimum value of the front view similarity, the rear view similarity, the first side view similarity, the second side view similarity, the top view similarity and the bottom view similarity, and set it as the first image similarity.

[0039] Among them, the front view to be verified and the front view of the material collection area are compared to obtain the front view similarity, and each view similarity acquisition micro-unit may further include: an image comparison network acquisition sub-micro-unit is used to obtain an image comparison network, wherein the image comparison network includes a first feature extraction channel, a second feature extraction channel and a feature comparison channel, the model structure and model parameters of the first feature extraction channel and the second feature extraction channel are exactly the same, and the feature comparison channel is used to evaluate the proportion of feature descriptor deviations less than or equal to the deviation threshold; a feature description sub-matrix acquisition sub-micro-unit is used to input the front view to be verified into the first feature extraction channel to obtain a first feature description sub-matrix, and input the front view of the material collection area into the second feature extraction channel to obtain a second feature description sub-matrix; a front view similarity acquisition sub-micro-unit is used to input the first feature description sub-matrix and the second feature description sub-matrix into the feature comparison channel to obtain the front view similarity.

[0040] The specific configuration of the material picking path optimization module 30 will be described in detail below. As described above, the material picking path is optimized according to the hardware model recognition result to obtain the recommended material picking path. The material picking path optimization module 30 may further include: a space diameter threshold extraction unit is used to extract the space diameter threshold according to the hardware model recognition result, wherein the space diameter threshold is the distance between the two farthest points on the hardware model recognition result; a constraint information acquisition unit is used to obtain the material picking target position and the robot arm movement constraint area; a shortest path planning unit is used to take the preset material picking area as the starting point and the material picking target position as the end point, and perform the shortest path planning in the robot arm movement constraint area based on the space diameter threshold to obtain the recommended material picking path.

[0041] Among them, taking the preset material picking area as the starting point and the material picking target position as the end point, the shortest path planning is performed in the robot arm movement constraint area based on the passing space diameter threshold to obtain the recommended material picking path, and the shortest path planning unit may further include: a virtual obstacle area setting subunit is used to set the passage area of ​​the robot arm movement constraint area that is less than or equal to the passing space diameter threshold as a virtual obstacle area according to the passing space diameter threshold; a physical obstacle area acquisition subunit is used to obtain the physical obstacle area of ​​the robot arm movement constraint area; a material picking recommended path acquisition subunit is used to perform the shortest path planning in the robot arm movement constraint area according to the virtual obstacle area and the physical obstacle area, taking the preset material picking area as the starting point and the material picking target position as the end point to obtain the recommended material picking path.

[0042] An intelligent control system for a material picking device for precision hardware provided in an embodiment of the present invention can execute an intelligent control method for a material picking device for precision hardware provided in any embodiment of the present invention, and has functional modules and beneficial effects corresponding to the execution method.

[0043] Although the present application makes various references to certain modules in the system according to the embodiments of the present application, any number of different modules may be used and run on the user terminal and / or server, and the various units and modules included are only divided according to functional logic, but are not limited to the above division, as long as the corresponding functions can be achieved; in addition, the specific names of the functional units are only for the convenience of distinguishing each other, and are not used to limit the scope of protection of the present invention.

[0044] The above specific implementation manner does not constitute a limitation to the protection scope of the present application. It should be understood by those skilled in the art that various modifications, combinations and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions and improvements made within the spirit and principles of the present application should be included in the protection scope of the present application. In some cases, the actions or steps recorded in the present application can be performed in an order different from that in the embodiment and can still achieve the desired results. In addition, the process depicted in the accompanying drawings does not necessarily require the specific order or continuous order shown to achieve the desired results. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

Claims

1. An intelligent control method for a material taking device for precision hardware, characterized in that: include: When the weight sensor of the preset material picking area shows that the weight increases, the preset material picking area is collected by the image collector in six views to obtain the image of the material picking area, wherein the weight sensor is mounted on the surface of the preset material picking area, and the preset material picking area is a transparent area; Performing hardware identification on the material picking area image to obtain a hardware model identification result and a hardware model identification result, wherein the hardware model identification result has a hardware positioning identification area; Optimize the material picking path according to the hardware model recognition result to obtain the recommended material picking path; Material picking control is performed according to the hardware positioning identification area and the recommended material picking path.

2. The intelligent control method for a material taking device for precision hardware according to claim 1, characterized in that: Performing hardware identification on the material picking area image to obtain hardware model identification results and hardware model identification results includes: Communicate with the weight sensor to obtain added weight information; Obtaining a first hardware model to be verified that meets the added weight information, wherein the first hardware model to be verified has a plurality of material collection area images to be verified in a plurality of placement postures; For the material taking area image, traverse the plurality of material taking area images to be verified for comparison to obtain a plurality of image similarities; When any one of the plurality of image similarities is greater than or equal to the image similarity threshold, setting the first hardware model to be verified as the hardware model recognition result; At the same time, according to the placement postures of the images with a similarity greater than or equal to the image similarity threshold, the hardware model recognition result is extracted.

3. The intelligent control method for a material taking device for precision hardware according to claim 2, characterized in that: For the material taking area image, traverse the plurality of material taking area images to be verified for comparison to obtain a plurality of image similarities, including: Extracting a first material picking area image to be checked from the plurality of material picking area images to be checked, wherein the first material picking area image to be checked has a front view to be checked, a rear view to be checked, a first side view to be checked, a second side view to be checked, a top view to be checked, and a bottom view to be checked; The material picking area image includes a front view of the material picking area, a rear view of the material picking area, a first side view of the material picking area, a second side view of the material picking area, a top view of the material picking area, and a bottom view of the material picking area; According to the front view to be verified, the rear view to be verified, the first side view to be verified, the second side view to be verified, the top view to be verified and the bottom view to be verified, the front view of the material picking area, the rear view of the material picking area, the first side view of the material picking area, the second side view of the material picking area, the top view of the material picking area and the bottom view of the material picking area are compared to obtain a first image similarity; The first image similarity is added to the plurality of image similarities.

4. The intelligent control method for a material taking device for precision hardware according to claim 3, characterized in that: The first image similarity is obtained by comparing the front view to be verified, the rear view to be verified, the first side view to be verified, the second side view to be verified, the top view to be verified and the bottom view to be verified with the front view of the material picking area, the rear view of the material picking area, the first side view of the material picking area, the second side view of the material picking area, the top view of the material picking area and the bottom view of the material picking area, including: Comparing the front view to be verified with the front view of the material taking area to obtain the similarity of the front views; Comparing the rear view to be verified with the rear view of the material taking area to obtain the rear view similarity; Comparing the first side view to be verified with the first side view of the material taking area to obtain the similarity of the first side views; Comparing the second side view to be verified with the second side view of the material taking area to obtain the similarity of the second side views; Comparing the top view to be verified with the top view of the material taking area to obtain the similarity of the top views; Comparing the bottom view to be verified with the bottom view of the material taking area to obtain the similarity of the bottom views; The minimum value of the front view similarity, the rear view similarity, the first side view similarity, the second side view similarity, the top view similarity and the bottom view similarity is taken as the first image similarity.

5. The intelligent control method for a material taking device for precision hardware according to claim 4, characterized in that: Comparing the front view to be verified with the front view of the material taking area to obtain the similarity of the front views, including: Obtaining an image comparison network, wherein the image comparison network includes a first feature extraction channel, a second feature extraction channel, and a feature comparison channel, the model structure and model parameters of the first feature extraction channel and the second feature extraction channel are completely the same, and the feature comparison channel is used to evaluate the proportion of feature descriptor deviations that are less than or equal to a deviation threshold; Inputting the front view to be verified into the first feature extraction channel to obtain a first feature descriptor matrix; Inputting the front view of the material extraction area into the second feature extraction channel to obtain a second feature descriptor matrix; The first feature description sub-matrix and the second feature description sub-matrix are input into the feature comparison channel to obtain the front view similarity.

6. The intelligent control method for a material taking device for precision hardware according to claim 1, characterized in that: The material picking path is optimized according to the hardware model recognition result to obtain the recommended material picking path, including: Extracting a passing space diameter threshold according to the hardware model recognition result, wherein the passing space diameter threshold is the distance between the two farthest points on the hardware model recognition result; Obtain the material picking target position and the robot arm movement constraint area; Taking the preset material picking area as the starting point and the material picking target position as the end point, the shortest path planning is performed in the robot arm movement constraint area based on the passing space diameter threshold to obtain the recommended material picking path.

7. The intelligent control method for a material taking device for precision hardware according to claim 6, characterized in that: Taking the preset material picking area as the starting point and the material picking target position as the end point, the shortest path planning is performed in the robot arm movement constraint area based on the passing space diameter threshold to obtain the material picking recommended path, including: According to the passing space diameter threshold, setting the passing area where the robot arm movement constraint area is less than or equal to the passing space diameter threshold as a virtual obstacle area; Obtaining a physical obstacle area of ​​the robot arm movement constraint area; According to the virtual obstacle area and the physical obstacle area, taking the preset material picking area as the starting point and the material picking target position as the end point, the shortest path planning is performed in the robot arm movement constraint area to obtain the recommended material picking path.

8. An intelligent control system for a material taking device for precision hardware, characterized in that: The system is used to implement the intelligent control method of a material taking device for precision hardware according to any one of claims 1 to 7, and the system comprises: A six-view acquisition module, wherein when a weight sensor of a preset material picking area indicates that the weight has increased, the six-view acquisition module acquires six views of the preset material picking area through an image acquisition device to obtain an image of the material picking area, wherein the weight sensor is mounted on the surface of the preset material picking area, and the preset material picking area is a transparent area; A hardware identification module, which is used to identify hardware on the material picking area image, and obtain a hardware model identification result and a hardware model identification result, wherein the hardware model identification result has a hardware positioning identification area; A material picking path optimization module, which is used to optimize the material picking path according to the hardware model recognition result to obtain a recommended material picking path; A material fetching control module is used to control material fetching according to the hardware positioning identification area and the recommended material fetching path.

Images