Mechanical arm grabbing method based on deep neural network

Abstract

The invention discloses a mechanical arm grabbing method based on a deep neural network. The mechanical arm grabbing method comprises the steps that an image of an object to be grabbed is acquired; inputting the image of the to-be-grabbed article into a pre-trained deep neural network recognition model for recognition, and determining grabbing information corresponding to the to-be-grabbed article; controlling a mechanical arm to grab a to-be-grabbed object based on the grabbing information; through the strong recognition capability of the deep neural network, the grabbing information can be determined more accurately, and the grabbing success rate and precision are improved; high-degree intellectualization is realized, and dependence on artificial experience and setting is reduced; the mechanical arm can adapt to objects to be grabbed of various shapes, sizes and characteristics, and the application range of the mechanical arm is expanded.

Description

Technical Field

[0001] The present invention relates to the field of automation technology, and in particular to a robotic arm grasping method based on a deep neural network. Background Art

[0002] Deep neural networks currently hold a promising future and are widely used in predictive maintenance, quality control, and production optimization. Robotic arms are increasingly replacing human operators in complex tasks such as intelligent sorting, handling, and loading and unloading. These tasks require the ability to grasp objects in complex stacking scenarios.

[0003] The following problems exist in the application of robotic arms in the existing technology: (1) During the industrial production process, when the robotic arm performs the task of grasping objects that are piled up in a disorderly manner, the judgment of the object posture is not accurate enough, resulting in low efficiency in the execution of the grasping task; (2) When performing the grasping task, a fixed grasping path is often adopted. When the position of the grasped object changes, the grasping path cannot be adjusted in time; (3) When grasping objects, the same grasping mode is adopted for the same type of objects. When the type of objects changes, the grasping mode for different objects cannot be adjusted in time, resulting in damage to the grasped objects.

[0004] Therefore, a robotic arm grasping method based on deep neural networks is urgently needed to solve the above problems. Summary of the Invention

[0005] The present invention aims to at least partially address one of the technical problems in the aforementioned technologies. To this end, the present invention proposes a robotic arm grasping method based on a deep neural network. This method utilizes the deep neural network to more accurately identify the grasping position and path of an object, reducing grasping errors. It can adapt to grasping objects of different shapes, sizes, and types, thereby improving the versatility of the robotic arm.

[0006] To achieve the above objectives, an embodiment of the present invention proposes a robotic arm grasping method based on a deep neural network, comprising:

[0007] Get the image of the object to be grasped;

[0008] Input the image of the object to be grasped into the pre-trained deep neural network recognition model for recognition, and determine the grasping information corresponding to the object to be grasped;

[0009] Control the robotic arm to grasp the object based on the grasping information

[0010] Preferably, the method for constructing a deep neural network recognition model includes:

[0011] Obtain a training set of images of grasped objects;

[0012] Inputting the grasped object image training set into a deep neural network model for iterative training to obtain an initial deep neural network recognition model;

[0013] Get a test set of grasped object images;

[0014] The initial deep neural network recognition model is tested based on the grasped object image test set, and when the test result is determined to be qualified, a trained deep neural network recognition model is obtained.

[0015] Preferably, the grasping information includes grasping posture and grasping path;

[0016] Before controlling the robotic arm to grasp the object based on the grasping information, the method further includes:

[0017] Determine the distribution of obstacles within a preset range around the robotic arm based on the image of the object to be grasped and the grasping path;

[0018] Get the preset mobile crawling simulation space;

[0019] Mapping the robotic arm, grasping posture, grasping path and obstacle distribution into a preset mobile grasping simulation space;

[0020] Based on the simulation of the process of the robotic arm grasping the object to be grasped in the preset mobile grasping simulation space, it is determined whether the robotic arm collides with an obstacle when grasping the object to be grasped;

[0021] If a collision occurs, any obstacle that collides with the robotic arm is selected as the target collision obstacle;

[0022] Obtaining a first target collision point and a second target collision point where the robotic arm collides with a target collision obstacle;

[0023] determining a revised grasping path based on the first target collision point and the second target collision point;

[0024] Traverse all target collision obstacles and obtain several corrected grasping paths;

[0025] The grasping path is revised based on a plurality of revised grasping paths.

[0026] Preferably, determining the distribution of obstacles within a preset range around the robotic arm based on the image of the object to be grasped and the grasping path includes:

[0027] Based on the image of the object to be grasped and the grasping path, the image of the object within a preset range around the robotic arm when the robotic arm grasps through the grasping path is extracted to obtain multiple object images;

[0028] querying a preset obstacle database based on the plurality of object images to determine whether the objects are obstacles;

[0029] If the object is an obstacle, the object is used as a target collision obstacle to obtain a plurality of target collision obstacles;

[0030] Based on the plurality of target collision obstacles, the distribution of obstacles within a preset range around the robotic arm when the robotic arm grasps through the grasping path is determined.

[0031] Preferably, the first target collision point is a collision position point of a target collision obstacle; and the second target collision point is a collision position point of a robotic arm.

[0032] Preferably, determining a revised grasping path based on the first target collision point and the second target collision point includes:

[0033] Obtaining an edge contour of a target collision obstacle on a plane where a first target collision point is located;

[0034] determining a maximum distance point between the first target collision point and the edge contour of the target collision obstacle based on the first target collision point and the edge contour of the target collision obstacle, and obtaining a first corrected position point;

[0035] Obtain the distance between the first collision point and the first corrected position point as the target distance;

[0036] Determining a second corrected position point in the grasping path based on the second target collision point and the target distance; the second corrected position point is a position point in the grasping path that is the target distance from the second target collision point;

[0037] A connection line between the first corrected position point and the second corrected position point is obtained to determine a corrected grasping path.

[0038] Preferably, after controlling the robotic arm to grasp the object based on the grasping information, the method further includes:

[0039] Get the image of the object after grasping;

[0040] The similarity between the image of the object after grasping and the image of the object before grasping is calculated, and when it is determined that the similarity is less than a preset similarity threshold, an early warning prompt is issued.

[0041] Preferably, before inputting the image of the object to be grasped into the pre-trained deep neural network recognition model for recognition, the method further includes:

[0042] Performing image enhancement on the image of the object to be grasped to obtain an enhanced image of the object to be grasped;

[0043] The enhanced image of the object to be grasped is input into the pre-trained deep neural network recognition model for recognition.

[0044] Preferably, performing image enhancement on the image of the object to be grasped to obtain an enhanced image of the object to be grasped includes:

[0045] Take any image of the object to be grasped and convert it into grayscale to obtain a grayscale image;

[0046] Evenly divide the grayscale image into several grayscale sub-images;

[0047] Take any grayscale sub-image as the image to be processed;

[0048] Get the grayscale value corresponding to each pixel in the image to be processed;

[0049] Randomly select a pixel point in the image to be processed as the target pixel point;

[0050] Determine the target area with the target pixel as the center and the preset distance as the radius;

[0051] Obtain the grayscale mean of the pixels in the target area to obtain the target grayscale mean;

[0052] Comparing the grayscale value of each pixel in the target area with the target grayscale mean; counting the number of pixels whose grayscale values ​​are greater than or equal to the target grayscale mean to obtain a first number; and counting the number of pixels whose grayscale values ​​are less than the target grayscale mean to obtain a second number;

[0053] The ratio of the first number to the second number is used as the contrast value of the target pixel;

[0054] Traverse all the pixels in the image to be processed and obtain the contrast values ​​corresponding to several pixels;

[0055] Comparing the contrast values ​​corresponding to the plurality of pixel points with a preset contrast threshold, and selecting pixel points whose contrast values ​​are less than or equal to the preset contrast threshold as pixel points to be enhanced, thereby obtaining a plurality of pixel points to be enhanced;

[0056] Enhance the grayscale values ​​of the plurality of pixels to be enhanced based on an enhancement algorithm to obtain an enhanced grayscale sub-image;

[0057] Traverse all grayscale sub-images to obtain the enhanced image of the object to be grasped.

[0058] Preferably, the enhancement algorithm includes:

[0059]

[0060] Among them, Hi H represents the gray value of the i-th pixel point to be enhanced after enhancement; H i H represents the gray value of the i-th pixel point to be enhanced before enhancement; H max H represents the maximum gray value of the pixel points in the target region corresponding to the i-th pixel point to be enhanced; f represents an error factor, and the value is [0.8, 1.5].

[0061] The application discloses a mechanical arm grabbing method based on a deep neural network, which can more accurately determine the grabbing information and improve the success rate and precision of grabbing through the powerful recognition capability of the deep neural network, realizes high degree of intelligence, reduces the dependence on artificial experience and setting, can cope with various shapes, sizes and characteristics of the to-be-grabbed articles, expands the application range of the mechanical arm, adjusts the grabbing path and grabbing mode of the mechanical arm in time through the powerful recognition capability of the deep neural network, improves the grabbing efficiency, and avoids damage to the to-be-grabbed articles.

[0062] Other features and advantages of the present application will be set forth in the following description, and in part will become apparent to those skilled in the art from the description, or can be learned by practice of the application. The objects and other advantages of the present application can be achieved and obtained by the structure particularly pointed out in the written description and the accompanying drawings.

[0063] The technical solutions of the present application will be further described in detail below with the help of the accompanying drawings and examples. BRIEF DESCRIPTION OF DRAWINGS

[0064] The accompanying drawings are used to provide a further understanding of the present application, and constitute a part of the specification, together with the embodiments of the present application, used to explain the present application, and do not constitute a limitation on the present application. In the drawings:

[0065] Figure 1 is a flow chart of a mechanical arm grabbing method based on a deep neural network according to an embodiment of the present application;

[0066] Figure 2 is a flow chart of a construction method of a deep neural network recognition model according to an embodiment of the present application;

[0067] Figure 3 is a flow chart of pre-warning after grabbing according to an embodiment of the present application. DETAILED DESCRIPTION

[0068] The preferred embodiments of the present application are described below in combination with the accompanying drawings, and it should be understood that the preferred embodiments described herein are only used to illustrate and explain the present application, and do not limit the present application.

[0069] Example 1

[0070] As Figure 1As shown, a robotic arm grasping method based on a deep neural network includes steps S1-S3:

[0071] S1: Get the image of the object to be grasped;

[0072] S2: Input the image of the object to be grasped into the pre-trained deep neural network recognition model for recognition, and determine the grasping information corresponding to the object to be grasped;

[0073] S3: Control the robotic arm to grab the object based on the grabbing information.

[0074] In this embodiment, the method of acquiring the image of the object to be grasped includes acquiring the image of the object to be grasped based on an image acquisition device provided on the robotic arm.

[0075] In this embodiment, the grasping information includes but is not limited to grasping posture, grasping path and grasping mode.

[0076] In this embodiment, the deep neural network can identify the shape, size, texture and other features of the object by learning a large number of object images and their related grasping features; query the common grasping methods corresponding to the object through the shape, size, texture and other features of the object; obtain historical grasping environment images corresponding to the common grasping methods; the historical grasping environment images include the postures corresponding to the grasped objects; calculate the similarity between the image of the object to be grasped and the historical grasping environment images, and determine the posture corresponding to the historical grasping environment image with the highest similarity as the grasping posture; query the preset grasping path information table based on the shape, size, texture and other features of the object to determine the grasping path corresponding to the object to be grasped; and use the grasping posture and grasping path as the grasping information corresponding to the object to be grasped.

[0077] The beneficial effects of the above technical solution are: through the powerful recognition ability of deep neural networks, it can more accurately determine the grasping information, improve the success rate and accuracy of grasping; achieve a high degree of intelligence, reduce dependence on manual experience and settings; can cope with objects to be grasped of various shapes, sizes and characteristics, and expand the application range of the robotic arm; through the powerful recognition ability of deep neural networks, the grasping path and grasping mode of the robotic arm can be adjusted in time to improve the grasping efficiency and avoid damage to the grasped objects.

[0078] Example 2

[0079] like Figure 2 As shown, the construction method of the deep neural network recognition model includes S21-S24:

[0080] S21: Obtain a training set of images of grasped objects;

[0081] S22: Inputting the grasped object image training set into a deep neural network model for iterative training to obtain an initial deep neural network recognition model;

[0082] S23: Obtain a test set of grasped object images;

[0083] S24: Testing the initial deep neural network recognition model based on the grasped object image test set, and obtaining a trained deep neural network recognition model when the test result is determined to be qualified.

[0084] In this embodiment, the captured object image training set is input into a deep neural network model for iterative training. When the success rate of successfully identifying objects in the training results is greater than or equal to a training success rate threshold, an initial deep neural network recognition model is obtained.

[0085] In this embodiment, the test results of the initial deep neural network recognition model are obtained, the test results of the initial deep neural network recognition model are compared with the manual annotation results, and the accuracy of the test results is determined. When it is determined that the accuracy is greater than or equal to a preset accuracy threshold, the test results are determined to be qualified, and a trained deep neural network recognition model is obtained.

[0086] The beneficial effects of the above technical solution are: iterative training through a special training set of images of grasped objects can effectively improve the performance and accuracy of the model; testing using a test set can ensure the reliability and accuracy of the model in practical applications; targeted training can be carried out according to different types of grasped objects to adapt to the needs of various scenarios; it helps to continuously adjust and optimize the parameters of the deep neural network model to improve its recognition effect; based on a large amount of training and test data, it can continuously improve the performance of the model; after sufficient training and testing, the model has better generalization capabilities to cope with new grasping tasks.

[0087] Example 3

[0088] The grasping information includes grasping posture and grasping path;

[0089] Before controlling the robotic arm to grasp the object based on the grasping information, the method further includes:

[0090] Determine the distribution of obstacles within a preset range around the robotic arm based on the image of the object to be grasped and the grasping path;

[0091] Get the preset mobile crawling simulation space;

[0092] Mapping the robotic arm, grasping posture, grasping path and obstacle distribution into a preset mobile grasping simulation space;

[0093] Based on the simulation of the process of the robotic arm grasping the object to be grasped in the preset mobile grasping simulation space, it is determined whether the robotic arm collides with an obstacle when grasping the object to be grasped;

[0094] If a collision occurs, any obstacle that collides with the robotic arm is selected as the target collision obstacle;

[0095] Obtaining a first target collision point and a second target collision point where the robotic arm collides with a target collision obstacle;

[0096] determining a revised grasping path based on the first target collision point and the second target collision point;

[0097] Traverse all target collision obstacles and obtain several corrected grasping paths;

[0098] The grasping path is revised based on a plurality of revised grasping paths.

[0099] The beneficial effects of the above technical solution are: by simulating and detecting collisions, the robot arm can avoid accidental collisions with obstacles during actual grasping, thereby ensuring operational safety; it can continuously correct the grasping path to make it more reasonable and efficient, reducing unnecessary actions and time waste; it can effectively reduce the risk of damage to the robot arm or objects due to collisions, and reduce maintenance costs; the grasping path after multiple corrections can improve the reliability and stability of the robot arm's grasping action; it embodies intelligent path planning capabilities and enhances the intelligence level of the entire system; it can flexibly adjust and optimize for complex environments and multiple obstacles.

[0100] Example 4

[0101] Determine the distribution of obstacles within a preset range around the robotic arm based on the image of the object to be grasped and the grasping path, including:

[0102] Based on the image of the object to be grasped and the grasping path, the image of the object within a preset range around the robotic arm when the robotic arm grasps through the grasping path is extracted to obtain multiple object images;

[0103] querying a preset obstacle database based on the plurality of object images to determine whether the objects are obstacles;

[0104] If the object is an obstacle, the object is used as a target collision obstacle to obtain a plurality of target collision obstacles;

[0105] Based on the plurality of target collision obstacles, the distribution of obstacles within a preset range around the robotic arm when the robotic arm grasps through the grasping path is determined.

[0106] In this embodiment, the preset obstacle database is a database of types of obstacles in the working environment of the robotic arm set in advance.

[0107] The beneficial effects of the above technical solution are: it can accurately extract the image of the object from the range related to the image of the object to be grasped and the grasping path, thereby improving the ability to accurately identify obstacles; by querying the preset obstacle database, it can quickly determine whether the object is an obstacle, thereby improving the judgment efficiency; it can comprehensively consider the possible obstacles within the preset range around the robotic arm, and ensure the safety and feasibility of the grasping process; it is applicable to a variety of complex scenarios, and enhances the system's adaptability to the distribution of obstacles in different environments; clarifying the obstacle distribution in advance can effectively reduce the risk of collision with obstacles during the grasping process; it provides more reliable protection for the grasping action of the robotic arm, and reduces accidents caused by unaware obstacles.

[0108] Example 5

[0109] The first target collision point is a collision position point of the target collision obstacle; the second target collision point is a collision position point of the robotic arm.

[0110] Example 6

[0111] Determining a corrected grasping path based on the first target collision point and the second target collision point includes:

[0112] Obtaining an edge contour of a target collision obstacle on a plane where a first target collision point is located;

[0113] determining a maximum distance point between the first target collision point and the edge contour of the target collision obstacle based on the first target collision point and the edge contour of the target collision obstacle, and obtaining a first corrected position point;

[0114] Obtain the distance between the first collision point and the first corrected position point as the target distance;

[0115] Determining a second corrected position point in the grasping path based on the second target collision point and the target distance; the second corrected position point is a position point in the grasping path that is the target distance from the second target collision point;

[0116] A connection line between the first corrected position point and the second corrected position point is obtained to determine a corrected grasping path.

[0117] In this embodiment, the first target collision point and the second target collision point are in one-to-one correspondence and appear in pairs; the first target collision point is on the target collision obstacle, and the second target collision point is on the robotic arm.

[0118] In this embodiment, the edge profile of the target collision obstacle in the plane where the first target collision point is located is obtained, that is, the first target collision point is on a face of the target collision obstacle, and this face is the face where the robot collides with the target collision obstacle, and the edge profile of this face is obtained.

[0119] In this embodiment, the first modified position point is the point with the maximum distance between the first target collision point and the edge profile of the target collision obstacle.

[0120] In this embodiment, the second modified position point is determined in the grabbing path with the second target collision point as the starting point and the target distance as the step size, so that the distance between the second modified position point and the second target collision point is the target distance.

[0121] The beneficial effects of the above technical solutions are: through specific analysis of the target collision point and related positions, the modified grabbing path can be accurately determined to improve the accuracy of the modification; specific collision points and obstacle profiles are operated to make the modification more targeted and effectively avoid specific obstacles; the modified grabbing path is ensured to be reasonable, unnecessary detours or deviations are reduced, and the grabbing efficiency is improved; the modified position point and the connecting line can be flexibly determined according to different collision conditions to adapt to various complex obstacle scenes; the interference of specific obstacles on the grabbing path is effectively reduced to ensure the smooth performance of the grabbing action; the safety of the grabbing process is enhanced, and the risk and loss caused by collision are reduced.

[0122] Embodiment 7

[0123] As shown in Figure 3 based on the grabbing information, the robot controls the robot to grab the to-be-grabbed object, and further comprises S31-S32:

[0124] S31: obtaining an image of the object after completing grabbing;

[0125] S32: performing similarity calculation on the image of the object after completing grabbing and the image of the object before grabbing, and issuing a warning prompt when it is determined that the similarity is less than a preset similarity threshold.

[0126] In this embodiment, the calculation method of the similarity includes but is not limited to cosine similarity.

[0127] In this embodiment, the preset similarity threshold is set in advance based on industry experience.

[0128] The beneficial effects of the above technical solution are: it can promptly detect whether there are any abnormalities in the object after grasping, which plays an effective role in monitoring the grasping quality; ensure that the grasped object is consistent with the expectation through similarity comparison, and ensure the accuracy of grasping; when the similarity does not meet the requirements, it will issue an early warning in time to remind relevant personnel to pay attention to possible problems; it helps to avoid problems in subsequent processes due to incorrect grasping or damage to the object during the grasping process that goes unnoticed.

[0129] Example 8

[0130] Before the image of the object to be grasped is input into the pre-trained deep neural network recognition model for recognition, it also includes:

[0131] Performing image enhancement on the image of the object to be grasped to obtain an enhanced image of the object to be grasped;

[0132] The enhanced image of the object to be grasped is input into the pre-trained deep neural network recognition model for recognition.

[0133] The beneficial effects of the above technical solution are: image enhancement can improve the clarity and contrast of the image, making the features of the object more obvious, which helps to improve the recognition accuracy; reduce the influence of adverse factors such as noise and blur that may exist in the image, and improve the reliability of recognition; enable the deep neural network to better extract the key features of the object, so as to identify it more accurately; make the model more adaptable to images under different lighting and shooting conditions; and overall help to improve the performance and effect of the deep neural network recognition model.

[0134] Example 9

[0135] Performing image enhancement on the image of the object to be grasped to obtain an enhanced image of the object to be grasped, including:

[0136] Take any image of the object to be grasped and convert it into grayscale to obtain a grayscale image;

[0137] Evenly divide the grayscale image into several grayscale sub-images;

[0138] Take any grayscale sub-image as the image to be processed;

[0139] Get the grayscale value corresponding to each pixel in the image to be processed;

[0140] Randomly select a pixel point in the image to be processed as the target pixel point;

[0141] Determine the target area with the target pixel as the center and the preset distance as the radius;

[0142] Obtain the grayscale mean of the pixels in the target area to obtain the target grayscale mean;

[0143] Comparing the grayscale value of each pixel in the target area with the target grayscale mean; counting the number of pixels whose grayscale values ​​are greater than or equal to the target grayscale mean to obtain a first number; and counting the number of pixels whose grayscale values ​​are less than the target grayscale mean to obtain a second number;

[0144] The ratio of the first number to the second number is used as the contrast value of the target pixel;

[0145] Traverse all the pixels in the image to be processed and obtain the contrast values ​​corresponding to several pixels;

[0146] Comparing the contrast values ​​corresponding to the plurality of pixel points with a preset contrast threshold, and selecting pixel points whose contrast values ​​are less than or equal to the preset contrast threshold as pixel points to be enhanced, thereby obtaining a plurality of pixel points to be enhanced;

[0147] Enhance the grayscale values ​​of the plurality of pixels to be enhanced based on an enhancement algorithm to obtain an enhanced grayscale sub-image;

[0148] Traverse all grayscale sub-images to obtain the enhanced image of the object to be grasped.

[0149] In this embodiment, the contrast threshold is set based on industry experience.

[0150] The working principle of the above technical solution is: segmenting the image of the object to be grasped to obtain several grayscale sub-images, and calculating the contrast value of each pixel in the grayscale sub-image in the target area in units of pixels; screening each pixel based on the contrast threshold, and taking the pixel points when the contrast value corresponding to the pixel point is less than or equal to the preset contrast threshold as the pixel points to be enhanced, to obtain several pixel points to be enhanced; relatively low contrast will affect the object contour, texture and other details in the image, resulting in visual blur, and cannot well reflect the hierarchical relationship between different areas in the image; therefore, the pixel points less than or equal to the preset contrast threshold are taken as the pixel points to be enhanced; and the several pixel points to be enhanced are enhanced based on the enhancement algorithm.

[0151] The beneficial effects of the above technical solution are: through the analysis and enhancement of contrast values, the details of the object image can be highlighted, which is conducive to more accurate identification; processing each grayscale sub-image can better improve the local image features and enhance the targeted recognition; adaptive enhancement is performed on areas with different grayscale distributions, so that the model can cope with various complex object image situations; the enhanced image can improve the recognition of the object to be grasped and reduce misjudgment; it helps the deep neural network to better extract key features, thereby improving the performance of the recognition model; the model has stronger robustness when facing image inputs of different qualities; the overall quality of the image of the object to be grasped is comprehensively improved, providing a better foundation for subsequent accurate grasping.

[0152] Example 10

[0153] The enhancement algorithm includes:

[0154]

[0155] Among them, H i ' represents the gray value of the i-th pixel to be enhanced after enhancement; h i H represents the gray value of the i-th pixel to be enhanced before enhancement; max represents the maximum grayscale value of the pixel in the target area corresponding to the i-th pixel to be enhanced; f represents the error factor, which takes a value of [0.8, 1.5].

[0156] In this embodiment, the value of f is related to the radius of the target area corresponding to the pixel to be enhanced. The larger the radius, the smaller the value of f.

[0157] The beneficial effects of the above technical solution are: using logarithmic functions and the like for calculation to achieve nonlinear grayscale value adjustment, which can enhance the image more finely, highlight details and avoid distortion caused by excessive enhancement; adaptive calculation based on parameters such as the maximum grayscale value of the target area can better adapt to the enhancement needs of different local features; through the value range of the error factor, the degree and effect of the enhancement can be flexibly controlled to a certain extent; it helps to improve the overall contrast of the image, making the features of the object to be grasped more distinct, which is conducive to subsequent identification and processing; while enhancing, it can better retain the key features of the object and reduce damage to the features.

[0158] Obviously, those skilled in the art may make various changes and modifications to the present invention without departing from the spirit and scope of the present invention. Thus, if such changes and modifications fall within the scope of the claims and their equivalents, the present invention is intended to include such changes and modifications.

Claims

1. A robotic arm grasping method based on deep neural network, characterized in that: include: Get the image of the object to be grasped; Input the image of the object to be grasped into the pre-trained deep neural network recognition model for recognition, and determine the grasping information corresponding to the object to be grasped; The robotic arm is controlled to grasp the object based on the grasping information.

2. The robotic arm grasping method based on deep neural network according to claim 1, characterized in that: The method for constructing a deep neural network recognition model includes: Obtain a training set of images of grasped objects; Inputting the grasped object image training set into a deep neural network model for iterative training to obtain an initial deep neural network recognition model; Get a test set of grasped object images; The initial deep neural network recognition model is tested based on the grasped object image test set, and when the test result is determined to be qualified, a trained deep neural network recognition model is obtained.

3. The robotic arm grasping method based on deep neural network according to claim 1, characterized in that: The grasping information includes grasping posture and grasping path; Before controlling the robotic arm to grasp the object based on the grasping information, the method further includes: Determine the distribution of obstacles within a preset range around the robotic arm based on the image of the object to be grasped and the grasping path; Get the preset mobile crawling simulation space; Mapping the robotic arm, grasping posture, grasping path and obstacle distribution into a preset mobile grasping simulation space; Based on the simulation of the process of the robotic arm grasping the object to be grasped in the preset mobile grasping simulation space, it is determined whether the robotic arm collides with an obstacle when grasping the object to be grasped; If a collision occurs, any obstacle that collides with the robotic arm is selected as the target collision obstacle; Obtaining a first target collision point and a second target collision point where the robotic arm collides with a target collision obstacle; determining a revised grasping path based on the first target collision point and the second target collision point; Traverse all target collision obstacles and obtain several corrected grasping paths; The grasping path is revised based on a plurality of revised grasping paths.

4. The robotic arm grasping method based on deep neural network according to claim 3, characterized in that: Determine the distribution of obstacles within a preset range around the robotic arm based on the image of the object to be grasped and the grasping path, including: Based on the image of the object to be grasped and the grasping path, the image of the object within a preset range around the robotic arm when the robotic arm grasps through the grasping path is extracted to obtain multiple object images; querying a preset obstacle database based on the plurality of object images to determine whether the objects are obstacles; If the object is an obstacle, the object is used as a target collision obstacle to obtain a plurality of target collision obstacles; Based on the plurality of target collision obstacles, the distribution of obstacles within a preset range around the robotic arm when the robotic arm grasps through the grasping path is determined.

5. The robotic arm grasping method based on deep neural network according to claim 4, characterized in that: The first target collision point is a collision position point of the target collision obstacle; the second target collision point is a collision position point of the robotic arm.

6. The robotic arm grasping method based on deep neural network according to claim 5, characterized in that: Determining a corrected grasping path based on the first target collision point and the second target collision point includes: Obtaining an edge contour of a target collision obstacle on a plane where a first target collision point is located; determining a maximum distance point between the first target collision point and the edge contour of the target collision obstacle based on the first target collision point and the edge contour of the target collision obstacle, and obtaining a first corrected position point; Obtain the distance between the first collision point and the first corrected position point as the target distance; Determining a second corrected position point in the grasping path based on the second target collision point and the target distance; the second corrected position point is a position point in the grasping path that is the target distance from the second target collision point; A connection line between the first corrected position point and the second corrected position point is obtained to determine a corrected grasping path.

7. The robotic arm grasping method based on deep neural network according to claim 3, characterized in that: After controlling the robotic arm to grasp the object based on the grasping information, the method further includes: Get the image of the object after grasping; The similarity between the image of the object after grasping and the image of the object before grasping is calculated, and when it is determined that the similarity is less than a preset similarity threshold, an early warning prompt is issued.

8. The robotic arm grasping method based on deep neural network according to claim 1, characterized in that: Before the image of the object to be grasped is input into the pre-trained deep neural network recognition model for recognition, it also includes: Performing image enhancement on the image of the object to be grasped to obtain an enhanced image of the object to be grasped; The enhanced image of the object to be grasped is input into the pre-trained deep neural network recognition model for recognition.

9. The robotic arm grasping method based on deep neural network according to claim 8, characterized in that: Performing image enhancement on the image of the object to be grasped to obtain an enhanced image of the object to be grasped, including: Take any image of the object to be grasped and convert it into grayscale to obtain a grayscale image; Evenly divide the grayscale image into several grayscale sub-images; Take any grayscale sub-image as the image to be processed; Get the grayscale value corresponding to each pixel in the image to be processed; Randomly select a pixel point in the image to be processed as the target pixel point; Determine the target area with the target pixel as the center and the preset distance as the radius; Obtain the grayscale mean of the pixels in the target area to obtain the target grayscale mean; Comparing the grayscale value of each pixel in the target area with the target grayscale mean; counting the number of pixels whose grayscale values ​​are greater than or equal to the target grayscale mean to obtain a first number; and counting the number of pixels whose grayscale values ​​are less than the target grayscale mean to obtain a second number; The ratio of the first number to the second number is used as the contrast value of the target pixel; Traverse all the pixels in the image to be processed and obtain the contrast values ​​corresponding to several pixels; Comparing the contrast values ​​corresponding to the plurality of pixel points with a preset contrast threshold, and selecting pixel points whose contrast values ​​are less than or equal to the preset contrast threshold as pixel points to be enhanced, thereby obtaining a plurality of pixel points to be enhanced; Enhance the grayscale values ​​of the plurality of pixels to be enhanced based on an enhancement algorithm to obtain an enhanced grayscale sub-image; Traverse all grayscale sub-images to obtain the enhanced image of the object to be grasped.

10. The robotic arm grasping method based on deep neural network according to claim 9, characterized in that: The enhancement algorithm includes: Among them, H i' H represents the gray value of the i-th pixel to be enhanced after enhancement; i H represents the gray value of the i-th pixel to be enhanced before enhancement; max represents the maximum grayscale value of the pixel in the target area corresponding to the i-th pixel to be enhanced; f represents the error factor, which takes a value of [0.8, 1.5].

Images