A part pose determination method for ship small assembly part polishing equipment

Abstract

The application provides a part pose determination method for ship small assembly part polishing equipment, comprising: acquiring contour information of a part through a visual sensor, grabbing the part, and moving to an initial position; inputting the contour information of the part into a trained deep learning model to predict the part position and generate a candidate pose; acquiring the candidate pose, comparing it with a set constraint condition, and selecting a candidate pose satisfying the constraint condition as a final pose; and acquiring the final pose, adjusting the pose of the part, and polishing it. The application acquires the contour information of the part through a visual sensor, determines the placement pose of the part using a deep learning algorithm after grabbing the part, and automatically polishes the free edge using a polishing tool, thereby solving the problems of low precision, low efficiency and complex constraint conditions in the prior art.

Description

Technical Field

[0001] This application relates to the field of shipbuilding, and more specifically, to a method for determining the position and orientation of parts in a grinding equipment for ship assembly parts. Background Technology

[0002] Currently, the grinding process for ship assembly parts mainly relies on manual operation, which presents the following problems: Low precision: Manual measurement and positioning of parts' positions and orientations are difficult to guarantee accuracy, leading to inconsistent grinding quality. Low efficiency: Manual operation is time-consuming and requires a large workforce, making it difficult to meet the needs of large-scale production. Complex constraints: Grinding the free edges of parts requires satisfying both inward and outward constraints to avoid interference with magnetic nail supports, while also ensuring the stability of the parts on the magnetic nail supports. Insufficient automation: Although automated equipment has been gradually applied in the shipbuilding industry, efficient and precise solutions are still lacking in the process of determining the orientation of parts and grinding.

[0003] In the prior art, patent application CN202310335046.8 discloses a method, system, and device for rapid visual matching and positioning of workpieces. Given a workpiece drawing for processes such as grinding, it can accurately match and position the workpiece, enabling the tool head on a robotic arm to automatically process the workpiece, saving labor costs and improving efficiency. However, by focusing on the visual positioning of the workpiece, it cannot solve the problems of determining the orientation of the parts and handling complex constraints. Summary of the Invention

[0004] In view of the deficiencies in the prior art, the purpose of this application is to provide a part placement attitude calculation algorithm for an automated grinding device for ship assembly parts.

[0005] The first aspect of this application provides a method for determining the pose of parts in a grinding equipment for ship assembly parts, comprising:

[0006] The contour information of the part is obtained through a vision sensor, the part is grasped, and moved to the initial position;

[0007] The contour information of the part is input into a trained deep learning model to predict the position of the part and generate candidate poses.

[0008] The candidate poses are obtained, and by comparing them with the set constraints, the candidate poses that satisfy the constraints are selected as the final poses.

[0009] The final pose is obtained, the pose of the part is adjusted, and then it is polished.

[0010] Optionally, the step of acquiring the contour information of the part through a vision sensor, grasping the part, and moving it to the initial position includes:

[0011] The visual sensor is used to acquire the contour information of the part;

[0012] Based on the contour information, it is matched with the 2D part model that needs to be polished to determine the geometric features of the part, including the free edges, origin coordinates and initial rotation angle of the part;

[0013] Based on the determined geometric features, the gripping point is determined, and the part is gripped by a magnetic suction robot arm;

[0014] The gripped part is moved to the initial position of the grinding equipment.

[0015] Optionally, the training method for the deep learning model includes:

[0016] A deep learning model is constructed to obtain training data for the polished parts. The training data includes the contour point set of the free edge of the polished parts, the contour point set of the magnetic nail, the origin position and rotation angle of the polished parts on the polishing platform.

[0017] The training data is labeled, with the set of contour points of the free edge of the grinding part and the set of contour points of the magnetic nail as input, and the origin position and rotation angle of the grinding part on the grinding platform as output.

[0018] The labeled training data is used to train the deep learning model, so that the trained deep learning model can generate poses that satisfy the constraints.

[0019] Optionally, after obtaining the training data of the polished parts, the training data needs to be standardized. The standardization process involves using sampling or interpolation methods to ensure that the dimensions of the input data are consistent.

[0020] Optionally, obtaining the candidate pose and selecting the candidate pose that satisfies the constraints as the final pose by comparing it with the set constraints includes:

[0021] The candidate pose is obtained, which includes the origin position and rotation angle of the part on the grinding platform;

[0022] The candidate pose is compared with the set constraints to check whether the candidate pose meets the set constraints. If it does, the current candidate pose is determined to be the final pose. If not, the pose parameters are adjusted and the verification is repeated.

[0023] Optionally, when checking whether the candidate pose meets the set constraints, if not, adjusting the pose parameters and re-verifying includes:

[0024] Adjustments are made through neighborhood search, and a set of solutions adjacent to the original solution is constructed for testing.

[0025] Optionally, the constraints include: free edge constraints, support area constraints, and site boundary constraints.

[0026] Optionally, the free edge constraint is: there is no magnetic nail interference when the free edges of the candidate pose information are extended inward within a first preset range and outward within a second preset range;

[0027] The support area constraint is that the support area of ​​the part on the magnetic nail is greater than 30%-90% of the part area;

[0028] The site constraint is that the orientation of the part does not exceed the boundary of the grinding site.

[0029] Optionally, the constraint condition includes a loss function, including: when the free edge is constrained, the overlapping area of ​​the free edge with the magnetic nail within the first preset range inward and the second preset range outward is used as part of the loss function;

[0030] When the support area constraint is applied: the difference between the proportion of the support area of ​​the part on the magnetic nail and the target proportion is used as part of the loss function;

[0031] When considering the site boundary constraints, whether the part exceeds the boundary of the grinding site is considered as part of the loss function.

[0032] Optionally, obtaining the final pose, adjusting the pose of the part, and performing polishing include:

[0033] The robotic arm acquires the final pose and adjusts the position and orientation of the part based on the final pose;

[0034] The robotic arm places the adjusted part into the polishing area of ​​the polishing equipment.

[0035] Start the grinding equipment to grind the free edge of the part, and move the ground part to the designated position to prepare for grinding the next part.

[0036] This application provides a method for determining the pose of parts in a grinding equipment for ship assembly parts. It employs visual recognition and deep learning technologies, acquiring part contour information through a visual sensor and generating candidate poses by combining them with a deep learning model. This significantly improves the efficiency and accuracy of pose determination. Simultaneously, it determines whether the candidate poses meet the set constraints to avoid interference with the magnetic nail support and ensures the stability of the parts on the magnetic nail support. This enables rapid determination of the part pose and solves the problems of low efficiency and insufficient accuracy of manual operation.

[0037] Other technical effects resulting from the additional features will be further illustrated in the corresponding embodiments. Attached Figure Description

[0038] Other features, objects, and advantages of this application will become more apparent from the following detailed description of non-limiting embodiments with reference to the accompanying drawings:

[0039] Figure 1 This is a flowchart illustrating a method for determining the pose of parts in a grinding equipment for ship assembly parts, according to an exemplary embodiment.

[0040] Figure 2 This is a schematic diagram illustrating a suitable structure for arranging parts according to an exemplary embodiment. Detailed Implementation

[0041] The present application will now be described in detail with reference to specific embodiments. These embodiments will help those skilled in the art to further understand the present application, but do not limit the present application in any way. It should be noted that those skilled in the art can make several modifications and improvements without departing from the concept of the present application, and these all fall within the protection scope of the present application. Parts not described in detail in the following embodiments can be implemented using existing technology.

[0042] It should be noted that all information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of related data must comply with relevant regulations.

[0043] In existing technologies, the placement and orientation of parts in automated grinding processes for ship assembly units require manual intervention, resulting in low efficiency and insufficient precision. To address these issues, this application provides a method for determining the orientation of parts in a grinding equipment for ship assembly units, thereby resolving these problems.

[0044] Reference Figure 1As shown in one embodiment of this application, a method for determining the pose of parts in a grinding equipment for ship assembly parts includes:

[0045] S1. Obtain the contour information of the part through the vision sensor, grasp the part, and move it to the initial position;

[0046] S2. Input the contour information of the part into the trained deep learning model to predict the position of the part and generate candidate poses.

[0047] S3. Obtain candidate poses and select the candidate poses that meet the set constraints as the final poses by comparing them with the set constraints.

[0048] S4. Obtain the final pose, adjust the pose of the part, and perform grinding.

[0049] Specifically, firstly, industrial vision sensors (such as cameras or laser scanners) are used to non-contactly scan the assembled parts of the ship assembly to obtain their contour information. A robotic arm is then controlled to automatically grasp the parts based on this contour information and move them to their initial position at the grinding station. Next, the contour information is input into a pre-trained deep learning model (such as PointNet++ or Transformer architecture). The model learns the geometry-pose mapping relationship from massive amounts of labeled data to quickly generate multiple candidate poses (including position coordinates and rotation angles). The candidate poses are validated according to set pose constraints, and the optimal pose that simultaneously satisfies the constraints is selected. Finally, the final pose parameters are converted into robotic arm motion commands to adjust the position and orientation of the parts, and the grinding tool is activated to perform automated grinding along a preset path.

[0050] The deep learning model can be a general multilayer perceptron. Of course, in other embodiments, other types of models can be selected as needed, as long as they can achieve the purpose of this application.

[0051] The embodiments described above in this application, by integrating visual perception, deep learning, and constraint optimization technologies, achieve full automation and intelligence in the grinding process of ship assembly parts. By identifying parts through visual sensors, the errors of manual measurement and operation are significantly reduced. At the same time, by combining deep learning models to generate candidate poses, the efficiency and accuracy of pose determination are greatly improved. When determining the final pose, the interference of free edge grinding and the stability of parts are solved by setting constraints. This realizes automated grinding of free edges of ship assembly parts, reduces human intervention, improves the consistency and quality of grinding, and promotes the automation, intelligence, and efficiency of the ship manufacturing process.

[0052] In some specific embodiments of this application, the process of acquiring contour information of a part through a vision sensor, grasping the part, and moving it to an initial position includes: acquiring contour information of the part through a vision sensor, wherein the contour information is a set of free edge contour points of the part; matching the contour information with a 2D part model that needs to be polished to determine the free edge, origin coordinates, and initial rotation angle of the part; determining the grasping point based on the determined geometric features, and grasping the part using a magnetic suction robot; and moving the grasped part to the initial position of the polishing equipment.

[0053] Specifically, the contour information of the part is obtained through a visual sensor (such as a camera or laser scanner), matched with a 2D model, and the origin coordinates, initial rotation angle, and free edge point set of the part are retrieved from the model. Based on the geometric features of the part, the gripping point is determined, and the part is gripped by a magnetic suction robot. The gripped part is moved to the initial position of the grinding equipment in preparation for pose determination.

[0054] In this application, the vision sensor is fixed above the part placement position to ensure the accuracy of the part contour information. The contour point set of the magnetic nail is the edge contour of each support platform with a different shape. After obtaining the contour information of the part, it is compared with the two-dimensional part model that needs to be polished. Through the two-dimensional part model, the origin and angle of the part placement can be known, and then the gripping point is given for the robot to grip.

[0055] In this application, the origin coordinates and the initial rotation angle are used to determine the actual gripping position and angle. After obtaining the feasible pose, it is still necessary to move the part to that feasible pose using a magnetic manipulator. The initial rotation angle facilitates the calculation of the movement and rotation of the manipulator when placing the part.

[0056] It should be noted that, based on the defined geometric features of the part, when gripping the part, the centroid of the 2D contour is used as the gripping point. If the centroid is not within the contour, the neighborhood of the centroid is explored. Specifically, in the 2D model, a circle with a certain radius is drawn with the centroid as the center. A point is determined at regular intervals, and each point is checked to see if it can be used as a gripping point. If none of these conditions are met, the radius of the circle is increased, and the search continues. If the process is repeated multiple times without finding a gripping point, an error message is displayed.

[0057] In the above embodiments, the visual sensor accurately captures the free edge contour point set of the part and performs high-precision matching with the preset 2D model to quickly lock key geometric parameters such as the free edge, origin coordinates and initial rotation angle of the part. At the same time, the magnetic suction robot achieves stable gripping based on geometric features, effectively avoiding the positioning deviation and damage risk of traditional manual gripping. Finally, the part is automatically moved to the initial position for grinding, which significantly shortens the process preparation time and improves the repeatability of positioning accuracy.

[0058] In some specific embodiments of this application, the training method for the deep learning model may include: constructing a deep learning model and obtaining training data for the grinding part, the training data including the contour point set of the free edge of the grinding part, the contour point set of the magnetic nail, the origin position and rotation angle of the grinding part on the grinding platform; taking the contour point set of the free edge of the grinding part and the contour point set of the magnetic nail as input, and taking the origin position and rotation angle of the grinding part on the grinding platform as output; labeling the training data, including the contour of the grinding part, the arrangement of the magnetic blocks and the corresponding feasible poses; and training the deep learning model with the labeled training data so that the trained deep learning model can generate poses that satisfy the constraints.

[0059] Specifically, first, a deep learning model is constructed. The input consists of the contour point set of the free edges of the part and the contour point set of the magnetic nails. The output is the origin position and rotation angle of the part in the grinding area. A large amount of labeled training data, including the part contour, the arrangement of the magnetic blocks, and the corresponding feasible poses, is used for supervised learning. The deep learning model is then trained to ensure that it can generate poses that satisfy the constraints.

[0060] In this application, the labeled training data includes: because the magnetic blocks are composed of magnetic nails, their shape and position on the grinding platform can be changed. Therefore, the arrangement of magnetic blocks used in actual production may not be the same as that used during training. However, we have several preset magnetic block arrangements, and one of these preset arrangements will be used in both neural network training and actual production.

[0061] The labeled training data uses the part outline and magnetic block arrangement as input variables, and the feasible origin position and rotation angle of the part in the grinding area as output variables. That is, it gives the origin position and rotation angle of the part in the grinding area. The specific process is as follows: first, a preset magnetic block arrangement is selected, then a part is selected. The part will determine its origin position and rotation angle in a random way. Then, the feasibility (that is, the part position and rotation angle that meet the requirements of stability and no collision) is calculated. The feasible results are included in the dataset. By changing the magnetic block arrangement and the part, the entire dataset can be obtained.

[0062] It should be noted that the grinding platform is a platform inside the entire grinding equipment, rather than a separate grinding platform. Magnetic nails are arranged on the grinding platform, and a deep learning model is used to determine how the parts should be placed on the magnetic nails, i.e., on the grinding platform.

[0063] In some specific embodiments of this application, after obtaining the training data of the polished parts, it is necessary to standardize the training data. The standardization process involves using sampling or interpolation methods to ensure that the dimensions of the input data are consistent.

[0064] Specifically, first determine the size of the point set. When the actual amount of data exceeds the set size, some points need to be discarded. When the actual amount of data is too small, it needs to be supplemented to the set size.

[0065] The principle for discarding is as follows: Calculate the cosine of the angle formed by the lines connecting each point to its two preceding and following points, sort them in ascending order, and delete them sequentially up to the set size. The principle is that the lower the cosine of the angle, the closer it is to -1, that is, the closer the angle is to 180°, indicating that the point is more likely to be on the line connecting its preceding and following points, and therefore is not important.

[0066] The principle of supplementation is as follows: Connect adjacent points in the existing point set to form n straight lines. The number of points to be supplemented is k*n+b (k, n, and b are all integers, and b is less than n). Then, randomly select b of the n straight lines, and insert k+1 points into each line according to the rule of equal division. For the other nb straight lines, insert k points into each line according to the rule of equal division.

[0067] The above embodiments of this application solve the problem of inconsistent numbers of contour point sets by standardizing the training data.

[0068] In some specific embodiments of this application, for obtaining candidate poses, the candidate poses that satisfy the set constraints are selected as the final poses by comparing them with the set constraints, including:

[0069] Obtain candidate poses, which include the origin position and rotation angle of the part in the grinding area;

[0070] The candidate pose is compared with the set constraints to check whether the candidate pose meets the set constraints. If it does, the current candidate pose is determined as the final pose. If not, the pose parameters are adjusted and the verification is repeated.

[0071] Specifically, check whether the candidate pose meets the following constraints. If the candidate pose does not meet the constraints, adjust the pose parameters (such as rotation angle or origin coordinates) and re-verify. If the constraints still cannot be met after multiple adjustments, report the pose and manually determine the pose. Select the feasible pose that meets all constraints as the final pose.

[0072] The embodiments described above in this application employ constraint verification and adjustment algorithms to ensure that the pose meets complex constraints, solve the problems of free edge grinding interference and part stability, and improve the reliability and safety of the grinding process.

[0073] In some specific embodiments of this application, when checking whether the candidate pose meets the set constraints, if not, the pose parameters are adjusted and re-verified, including: adjusting through neighborhood search and constructing a set of solutions adjacent to the original solution for testing.

[0074] Specifically, when the candidate pose output by the deep learning model does not meet the constraints, adjustments are made through neighborhood search to construct a set of solutions adjacent to the original solution. (That is, a grid with a certain step size is constructed with the coordinates of the original solution as the center, and the original solution is rotated multiple times by a certain angle to obtain a set of rotation angles. Each rotation angle at each grid point is tried one by one.) This is the adjustment algorithm in the above embodiment.

[0075] In some specific embodiments of this application, the constraints include: free edge constraints, support area constraints, and site boundary constraints.

[0076] In some specific embodiments of this application, the free edge constraint is as follows: there is no magnetic nail interference within the range of the free edge of the candidate pose information being extended inward by a first preset range (e.g., 10mm) and outward by a second preset range (e.g., 60mm).

[0077] The support area constraint is: the support area of ​​the part on the magnetic nail is greater than 30%-90% of the part area;

[0078] The site constraint is that the orientation of the part does not exceed the boundary of the grinding site.

[0079] In some specific embodiments of this application, a loss function is included within the constraints, including: for free edge constraints, the overlapping area of ​​the free edge with the magnetic nail within a first preset range (e.g., 10mm) inward and a second preset range (e.g., 60mm) outward of the free edge is included as part of the loss function; for support area constraints, the difference between the ratio of the support area of ​​the part on the magnetic nail and the target ratio is included as part of the loss function; for site boundary constraints, whether the part exceeds the boundary of the grinding site is included as part of the loss function. The calculation of the loss function can all employ existing technologies, and the calculation of the overlapping area of ​​the two-dimensional graphic can utilize readily available tool libraries.

[0080] The embodiments described above in this application incorporate factors such as the overlap area between the free edge and the magnetic nail within the extended range of the free edge under free edge constraints, the difference between the ratio of the support area of ​​the part on the magnetic nail and the target ratio under support area constraints, and whether the part exceeds the boundary of the grinding site under site boundary constraints into the loss function. This allows for a comprehensive and accurate measurement of the deviation of various indicators of the part from the ideal state during the grinding process, providing clear and quantitative basis for optimizing the grinding process, improving grinding quality, ensuring that the part meets design requirements, and guaranteeing the safety and standardization of grinding operations within the site area.

[0081] In some specific embodiments of this application, obtaining the final pose, adjusting the pose of the part, and grinding include: a robot arm obtaining the final pose and adjusting the position and orientation of the part using the final pose; the robot arm placing the part with the adjusted position and orientation onto the grinding area of ​​the grinding equipment; starting the grinding equipment, grinding the free edge of the part, and moving the ground part to a designated position to prepare for grinding the next part.

[0082] Specifically, the determined feasible pose is sent to the robot arm, which adjusts the position and orientation of the part; the robot arm moves downwards, demagnetizes and releases the part, and removes it; (Refer to...) Figure 2 As shown, place the part in a suitable position on the magnetic nail support (including the welded edge, free edge, and magnetic nail support), start the grinding tool, and grind the free edge of the part on both the top and bottom sides; after grinding is completed, the robot arm places the ground part in the designated position, ready for the grinding of the next part.

[0083] The embodiments described above in this application utilize a robotic arm to grasp parts, an algorithm to determine their placement, and a grinding tool to automatically grind the free edges. By combining magnetic nail support positioning and grinding tools, automated grinding of the free edges of ship assembly parts is achieved, reducing human intervention, improving grinding consistency and quality, and promoting the automation, intelligence, and efficiency of the shipbuilding process.

[0084] The preferred features in the above embodiments can be used individually in any embodiment, or in any combination thereof, provided they do not conflict with each other. Furthermore, parts not described in detail in the embodiments can be implemented using existing technologies.

[0085] The foregoing has described some specific embodiments of this application. It should be understood that this application is not limited to the specific embodiments described above, and those skilled in the art can make various modifications or variations within the scope of the claims, which do not affect the substantive content of this application. The above-described preferred features can be used in any combination without conflict.

Claims

1. A method for determining the position and orientation of parts in a grinding equipment for ship assembly parts, characterized in that, include: The contour information of the part is obtained through a vision sensor, the part is grasped, and moved to the initial position; The contour information of the part is input into a trained deep learning model to predict the position of the part and generate candidate poses. The candidate poses are obtained, and by comparing them with the set constraints, the candidate poses that satisfy the constraints are selected as the final poses. The final pose is obtained, the pose of the part is adjusted, and then polished. The training method for the deep learning model includes: A deep learning model is constructed to obtain training data for the polished parts. The training data includes the contour point set of the free edge of the polished parts, the contour point set of the magnetic nail, the origin position and rotation angle of the polished parts on the polishing platform. The training data is labeled, with the set of contour points of the free edge of the grinding part and the set of contour points of the magnetic nail as input, and the origin position and rotation angle of the grinding part on the grinding platform as output. The labeled training data is used to train the deep learning model so that the trained deep learning model can generate poses that satisfy the constraints. The constraints include: free edge constraints, support area constraints, and site boundary constraints; The free edge constraint is: there is no magnetic nail interference when the free edge of the candidate pose information is extended inward within a first preset range and outward within a second preset range; The support area constraint is that the support area of ​​the part on the magnetic nail is greater than 30%-90% of the part area; The site constraint is that the orientation of the part does not exceed the boundary of the grinding site. The constraints include a loss function, including: When the free edge is constrained, the area of ​​overlap between the free edge and the magnetic nail within the first preset range inward and the second preset range outward is used as part of the loss function; When the support area constraint is applied: the difference between the proportion of the support area of ​​the part on the magnetic nail and the target proportion is used as part of the loss function; When considering the site boundary constraints, whether the part exceeds the boundary of the grinding site is considered as part of the loss function.

2. The method for determining the position and orientation of parts in a grinding equipment for ship assembly parts according to claim 1, characterized in that, The step of acquiring the contour information of the part through a vision sensor, grasping the part, and moving it to an initial position includes: The visual sensor is used to acquire the contour information of the part; Based on the contour information, it is matched with the 2D part model that needs to be polished to determine the geometric features of the part, including the free edges, origin coordinates and initial rotation angle of the part; Based on the determined geometric features, the gripping point is determined, and the part is gripped by a magnetic suction robot arm; The gripped part is moved to the initial position of the grinding equipment.

3. The method for determining the position and orientation of parts in a grinding equipment for ship assembly parts according to claim 1, characterized in that, After obtaining the training data of the polished parts, the training data is standardized. The standardization process involves using sampling or interpolation methods to ensure that the dimensions of the input data are consistent.

4. The method for determining the position and orientation of parts in a grinding equipment for ship assembly parts according to claim 1, characterized in that, The step of obtaining the candidate pose and selecting the candidate pose that satisfies the set constraints as the final pose includes: The candidate pose is obtained, which includes the origin position and rotation angle of the part on the grinding platform; The candidate pose is compared with the set constraints to check whether the candidate pose meets the set constraints. If it does, the current candidate pose is determined to be the final pose. If not, the pose parameters are adjusted and the verification is repeated.

5. The method for determining the position and orientation of parts in a grinding equipment for ship assembly parts according to claim 4, characterized in that, When checking whether the candidate pose meets the set constraints, if not, the pose parameters are adjusted and re-verified, including: Adjustments are made through neighborhood search, and a set of solutions adjacent to the original solution is constructed for testing.

6. The method for determining the position and orientation of parts in a grinding equipment for ship assembly parts according to claim 2, characterized in that, The steps of obtaining the final pose, adjusting the pose of the part, and polishing include: The robotic arm acquires the final pose and adjusts the position and orientation of the part based on the final pose; The robotic arm places the adjusted part into the polishing area of ​​the polishing equipment. Start the grinding equipment to grind the free edge of the part, and move the ground part to the designated position to prepare for grinding the next part.

Images