Method for tracking and extracting contour of ring forging in real time in intelligent ring rolling forming process

Abstract

The invention belongs to the technical field of photoelectric measurement, and particularly relates to a ring forging contour real-time tracking and extracting method in the intelligent ring rolling forming process. According to the method, the pixel matching relation between the two adjacent frames of images is fully utilized, and accurate tracking and extraction of the annular forge piece are achieved. Particularly, after the contour of the annular forge piece in the first frame image is extracted, images of subsequent continuous frames are intelligently recognized, the contour of the annular forge piece is accurately and rapidly tracked and extracted, online contour extraction of the annular forge piece in the intelligent ring forging process is achieved, and the accuracy of ring forging is improved. Reliable data are provided for tasks such as real-time adjustment of parameters of ring forging equipment or measurement of main sizes of ring forgings. Moreover, the method is suitable for annular forgings with various bottom surface shapes, and has good robustness and wide application scenes.

Description

Technical Field

[0001] The present invention belongs to the technical field of optoelectronic measurement, and particularly relates to a method for real-time tracking and extraction of the contour of a ring forging during the intelligent ring rolling forming process. Background Art

[0002] As one of the key processes in metal material processing, ring forging (abbreviated as "ring rolling") is mainly used to manufacture components with a ring-shaped cross-section. During the ring rolling process, the metal blank is first heated to a high temperature, and then undergoes plastic deformation under the action of a press or forging equipment. This is mainly reflected in the gradual increase in the diameters of the inner and outer rings, the gradual decrease in the cross-sectional thickness between the inner and outer rings, and at the same time, under the action of the conical rollers, the height of the blank remains basically unchanged. When the diameters of the inner and outer rings of the blank basically reach the preliminary requirements, the ring rolling process terminates. During the entire ring rolling processing stage, it is necessary to accurately and real-time obtain the diameters of the inner and outer rings of the ring forging and their changing trends, so as to timely adjust the processing parameters of each working component of the equipment and ensure the acquisition of an ideal ring forging. Obviously, real-time tracking and extraction of the inner and outer ring contours of the forging is particularly crucial for the intelligent control of the entire ring rolling process.

[0003] In the field of ring forging or more extensive metal forging, due to the limitations of relevant computer algorithms or considering the cost issues of industrial equipment, existing measurement technologies mostly focus on static non-deformed forgings or deformed forgings with simple shapes, such as laser scanning or multi-view stereo vision technology. Such technologies have achieved relatively ideal accuracy in obtaining the main dimensions (such as diameter) of the forging, but they still cannot meet the requirements in terms of contour tracking and extraction. They cannot accurately obtain the complete contour of the forging, which is not conducive to processing forgings with non-standard shapes (such as ellipses), or the robustness of the algorithm is poor and it is only applicable to forgings with specific shapes (such as circles), thus unable to be migrated and applied to forging scenarios of other shapes.

[0004] In 2017, the Zhang Yucun team proposed a scanning method applicable to rotary forgings. This method mainly focuses on the outer ring diameter of circular forgings and completes one data collection through three scans. The algorithm cannot be migrated and applied to non-circular ring forgings. In 2018, the team further developed a measurement system for complex ring forgings. The algorithm mainly focuses on the three diameters and height of the ring forging and has not studied the contour measurement problems of other shaped forgings during the deformation process. Regarding contour extraction, Zhou Yijun et al. established a binocular stereo vision measurement system, which uses feature lines to replace traditional feature points for 3D measurement of hot forgings. The measurement error of this system in the forging workshop is 0.79%, and the measurement time is 1.9128 seconds, but it does not meet the time requirement for real-time measurement.

[0005] The robustness of the measurement algorithm and the limitation of measurement time affect the contour tracking and extraction of high-temperature forgings during the processing, which makes it difficult to measure complex forgings in real time. This is one of the urgent problems to be solved in the field of intelligent forging (including intelligent ring forging). Summary of the invention

[0006] In view of this, in order to solve the problems existing in the prior art, the present invention provides a method for real-time tracking and extraction of the contour of a ring forging during an intelligent rolling forming process.

[0007] In order to achieve the above object, the present invention adopts the following technical solution:

[0008] A method for real-time tracking and extraction of the contour of a ring forging during an intelligent rolling forming process is proposed. Intelligent sensing equipment and computer vision algorithms are used to complete real-time tracking and extraction of the contour of a ring forging. The optical flow information between two frames of RGB images is predicted by an optical flow model. Then, according to the optical flow information, each point of the existing target contour is screened and matched among the candidate contour points, and finally the key points of the new target contour are obtained. The obtained key points are connected to form the final new target contour.

[0009] Existing high-temperature forging contour extraction and measurement technologies mainly focus on metal components with relatively simple shapes (such as circles or rectangles), and cannot be extended to other complex shapes, and have poor robustness. In addition, the data processing algorithms for scanning or images of forging measurements are time-consuming and cannot meet the requirements for real-time measurement of deformed forgings during the forging process.

[0010] The present invention makes full use of the pixel matching relationship between two adjacent frames of images to achieve accurate tracking and extraction of ring forgings. In particular, after extracting the contour of the ring forging in the first frame of the image, the present invention intelligently identifies the images of the subsequent continuous frames, accurately and quickly tracks and extracts the contour of the ring forging, and realizes the online contour extraction of the ring forging during the intelligent ring forging process, providing reliable data for tasks such as real-time adjustment of ring forging equipment parameters or measurement of the main dimensions of the ring forging. In addition, the present invention is applicable to ring forgings of various bottom surface shapes, and has good robustness and a wide range of application scenarios.

[0011] Furthermore, the specific steps of the method for real-time tracking and extracting the contour of the ring forging during the intelligent rolling forming process include:

[0012] (1) Preparation: Place the initial forging blank in the processing area of ​​the ring rolling equipment, place an industrial camera directly above it, adjust the camera parameters and optimize the position and posture, and connect it to the computer port to provide a continuous video stream to the computer end. The computer synchronously parses the video stream into RGB images;

[0013] (2) Semantic segmentation: Perform mask recognition on the static ring forging in the RGB image in step (1), that is, segment the ring forging from the surrounding background;

[0014] (3) Target contour extraction: Based on the segmentation result in step (2), extract the inner and outer ring contours of the upper bottom surface of the ring forging by combining edge detection algorithms, dilation and erosion morphological operations, or clustering unsupervised learning, and mark them as the target contours to be tracked and extracted;

[0015] (4) Obtain new contour candidate points: When the ring rolling equipment starts processing the ring forging, after morphological dilation with a specified number of cycles, the mask of the first-frame target contour obtained in step (3) increases. Based on this, segment and perform edge detection on the second-frame RGB image to obtain the candidate points of the second-frame ring forging contour;

[0016] (5) Optical flow prediction: Predict the optical flow information between the first frame in step (1) and the second frame in step (4) through an optical flow model to obtain the displacement information between the matching points in the two frames of images;

[0017] (6) Intelligent matching: For the target contour points in the first-frame RGB image in step (3), combine the optical flow displacement information obtained in step (5) to perform intelligent search and matching among the second-frame contour candidate points in step (4), and mark them as matching points after finding;

[0018] (7) Construction of a new target contour: When all the matching in step (6) is completed, it means that the tracking and extraction of the target contour in the first frame in the second-frame RGB image are realized, and all the matching points in the second frame are connected in sequence to form a new target contour;

[0019] (8) Repeat steps (4) to (7) to continuously track and extract the inner and outer ring contours of the ring forging during the intelligent ring forging process;

[0020] (9) Process end: Wait until the ring forging reaches the required size and stop the entire process.

[0021] It should be noted that in the intelligent ring forging process of the present invention, an industrial camera is used for real-time acquisition of images. Specifically, the industrial camera is installed directly above the ring-shaped forging (from the top view angle). By adjusting the camera parameters and optimizing the pose, the top surface morphology information of the complete ring-shaped forging can be obtained. This morphology information enters the computer terminal in real time in the form of a video stream through the communication interface. The computer vision algorithm synchronously analyzes and processes the video stream, continuously decomposing the video stream into consecutive frames of RGB images. First, under human supervision, mask recognition is performed on the ring-shaped forging in the first frame of RGB image (static and not yet processed), that is, the ring-shaped forging is segmented from the surrounding background. Then, the inner and outer ring contours of the top surface of the ring-shaped forging are extracted through edge detection algorithms, morphological operations, clustering unsupervised learning, etc., and marked as the target contours that need to be tracked and extracted. After starting to process the blank, the second frame of RGB image is obtained, and intelligent matching is performed according to the target contours obtained in the first frame. After morphological dilation with a specified number of loops, the mask of the target contour increases. Based on this, the second frame of RGB image is segmented and edge-detected to obtain candidate points for the contour of the second frame of ring-shaped forging. At the same time, by predicting the optical flow information between the first frame and the second frame of RGB images through the optical flow model, the displacement information (including the moving distance and moving direction) between the matching points in the two frames of images can be obtained. For the target contour points in the first frame of RGB image, combined with their optical flow displacement information, intelligent search and matching are performed among the contour candidate points in the second frame. In this step, the number of iterations is set to ensure that the matching accuracy and time cost are within the required range. When the matching is completed, it means that the tracking and extraction of the target contour in the first frame in the second frame of RGB image are realized. All the matching points in the second frame are connected in sequence to form a new target contour, and the matching of the same principle starts in the third frame of RGB image ( Figure 1 ).

[0022] Furthermore, in step (1), the initial blank includes a circular shape, an elliptical shape, a triangular shape, a polygonal shape or a custom shape; and the industrial camera is installed at the top view angle of the ring-shaped forging.

[0023] Furthermore, the algorithm used for mask recognition in step (2) includes the SAM semantic segmentation model.

[0024] Furthermore, the edge detection algorithm in step (3) is the Canny algorithm, and the clustering unsupervised learning includes the DBSACN algorithm, the Spectral Clustering algorithm or the OPTICS algorithm.

[0025] Furthermore, the optical flow model in step (5) includes the NeuFlow model or the RAFT model, and the displacement information between the matching points in the two frames of images obtained includes the moving distance and the moving direction.

[0026] It should be noted that the present invention can accurately and quickly track and extract the contour information of the ring forging during the intelligent ring forging process, providing reliable data for the intelligent regulation of the deformation of the ring forging or the acquisition of key dimensions. The present invention is not limited to ring forgings with a specific bottom shape and is applicable to ring forgings with various bottom shapes.

[0027] In the field of ring forging, compared with the existing forging measurement technologies, the method for real-time tracking and extraction of the contour of the ring forging during the intelligent ring rolling forming process disclosed by the present invention has better robustness. Without adjusting the key parameters, it can be applied to ring forgings with various bottom shapes, meeting the needs of a wider range of industrial scenarios. The bottom shapes of the ring forgings include but are not limited to circular, elliptical, triangular, rectangular, polygonal, and other custom shapes ( Figure 2 ). Secondly, on the premise of ensuring the accuracy rate of contour extraction, it can track the target contour in real time, meeting the requirements of online extraction and measurement. The present invention has relatively low requirements for the configuration of the measurement equipment, which can reduce the production cost. BRIEF DESCRIPTION OF THE DRAWINGS

[0028] In order to more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the following will briefly introduce the drawings required for use in the description of the embodiments or the prior art. Obviously, the drawings in the following description are only the embodiments of the present invention. For those of ordinary skill in the art, without creative efforts, other drawings can also be obtained based on the provided drawings.

[0029] Figure 1 It is the algorithm flow for tracking and extracting the new target contour of the ring forging in the RGB image of the present invention.

[0030] Figure 2 It is a part of the ring parts applicable to the algorithm route of the present invention.

[0031] Figure 3 It is a schematic diagram of the route for tracking and extracting the new target contour of the ring forging in Embodiment 1 of the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0032] The following will clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all of them. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative efforts belong to the scope of protection of the present invention.

[0033] As used herein, the term "embodiment" does not necessarily imply that any embodiment described as "exemplary" is superior or better than other embodiments. Unless otherwise specified, performance index tests in the embodiments of the present application are carried out using conventional test methods in the art. It should be understood that the terms used in the present application are only for describing specific embodiments and are not intended to limit the content disclosed in the present application.

[0034] Unless otherwise specified, the technical and scientific terms used herein have the same meaning as commonly understood by those of ordinary skill in the technical field to which this application belongs; other test methods and technical means not specifically noted in this application refer to the experimental methods and technical means commonly used by those of ordinary skill in the art.

[0035] To better illustrate the content of the present application, numerous specific details are given in the following specific embodiments. Those skilled in the art should understand that the present application can still be implemented without some specific details. In the embodiments, some methods, means, instruments, devices, etc. well-known to those skilled in the art are not described in detail in order to highlight the gist of the present application.

[0036] On the premise of no conflict, the technical features disclosed in the embodiments of the present application can be combined arbitrarily, and the obtained technical solutions belong to the content disclosed in the embodiments of the present application.

[0037] The present invention belongs to the technical field of optoelectronic measurement, and specifically relates to a method for real-time tracking and extraction of the contour of a ring forging during the intelligent rotary forging process. The present invention makes full use of the pixel matching relationship between two adjacent frames of images to achieve accurate tracking and extraction of the ring forging. Especially after extracting the contour of the ring forging in the first frame of image, the present invention performs intelligent recognition on the subsequent consecutive frames of images, accurately and quickly tracks and extracts the contour of the ring forging, realizes the on-line contour extraction of the ring forging during the intelligent ring forging process, and provides reliable data for tasks such as real-time adjustment of ring forging equipment parameters or measurement of the main dimensions of the ring forging. Moreover, the present invention is applicable to ring forgings with various bottom shapes, and has good robustness and a wide range of application scenarios.

[0038] To better understand the present invention, the following embodiments are used to further specifically elaborate on the present invention, but it should not be understood as a limitation to the present invention. For those skilled in the art, some non-essential improvements and adjustments made according to the above-mentioned invention content are also considered to fall within the protection scope of the present invention.

[0039] Embodiment 1

[0040] A method for real-time tracking and extraction of the contour of a ring forging during the intelligent rotary forging process:

[0041] (1) Preparation: Place the initial blank of the circular forging in the processing area of the ring rolling equipment. Install an industrial camera directly above it (from the top view angle) and connect it to the computer port to provide a continuous video stream to the computer side. The computer synchronously analyzes the video stream into RGB images;

[0042] (2) Semantic segmentation: Use the SAM semantic segmentation model to perform mask recognition on the static ring-shaped forging in the RGB image of step (1), and segment the ring-shaped forging from the surrounding background;

[0043] (3) Target contour extraction: By combining the Canny edge detection algorithm, dilation and erosion morphological operations, and the DBSACN algorithm for clustering unsupervised learning, based on the segmentation result of step (2), extract the inner and outer ring contours of the upper bottom surface of the ring-shaped forging and mark them as the target contours to be tracked and extracted.

[0044] (4) Obtain new contour candidate points: The ring rolling equipment starts to process the ring-shaped forging. After 5 times of morphological dilation, the mask of the first-frame target contour obtained in step (3) increases. Based on this, segment and perform edge detection on the second-frame RGB image to obtain the candidate points of the second-frame ring forging contour;

[0045] (5) Optical flow prediction: Predict the optical flow information between the first-frame and the second-frame RGB images of step (1) through the NeuFlow optical flow model to obtain the moving distance and moving direction information between the matching points in the two frames of images;

[0046] (6) Intelligent matching: For the target contour points (700) of the first-frame RGB image in step (3), combined with the optical flow displacement information obtained in step (5), search for and match them among the second-frame contour candidate points in step (4). After finding them, mark them as matching points;

[0047] (7) Construction of a new target contour: After all the matching in step (6) is completed, it means that the tracking and extraction of the target contour of the first frame in the second-frame RGB image are realized. All the matching points in the second frame are connected in sequence to form a new target contour;

[0048] (8) Repeat steps (4) to (7) to continuously track and extract the inner and outer ring contours of the ring-shaped forging during the intelligent ring forging process;

[0049] (9) Process end: When the inner ring diameter of the ring-shaped forging reaches the required 75 cm and the outer ring diameter reaches 95 cm, stop the entire process.

[0050] The foregoing description of the disclosed embodiments enables those skilled in the art to practice or use the present invention. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present invention. Thus, the present invention is not intended to be limited to the embodiments shown herein but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A method for real-time tracking and extraction of the contour of a ring forging during intelligent rolling forming, which uses intelligent sensing equipment and computer vision algorithms to complete the real-time tracking and extraction of the contour of a ring forging, and is characterized in that: The optical flow information between two frames of RGB images is predicted through the optical flow model; then, according to the optical flow information, each point of the existing target contour is screened and matched among the contour candidate points, and finally the key points of the new target contour are obtained; and the obtained key points are connected to form the final new target contour.

2. The method according to claim 1, characterized in that The specific steps include: (1) Preparation: Place the initial forging blank in the processing area of ​​the ring rolling equipment, place an industrial camera directly above it, adjust the camera parameters and optimize the position and posture, and connect it to the computer port to provide a continuous video stream to the computer end. The computer synchronously parses the video stream into RGB images; (2) Semantic segmentation: performing mask recognition on the static ring forging in the RGB image in step (1), that is, segmenting the ring forging from the surrounding background; (3) Target contour extraction: by combining edge detection algorithm, dilation corrosion morphological operation or clustering unsupervised learning, based on the segmentation results in step (2), the inner and outer ring contours of the bottom surface of the annular forging are extracted and marked as the target contours to be tracked and extracted; (4) Obtaining new contour candidate points: The ring rolling equipment starts processing the ring forging. After the morphological expansion is performed for a specified number of cycles, the mask of the target contour of the first frame obtained in step (3) is enlarged, and the second frame RGB image is segmented and edge detected based on the mask, thereby obtaining the candidate points of the contour of the ring forging in the second frame; (5) Optical flow prediction: The optical flow information between the first frame of step (1) and the second frame of step (4) RGB images is predicted by the optical flow model to obtain the displacement information between the matching points in the two frames of images; (6) Intelligent matching: For the target contour points of the first frame RGB image in step (3), combined with the optical flow displacement information obtained in step (5), intelligent search and matching are performed in the second frame contour candidate points in step (4), and once found, they are marked as matching points; (7) New target contour construction: When all matches in step (6) are completed, it means that the target contour of the first frame is tracked and extracted in the second frame RGB image, and all matching points in the second frame are connected in sequence to form a new target contour; (8) repeating steps (4) to (7) to continuously track and extract the inner and outer ring contours of the ring forging during the intelligent ring forging process; (9) End of process: When the ring forging reaches the required size, the entire process is stopped.

3. The method according to claim 2, characterized in that In the step (1), the initial blank includes a circular, elliptical, triangular, polygonal or custom shape; and the industrial camera is installed at a top view angle of the annular forging.

4. The method according to claim 2, characterized in that: The algorithm used for mask recognition in step (2) includes a SAM semantic segmentation model.

5. The method according to claim 2, characterized in that: The edge detection algorithm in step (3) is the Canny algorithm, and the clustering unsupervised learning includes the DBSACN algorithm, the Spectral Clustering algorithm or the OPTICS algorithm.

6. The method according to claim 4, characterized in that The optical flow model in step (5) includes a NeuFlow model or a RAFT model, and the obtained displacement information between matching points in two frames of images includes a moving distance and a moving direction.

Images