Real-time detection system in forging forming process and three-dimensional reconstruction method thereof

Abstract

The invention discloses a real-time detection system in a forge piece forming process and a three-dimensional reconstruction method thereof. According to the method, the size of a forge piece in the forge piece forming process is monitored in real time through calibration of a binocular camera, temperature field changes in a space are detected through thermal imaging equipment, and the target contour of the forge piece is extracted through temperature information according to the high-temperature characteristics of the forge piece; and then performing data fusion and fitting operation on the contour extracted by the binocular camera and the contour extracted by the thermal imaging based on a target feature alignment algorithm to realize the calculation of the size of the forge piece in the forge piece forming process, and further calculating the size of the forge piece by using a slice reconstruction algorithm based on multi-angle contour data. And the forming evolution process of the forge piece is simulated and displayed in real time in a three-dimensional visualization engine. Compared with a traditional structure and a three-dimensional reconstruction process of light or point cloud information, the detection efficiency is remarkably improved, and the urgent requirement of the free forging field for a high-real-time-performance and high-accuracy detection technology is completely met.

Description

Technical Field

[0001] The invention belongs to forging technology, and specifically relates to three-dimensional modeling and data monitoring technology in a workpiece forging process, in particular to a real-time detection and three-dimensional reconstruction method in a forging forming process. Background Art

[0002] Free forging involves extreme conditions such as high temperature and high impact. The process is complex and changeable, and it is difficult to achieve comprehensive control through simple, standard automation means. It is necessary to monitor the shape, size, thermal field and other factors in the forging process in real time to ensure the accuracy of the forging process, so as to achieve highly flexible and precise operation. In the field of free forging, the shape and size detection of castings and forgings in the forming process is usually estimated by the ruler on one side of the forging press, which places high demands on the experience and attention of the operator. After completing the specified process, the caliper is required to check the size. The disadvantage of this production process is that the dimensional monitoring during the forming process mainly relies on manual labor, which is unstable and has errors of varying sizes. On the other hand, although the data of the dimensional review after the process is completed is accurate, it cannot reflect the forming process in time, let alone guide the production process. If there is a problem, the process can only be corrected, which reduces production efficiency.

[0003] With the development of visual inspection and 3D model reconstruction technology, the following issues need to be considered when applying them in the forging process:

[0004] Accuracy: Although the vision-based size detection and shape detection algorithms are relatively mature, they are mainly aimed at highly automated production fields such as electronic products and automobile production. At the same time, there is a good detection environment during the process production, which greatly improves the detection accuracy. In the field of free forging, the traditional size measurement and 3D reconstruction technology are greatly limited by the deployment method, light texture, high temperature, thermal radiation and other environmental factors.

[0005] Real-time: For three-dimensional structure reconstruction, without considering real-time performance, high-precision three-dimensional scene restoration can be performed through structured light, oblique photography and other technologies. During the hot forging process, due to the high requirements for real-time performance, the existing technology cannot solve this problem well. Summary of the invention

[0006] Purpose of the invention: The present invention aims at the problem of real-time monitoring of forming parameters in the forging process of a workpiece, and provides a real-time detection system and a three-dimensional reconstruction method in the forging forming process.

[0007] Technical solution: A real-time detection and three-dimensional reconstruction method during the forging forming process, comprising the following steps:

[0008] S1. Build a detection system that uses binocular camera calibration to monitor the size of the forging process in real time, uses thermal imaging equipment to detect changes in the temperature field in space, and uses the high-temperature forging characteristics of the forging to extract the target contour of the forging through temperature information;

[0009] S2, based on the target feature alignment algorithm, the contour extracted by the binocular camera in step S1 and the contour extracted by thermal imaging are subjected to data fusion and fitting operations to realize the calculation of the forging size during the forging forming process;

[0010] S3. The three-dimensional structure of the forging is restored. Specifically, the direction of the forging body is the X direction, and the direction of the cross section is the Y direction. The forging model is sliced ​​according to the gradient change of the contour from the X direction perspective, and the cross-sectional shape is restored from the contour from the Y direction perspective. The forging model slices are reconstructed into a three-dimensional structure through the cross-sectional information, and the slices are reassembled to construct the overall three-dimensional structure of the forging.

[0011] Furthermore, step S1 for calibrating the binocular camera includes:

[0012] S11, establishing a camera imaging geometry model through binocular camera calibration, obtaining camera internal and external parameters to establish a distortion model of a monocular camera, which also includes obtaining distortion coefficients and stereo correction;

[0013] S12: using the camera intrinsic parameters and extrinsic parameters, and performing stereo correction through a Bouguet epipolar correction algorithm to eliminate physical distortion and improve the accuracy of measurement and positioning;

[0014] S13: performing algorithm preprocessing on the corrected image, including using algorithms such as contrast enhancement, edge sharpening, image filtering and / or contrast-limited adaptive histogram equalization to reduce the difficulty of subsequent extraction of the target area;

[0015] S14: Extract the forging contour from the preprocessed image and separate the non-target area including the fixture.

[0016] Step S1 processes the temperature field data detected by the thermal imaging device as follows:

[0017] Based on the FSRCNN algorithm, the temperature field image of thermal imaging is processed with super resolution to improve the perceived quality of the image;

[0018] The pixel difference between the reconstructed image and the true high-resolution image is measured by the following formula:

[0019]

[0020] Where MSE represents mean square error, N represents the total number of pixels in the image, HR represents the true high-resolution image, and SR represents the image output by the super-resolution reconstruction algorithm;

[0021] The temperature threshold is set according to the casting temperature of the forging, and the temperature profile of the forging is extracted by threshold segmentation.

[0022] Furthermore, step S2 extracts the features of the forging contour by using the SURF operator, and then performs edge fitting on the fused contour to remove abnormal points, thereby improving the smoothness and accuracy of the contour, and then calculates the forging dimensions during the forging forming process according to the contour.

[0023] Further, step S3 includes:

[0024] Set the detection step length, calculate the gradient change combined with the sampling point slope, and slice the model. The gradient calculation formula includes:

[0025] Horizontal gradient changes:

[0026] G x =|f(x2,y)-f(x1,y)|

[0027] Vertical gradient changes:

[0028] G y =|f(x,y2)-f(x,y1)|

[0029] Gradient vector norm:

[0030]

[0031] The model slices in the X direction are matched and fused with the cross-sectional information in the Y direction to construct the overall three-dimensional structure of the forging, and then the three-dimensional structure is post-processed, including smoothing and hole filling, to optimize the final three-dimensional model.

[0032] The present invention also provides a real-time detection system in the forging forming process to realize the measurement and data collection in the above method, comprising:

[0033] Binocular camera, used for dimensional measurement of forgings during the forming process;

[0034] Thermal imaging equipment, used to monitor the temperature field changes of forgings and the surrounding environment;

[0035] Laser distance measuring sensor, used for calibration and measurement of the spatial position of forgings;

[0036] Calibration board, used for calibration of binocular camera to ensure measurement accuracy;

[0037] A computer is used to execute the above-mentioned real-time detection and three-dimensional reconstruction method in a forging forming process.

[0038] Furthermore, the binocular camera and the thermal imaging device are aligned, and the visual angle includes the forging interval, and the laser ranging sensor is used to assist in the calibration of the binocular camera and the thermal imaging device and verify the collected data.

[0039] Beneficial effects: The real-time detection and three-dimensional reconstruction method in the forging forming process provided by the present invention sets up detection points in two horizontal directions of the forging processing process, and uses a binocular camera + thermal imaging + laser ranging sensor detection scheme to perform real-time detection of the process. It can use computer vision technology to perform real-time detection of the size and shape of the castings and forgings during the forming process, and dynamically reconstruct the forging model in the three-dimensional engine. It effectively avoids the lag of dimensional measurement in the traditional production process, and at the same time, completely restores the forming process through three-dimensional reconstruction technology, guides operators to perform related process operations, and also provides a basis for real-time production data for subsequent unmanned forging systems. BRIEF DESCRIPTION OF THE DRAWINGS

[0040] Figure 1 It is a schematic diagram of the process of the method of the present invention;

[0041] FIG2 is a schematic diagram of the working of the detection system, wherein FIG2(a) is the overall structure of the detection system, and FIG2(b) is a top view of the detection system;

[0042] Figure 3 is a schematic diagram of the camera correction process in the present invention;

[0043] Figure 4 It is a schematic diagram of the process of machine correction and contour fusion of the present invention;

[0044] Figure 5 This is the specific process of the forging size detection algorithm in the present invention;

[0045] Figure 6 It is a schematic diagram of the forging three-dimensional reconstruction method. DETAILED DESCRIPTION

[0046] In order to explain the technical solution disclosed in the present invention in detail, a further introduction is given below in conjunction with the specification and the accompanying drawings.

[0047] The present invention discloses a real-time detection and three-dimensional reconstruction method in the forging forming process. The method integrates cutting-edge computer vision technology to achieve real-time and accurate three-dimensional morphological reconstruction of the entire forging forming process. The specific implementation includes: first, using a high-precision binocular camera to continuously and uninterruptedly monitor the shape and size of the target forging in real time; second, using thermal imaging technology to capture the thermal field change information on the forging surface, and correcting the binocular camera results to eliminate the influence of thermal deformation; finally, based on multi-angle contour data, using a slice reconstruction algorithm, the forming evolution process of the forging is instantly simulated and displayed in a three-dimensional visualization engine. The innovation of the present invention is that it effectively simplifies the traditional three-dimensional reconstruction process based on structured light or point cloud information, significantly improves the detection efficiency, and fully meets the urgent demand for high real-time and high-accuracy detection technology in the field of free forging, providing strong technical support for quality control in the forging process, and laying the foundation for the precise control of unmanned forging systems.

[0048] A real-time detection system in a forging forming process, the system is used to detect and scan real-time relevant data of the forging during the forging process, the system comprising:

[0049] Binocular camera: A binocular camera system is composed of two high-resolution, low-distortion monocular cameras to ensure that the details of forging forming can be accurately captured.

[0050] Thermal imaging equipment: A high-sensitivity, high-resolution thermal imager is used to monitor the temperature field changes of forgings and the surrounding environment.

[0051] Laser ranging sensor: assists in spatial position calibration and precise measurement.

[0052] Computer and software: configure a high-performance computer, build an algorithm operating environment, install a three-dimensional engine, including Unreal 5, Pytorch, opencv, etc. that can be implemented in combination with existing technologies, and be able to execute the real-time detection and three-dimensional reconstruction method in the forging forming process described in the present invention.

[0053] Calibration board: used for calibrating binocular cameras to ensure measurement accuracy.

[0054] Brackets and fixtures: Ensure that cameras, thermal imaging equipment and laser ranging sensors can be stably installed in designated locations.

[0055] As shown in Figure 2, the detection systems are arranged on both sides of the forging press. The main measuring surface (X direction) is about 6m away from the center of the forging, and the distance in the cross-sectional direction (Y direction) is about 10m, ensuring that the forging forming process can be captured from different perspectives. The binocular camera and thermal imaging equipment need to be accurately aligned with the forging area, and the laser ranging sensor is used to assist calibration and verification.

[0056] Combination Figure 1, 3 -5, the present invention provides a real-time detection and three-dimensional reconstruction method in the forging forming process, and the implementation steps are as follows:

[0057] Step 1: Build a detection system that uses binocular camera calibration to monitor the dimensions of the forging process in real time.

[0058] Step 11. Use the calibration board to calibrate the binocular camera, and calculate the camera internal parameters M1 (focal length, optical center, etc.) and external parameters M2 (camera position, posture) through Zhang Zhengyou calibration method. Apply the stereo correction algorithm to eliminate image distortion and improve the accuracy of subsequent processing.

[0059] Distortion correction includes the following calculations:

[0060]

[0061] Image distortion includes radial distortion and tangential distortion. The radial distortion calculation is as follows:

[0062]

[0063] The tangential distortion is calculated as follows:

[0064]

[0065] Among them, the left side of the distortion formula is the pixel coordinate before distortion correction, and x and y on the right are the pixel coordinates when there is no distortion in an ideal situation. k and p are correction parameters.

[0066] Step 112, use the parameters M1 and M2 in step S11 to perform stereo correction through the Bouguet polar line correction algorithm, the purpose is to eliminate physical distortion and improve the accuracy of measurement and positioning. For the flowchart, refer to Figure 3 As shown;

[0067] Step 113, pre-process the corrected image, including contrast enhancement, edge sharpening, Gaussian filtering, etc., to improve image quality and facilitate subsequent processing. In this example, the Contrast-Limited Adaptive Histogram Equalization (CLAHE) algorithm is used to enhance image details while suppressing background noise, and the Sobel operator is used to sharpen the edges. The CLAHE algorithm process is as follows:

[0068] 1. Preprocessing, such as image block filling, etc.;

[0069] 2. For each block, calculate the mapping relationship, and use contrast limitation when calculating the mapping relationship;

[0070] 3. Use interpolation method to get the final enhanced image;

[0071] Edge sharpening algorithm using the Sobel operator:

[0072] Create templates in the X and Y directions, where GX is used to detect vertical edges and Gy is used to detect horizontal edges. The template is shown in the figure and the formula is as follows:

[0073] G x =(Z7+2Z8+Z9)-(Z1+2Z2+Z3)

[0074] G y =(Z3+2Z6+Z9)-(Z1+2Z4+Z7)

[0075] Step 14: Use image segmentation technology (such as threshold segmentation and edge detection) to extract the forging contour, and set a reasonable region of interest (ROI) to reduce background interference and improve extraction accuracy.

[0076] Step 2: Use thermal imaging equipment to detect changes in the temperature field in space, utilize the high temperature characteristics of the forging, and extract the target contour of the forging through temperature information.

[0077] Step 21: Use super-resolution reconstruction algorithms (such as SRCNN and SRGAN based on deep learning) to improve the image quality of thermal imaging images. In this example, the FSRCNN algorithm is used to meet the real-time requirements. The mean square error is a commonly used loss function that measures the pixel difference between the reconstructed image and the real high-resolution image. The formula is:

[0078]

[0079] Among them, MSE represents mean square error, N represents the total number of pixels in the image, HR represents the real high-resolution image, SR represents the image output by the super-resolution reconstruction algorithm, and the network structure refers to Figure 4 .

[0080] Step 22, using the high temperature characteristics of forgings, set a suitable temperature threshold, and extract the temperature profile of the forgings through threshold segmentation. Specifically, it includes using the super-resolution reconstruction results of the thermal imaging image, using the temperature threshold to segment the target and the background, and obtaining the segmentation results under the temperature field of the forgings.

[0081] Step 3, based on the target feature alignment algorithm, the contour extracted by the binocular camera in Step 1 and the contour extracted by thermal imaging in Step 2 are subjected to data fusion and fitting operations to realize the calculation of the forging size during the forging forming process;

[0082] Step 31: Use a target alignment algorithm based on feature points to accurately align and fuse the contour extracted by the binocular camera with the thermal imaging contour to improve the accuracy of detection. In this example, the SURF operator can be used for feature extraction.

[0083] Step 32: Perform edge fitting on the fused contour and remove outliers to ensure the smoothness and accuracy of the contour.

[0084] Step 33. Calculate the forging dimensions during the forging forming process based on the contour.

[0085] Step 4: Restore the 3D structure of the forging. Specifically, the direction of the forging body is the X direction, and the direction of the cross section is the Y direction. The forging model is sliced ​​according to the gradient change of the contour from the X direction perspective, and the cross-sectional shape is restored from the contour from the Y direction perspective. The forging model slices are reconstructed into 3D structure through the cross-sectional information, and the slices are reassembled to construct the overall 3D structure of the forging. Figure 5 shown.

[0086] In this example, the relevant detection step size is set, the gradient change is calculated in combination with the slope of the sampling point, and the model is sliced. The gradient calculation formula is as follows:

[0087] Horizontal gradient change: G x =|f(x2,y)-f(x1,y)|

[0088] Vertical gradient change: G y =|f(x,y2)-f(x,y1)

[0089] Gradient vector norm:

[0090] The model slices in the X direction are matched and fused with the cross-sectional information in the Y direction to construct the overall three-dimensional structure of the forging.

[0091] Post-process the 3D structure, such as smoothing and hole filling, to optimize the final 3D model.

[0092] The three-dimensional structure restoration of the forging is to take the direction of the forging body as the X direction and the direction of the cross section as the Y direction, slice the forging model according to the gradient change of the contour from the X direction perspective, restore the cross-sectional shape from the contour from the Y direction perspective, reconstruct the three-dimensional structure of the forging model slices through the cross-sectional information and gradient transformation information, and then reassemble multiple slices to construct the overall three-dimensional structure of the forging. The innovation of this part is to abandon the traditional technical route of modeling using point cloud data, and use the gradient change of the main perspective contour to slice and simplify the constructed forging model. Figure 6 As shown, the three-dimensional model of the forging can be quickly restored by combining the cross-sectional shape and the slice ΔH value.

[0093] The above description is only a specific implementation of the present application, so that those skilled in the art can understand or implement the present application. Various modifications to these embodiments will be apparent to those skilled in the art, and the general principles defined herein can be implemented in other embodiments without departing from the spirit or scope of the present application. Therefore, the present application will not be limited to the embodiments shown herein, but will conform to the widest range consistent with the principles and novel features applied for herein.

Claims

1. A method for real-time detection and three-dimensional reconstruction during forging forming, characterized in that the steps include: S1. Build a detection system that uses binocular camera calibration to monitor the size of the forging process in real time, uses thermal imaging equipment to detect changes in the spatial temperature field, and uses the forging temperature characteristics of the forging to extract the target contour of the forging through temperature information; S2, based on the target feature alignment algorithm, the contour extracted by the binocular camera in step S1 and the contour extracted by thermal imaging are subjected to data fusion and fitting operations to realize the calculation of the forging size during the forging forming process; S3. The three-dimensional structure of the forging is restored. Specifically, the direction of the forging body is the X direction, and the direction of the cross section is the Y direction. The forging model is sliced ​​according to the gradient change of the contour from the X direction perspective, and the cross-sectional shape is restored from the contour from the Y direction perspective. The forging model slices are reconstructed into a three-dimensional structure through the cross-sectional information, and the slices are reassembled to construct the overall three-dimensional structure of the forging.

2. The real-time detection and three-dimensional reconstruction method in the forging forming process according to claim 1 is characterized in that: Step S1 includes the following steps for calibrating the binocular camera: S11, establishing a camera imaging geometry model through binocular camera calibration, obtaining camera intrinsic parameters and extrinsic parameters to establish a distortion model of a monocular camera, which also includes obtaining distortion coefficients and stereo correction; S12: using the camera intrinsic parameters and extrinsic parameters, and performing stereo correction through a Bouguet epipolar correction algorithm to eliminate physical distortion and improve the accuracy of measurement and positioning; S13: performing algorithm preprocessing on the corrected image, including using algorithms such as contrast enhancement, edge sharpening, image filtering and / or contrast-limited adaptive histogram equalization to reduce the difficulty of subsequent extraction of the target area; S14: Extract the forging contour from the preprocessed image and separate the non-target area including the fixture.

3. The real-time detection and three-dimensional reconstruction method in the forging forming process according to claim 1 is characterized in that: Step S1 processes the temperature field data in the space detected by the thermal imaging device as follows: Based on the FSRCNN algorithm, the temperature field image of thermal imaging is processed with super resolution to improve the perceived quality of the image; The pixel difference between the reconstructed image and the true high-resolution image is measured by the following formula: Where MSE represents mean square error, N represents the total number of pixels in the image, HR represents the real high-resolution image, and SR represents the image output by the super-resolution reconstruction algorithm; The temperature threshold is set according to the casting temperature of the forging, and the temperature profile of the forging is extracted by threshold segmentation.

4. The real-time detection and three-dimensional reconstruction method in the forging forming process according to claim 1 is characterized in that: Step S3 extracts the features of the forging contour by using the SURF operator, then performs edge fitting on the fused contour to remove abnormal points, improve the smoothness and accuracy of the contour, and then calculates the forging dimensions during the forging forming process based on the contour.

5. The real-time detection and three-dimensional reconstruction method in the forging forming process according to claim 1 is characterized in that: Step S4 includes: Set the detection step length, calculate the gradient change combined with the sampling point slope, and slice the model. The gradient calculation formula includes: Horizontal gradient changes: G x =|f(x2,y)-f(x1,y) Vertical gradient changes: G y =|f(x,y2)-f(x,y1) Gradient vector norm: The model slices in the X direction are matched and fused with the cross-sectional information in the Y direction to construct the overall three-dimensional structure of the forging, and then the three-dimensional structure is post-processed, including smoothing and hole filling, to optimize the final three-dimensional model.

6. A real-time detection system in the forging forming process, characterized in that: include: Binocular camera, used for dimensional measurement of forgings during the forming process; Thermal imaging equipment, used to monitor the temperature field changes of forgings and the surrounding environment; Laser distance measuring sensor, used for calibration and measurement of the spatial position of forgings; Calibration board, used for calibration of binocular camera to ensure measurement accuracy; A computer for executing the method according to any one of claims 1 to 5.

7. The real-time detection system in the forging forming process according to claim 6 is characterized in that: The binocular camera and thermal imaging device are aligned, and the visual angle includes the forging interval. The laser ranging sensor is used to assist in the calibration of the binocular camera and thermal imaging device and verify the collected data.

Images