Binocular vision-based method and system for coaxially online monitoring of three-dimensional morphology of molten pool

Abstract

A binocular vision-based method and system for coaxially online monitoring of a three-dimensional morphology of a molten pool. The method comprises: acquiring dual-view images of a molten pool; inputting the dual-view images of the molten pool into an unsupervised adaptive loss neural network to obtain disparity information of the molten pool; using a checkerboard to calibrate a binocular vision monitoring system to obtain system parameter information; and obtaining depth information of the dual-view images of the molten pool on the basis of the disparity information of the molten pool and calibrated parameter information, and reconstructing the three-dimensional morphology of the molten pool on the basis of the depth information of the dual-view images of the molten pool.

Description

A Coaxial Online Monitoring Method and System for 3D Topography of Molten Pool Based on Binocular Vision Technical Field

[0001] This invention relates to the field of industrial vision technology, and in particular to a method and system for coaxial online monitoring of the three-dimensional shape of a molten pool based on binocular vision. Background Technology

[0002] In laser additive manufacturing, several key process variables can directly or indirectly represent the quality of the parts. The molten pool is the primary source of dynamic information in the laser additive manufacturing process, and its surface morphology is directly related to the quality of the manufactured parts. Therefore, monitoring the surface morphology characteristics of the molten pool is of great significance for achieving high-quality laser additive manufacturing.

[0003] For online monitoring of molten pool geometry, current methods primarily rely on commercially available equipment such as high-speed cameras and infrared thermal imagers to acquire two-dimensional information about the length and width of the molten pool, failing to capture its three-dimensional morphological features. For the three-dimensional morphology of the molten pool, the mainstream approach is to obtain the relative height of the molten pool through interferometry and binocular vision to achieve online monitoring. However, coaxial online coherent imaging methods struggle to capture the three-dimensional morphology of the entire molten pool region. While holographic interferometry can acquire the molten pool's three-dimensional morphology, it is difficult to apply to the complex coaxial systems of actual laser additive manufacturing processes. Traditional binocular vision sensing methods always use two independent cameras to detect the target from different angles, making it difficult to ensure good synchronization between the two cameras. Furthermore, using two cameras increases system complexity and equipment cost. A single-camera binocular vision sensing system based on the principle of biprism refraction can capture the molten pool's three-dimensional morphology in real time and has a simple structure; however, due to its fixed viewing angle, this system is difficult to apply to the narrow coaxial systems of laser additive manufacturing.

[0004] In summary, there is an urgent need to develop a simple, high-precision, and coaxial online monitoring method and system for the three-dimensional morphology of the molten pool that is applicable to laser additive manufacturing. Summary of the Invention

[0005] This invention provides a method and system for coaxial online monitoring of the three-dimensional morphology of a molten pool based on binocular vision, which solves the defects in the prior art. It has a simple structure and can acquire binocular vision images of the molten pool in the narrow coaxial system of laser additive manufacturing, thereby achieving high-precision online monitoring of the three-dimensional morphology of the molten pool.

[0006] In a first aspect, the present invention provides a method for coaxial online monitoring of the three-dimensional topography of a molten pool based on binocular vision, comprising:

[0007] Design a binocular vision system to coaxially acquire dual-view images of the molten pool;

[0008] The dual-view image of the molten pool is input into an unsupervised adaptive loss neural network to obtain molten pool disparity information;

[0009] The binocular vision monitoring system was calibrated using a checkerboard pattern to obtain the system's baseline and focal length parameters.

[0010] Based on the molten pool parallax information and the calibration monitoring system parameters, the depth information of the molten pool dual-view image is obtained, and the three-dimensional shape of the molten pool is reconstructed based on the depth information of the molten pool dual-view image.

[0011] According to the present invention, a coaxial online monitoring method for the three-dimensional topography of a molten pool based on binocular vision is provided, wherein the dual-view images of the molten pool are input into an unsupervised adaptive loss neural network to obtain molten pool disparity information, including:

[0012] An unsupervised adaptive loss neural network is determined, including a disparity extraction neural network Res_Unet, a view mapping module Unwrap, and a loss function module. The melt pool dual-view image includes melt pool images of view 1 and view 2.

[0013] The molten pool images of viewpoint 1 and viewpoint 2 are input into the disparity extraction neural network Res_Unet to obtain the molten pool disparity images of viewpoint 1 and viewpoint 2.

[0014] The melt pool image of viewpoint 1 and the melt pool parallax image of viewpoint 1 are used to obtain the melt pool image of predicted viewpoint 2 through the viewpoint mapping module Unwrap; the melt pool image of viewpoint 2 and the melt pool parallax image of viewpoint 2 are used to obtain the melt pool image of predicted viewpoint 1 through the viewpoint mapping module Unwrap; the viewpoint mapping module Unwrap includes matrix translation and rotation operations;

[0015] The loss function module calculates the appearance matching loss between the two viewpoint melt pool images and the predicted melt pool images, the smoothness loss of the two viewpoint melt pool disparity images, and the consistency loss of the two viewpoint melt pool disparity images using the loss function of the loss function module. The loss is then backpropagated so that the network can continuously learn to reduce the loss. When the preset number of iterations is reached, the learning stops, and the melt pool disparity information is obtained.

[0016] Furthermore, the appearance matching loss between the two viewpoint melt pool images and the predicted two viewpoint melt pool images includes the appearance matching loss between the melt pool image at viewpoint 1 and the predicted melt pool image at viewpoint 1, and the appearance matching loss between the melt pool image at viewpoint 2 and the predicted melt pool image at viewpoint 2; the smoothness loss of the two viewpoint melt pool disparity images is the gradient of each disparity image; the consistency loss is the error value obtained by inverting the melt pool disparity image at one viewpoint and subtracting it from the melt pool disparity image at the other viewpoint.

[0017] According to the present invention, a coaxial online monitoring method for the three-dimensional topography of a molten pool based on binocular vision is provided, wherein the molten pool image at viewpoint 1 and the molten pool image at viewpoint 2 are input into the disparity extraction neural network Res_Unet to obtain the molten pool disparity image at viewpoint 1 and the molten pool disparity image at viewpoint 2, including:

[0018] The disparity extraction neural network Res_Unet is defined to include a residual module, a max pooling module, and an upsampling module;

[0019] The residual module includes three convolutional layers. The first convolutional layer has a 1×1 kernel, and the second and third convolutional layers both have 3×3 kernels. The second and third convolutional layers include batch normalization and activation operations. The third convolutional layer includes connection with the original input features and activation operations.

[0020] The max pooling module halves the image size while keeping the number of layers unchanged, and the upsampling module enlarges the image size while varying the number of layers according to a setting.

[0021] The molten pool image from viewpoint 1 and the molten pool image from viewpoint 2 are sequentially input into the residual module, the max pooling module, and the upsampling module, and the predicted molten pool disparity image from viewpoint 1 and the predicted molten pool disparity image from viewpoint 2 are output.

[0022] According to the present invention, a coaxial online monitoring method for the three-dimensional topography of a molten pool based on binocular vision is provided, wherein the loss function includes appearance matching loss, disparity smoothness loss and disparity consistency loss;

[0023] The appearance matching loss includes the appearance matching loss between the melt pool image at viewpoint 1 and the predicted melt pool image at viewpoint 1, and the appearance matching loss between the melt pool image at viewpoint 2 and the predicted melt pool image at viewpoint 2.

[0024] The disparity smoothness loss includes the gradient of the molten pool disparity image at viewpoint 1 and the gradient of the molten pool disparity image at viewpoint 2.

[0025] The parallax consistency loss is the loss value obtained by subtracting the parallax image of the melt pool from the parallax image of the melt pool from the parallax image of the melt pool from the parallax image of the melt pool in two viewpoints.

[0026] The appearance matching loss, the disparity smoothness loss, and the consistency of the disparity images of the two viewpoints are weighted and summed to obtain the loss function.

[0027] According to the present invention, a coaxial online monitoring method for the three-dimensional topography of a molten pool based on binocular vision is provided, which uses a checkerboard pattern to calibrate the binocular vision monitoring system and obtain system parameter information, including:

[0028] Using the distance from the main lens to the sensor as the focal length, the center distance as the baseline, and the difference between the first and second differences as the parallax, a basic geometric relationship is established.

[0029] Three identical standard checkerboard grids are placed on the substrate of the laser additive manufacturing equipment. The thickness of the checkerboard grid is d. One checkerboard grid is placed alone as scene A, and the two checkerboard grids are stacked together as scene B. The height difference between the two is d.

[0030] Parallax information is obtained from the dual-view image of the chessboard in scene B. The imaging distance corresponding to the parallax is z'. Similarly, the imaging distance of the chessboard in scene A is z'+d.

[0031] Substituting the parallax and corresponding imaging distance of scene A and scene B into the basic geometric relationship, the baseline and focal length parameter information of the system are obtained;

[0032] According to the present invention, a coaxial online monitoring method for the three-dimensional shape of a molten pool based on binocular vision is provided, which obtains depth information of a molten pool from a dual-view image based on the molten pool disparity information and the parameter information, and reconstructs the three-dimensional shape of the molten pool based on the depth information of the molten pool from the dual-view image, including:

[0033] Based on the mapping relationship between parallax, depth, and three-dimensional coordinates, the three-dimensional shape of the molten pool is obtained by substituting the parallax or depth information of the dual-view image of the molten pool into the basic geometric relationship.

[0034] Secondly, the present invention also provides a coaxial online monitoring system for the three-dimensional topography of a molten pool based on binocular vision, comprising:

[0035] Binocular vision optical path and processing optical path to acquire dual-view images of the molten pool in real time;

[0036] The processing optical path includes a laser, a beam expander, a galvanometer, and a field mirror;

[0037] The binocular vision optical path includes five mirrors, two right-angle prisms, a lens group, and a camera.

[0038] According to the present invention, a coaxial online monitoring system for the three-dimensional topography of a molten pool based on binocular vision is provided. The processing optical path includes a laser, a beam expander, a long-pass dichroic mirror, a galvanometer, and a field mirror, comprising:

[0039] The laser emits a laser beam to the beam expander. The laser beam passes through the beam expander and the long-pass dichroic mirror, and is reflected by the galvanometer and focused by the field mirror to melt the metal powder and form a molten pool.

[0040] The molten pool light signal sequentially travels along the field mirror and the galvanometer, and is reflected by the long-pass dichroic mirror to the binocular vision optical path.

[0041] According to the present invention, a coaxial online monitoring system for the three-dimensional topography of a molten pool based on binocular vision is provided. The binocular vision optical path includes five mirrors, two right-angle prisms, a lens group, and a camera, comprising:

[0042] The molten pool light signal is reflected by the first reflector to the two right-angle faces of the first right-angle prism in the right-angle prism to form a first molten pool light signal and a second molten pool light signal. The first molten pool light signal and the second molten pool light signal are reflected and then reach the second reflector and the third reflector.

[0043] The first molten pool optical signal and the second molten pool optical signal are reflected by the second reflector and then reach the fourth reflector. The third reflector reflects the signal and then reaches the fifth reflector, which reflects the signal onto the two right-angled faces of the second right-angled prism in the right-angled prism.

[0044] The reflected first and second molten pool light signals are focused onto the camera by the lens group to form a dual-view image of the molten pool.

[0045] Thirdly, the present invention also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the coaxial online monitoring method for the three-dimensional topography of a molten pool based on binocular vision as described above.

[0046] The present invention provides a binocular vision-based coaxial online monitoring method and system for 3D molten pool topography, which enables the acquisition of binocular vision images of the molten pool in the narrow coaxial system of laser additive manufacturing for subsequent 3D molten pool reconstruction, and has strong adaptability. High-precision disparity reconstruction is achieved by establishing an unsupervised adaptive weight residual network. This network does not require manually labeled learning labels or set empirical loss weights, and can adaptively learn the disparity information of two viewpoint images. Attached Figure Description

[0047] To more clearly illustrate the technical solutions in this invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this invention. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0048] Figure 1 is a flowchart illustrating the coaxial online monitoring method for the three-dimensional morphology of a molten pool based on binocular vision provided by the present invention.

[0049] Figure 2 is a schematic diagram of the processing optical path and adjustable binocular vision optical path structure provided by the present invention;

[0050] Figure 3 is a schematic diagram of the unsupervised adaptive loss neural network structure provided by the present invention;

[0051] Figure 4 is a schematic diagram of the Res-Unet structure provided by the present invention;

[0052] Figure 5 is a schematic diagram of parallax and depth mapping provided by the present invention;

[0053] Figure 6 is a schematic diagram of the calibration of the equivalent baseline b and focal length f provided by the present invention;

[0054] Figure 7 shows the 3D reconstruction accuracy of the binocular vision system provided by the present invention;

[0055] Figure 8 is a schematic diagram of the three-dimensional morphology of the molten pool provided by the present invention;

[0056] Figure 9 is a schematic diagram of the structure of the electronic device provided by the present invention.

[0057] Figure label:

[0058] 1: Reflector; 2: Right-angle prism; 3: Lens group; 4: Camera;

[0059] 5: Long-pass dichroic mirror; 6: Beam expander; 7: Laser; 8: Galvanometer;

[0060] 9: Scene shot. Embodiments of the present invention

[0061] To make the objectives, technical solutions, and advantages of this invention clearer, the technical solutions of this invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this invention. All other embodiments obtained by those skilled in the art based on the embodiments of this invention without creative effort are within the scope of protection of this invention.

[0062] Figure 1 is a flowchart illustrating the coaxial online monitoring method for the three-dimensional topography of a melt pool based on binocular vision provided in an embodiment of the present invention. As shown in Figure 1, it includes:

[0063] Step 100: Obtain dual-view images of the molten pool;

[0064] Step 200: Input the dual-view image of the molten pool into an unsupervised adaptive loss neural network to obtain molten pool disparity information;

[0065] Step 300: Calibrate the binocular vision monitoring system using a checkerboard pattern to obtain system parameter information;

[0066] Step 400: Obtain the depth information of the molten pool dual-view image based on the molten pool parallax information and the parameter information, and reconstruct the three-dimensional shape of the molten pool based on the molten pool dual-view image depth information.

[0067] This invention provides a method and system for online monitoring of the three-dimensional shape of a laser additive manufacturing molten pool based on binocular vision, including hardware and algorithms. The hardware includes two optical paths: a processing optical path and a binocular vision optical path. The algorithm includes an unsupervised adaptive weighted-loss Res_Unet (UAWLRU) and a calibration method, specifically including:

[0068] Step 1: Design of the coaxial binocular vision optical path

[0069] As shown in Figure 2, the processing optical path and the binocular vision optical path are used to acquire binocular vision images of the molten pool. The processing optical path includes a laser 7, a beam expander 6, a galvanometer 8, and a field mirror 9. The binocular vision optical path includes a long-pass dichroic mirror 5, five reflecting mirrors 1, two right-angle prisms 2, a lens group 3, and a camera 4.

[0070] In this embodiment, laser 7 generates laser light, which passes through beam expander 6 and long-pass dichroic mirror 5, then is reflected by galvanometer 8 and focused by field mirror 9 to melt metal powder and form a molten pool. The molten pool light signal travels along field mirror 9 and galvanometer 8, and is reflected by long-pass dichroic mirror 5 to the binocular vision imaging system. Further, the molten pool light signal passes through mirrors to the two right-angled surfaces of right-angle prism 2, and after reflection, reaches two mirrors 1. It then passes through two other mirrors 1 and is reflected by another right-angle prism 2, outputting two beams of molten pool light signals with dual viewing angles. Finally, these beams are focused by a lens group to the camera sensor to record the dual-view image of the molten pool.

[0071] Step 2: Construct an unsupervised adaptive loss neural network, UAWLRU

[0072] An unsupervised neural network is a type of neural network trained using unsupervised learning methods. Its goal is to automatically learn the structure and patterns in data by minimizing the loss function, without requiring any manually labeled training samples.

[0073] Figure 3 is a schematic diagram of the unsupervised adaptive loss neural network structure in an embodiment of the present invention, including a disparity extraction neural network Res_Unet, a view mapping module Unwrap, and a loss function module. Res_Unet obtains disparity information of the melt pool dual-view images through feature learning. Res_Unet consists of three parts: a residual module, a max pooling module, and an upsampling module. Unwrap implements the function of obtaining the predicted view pool images of view 2 and view 1 by matrix shifting and rotation operations on the melt pool images of view 1 and view 2 and the predicted disparities of view 1 and view 2, respectively. The loss function consists of three parts: appearance matching loss, disparity smoothness loss, and disparity consistency loss of the two views. The loss function module implements the functions of appearance matching loss (calculating the error between the predicted melt pool images of view 1 and view 2 and the original image), disparity smoothness loss (calculating the gradient of the disparity images of the two views), and disparity consistency loss (calculating the error value by inverting one disparity and subtracting it from the other view).

[0074] The principle of UAWLRU is to input two-view images of the melt pool to obtain disparity information, predict the two-view images of the melt pool based on the disparity information, and calculate appearance matching loss, disparity smoothness loss, and disparity consistency loss between the two views through a loss function. This loss is then backpropagated, allowing the network to continuously learn and reduce errors. Learning stops after reaching the algorithm's iteration count N. Specifically, two-view melt pool images are input to Res_Unet to predict the disparity images of the corresponding views. The melt pool image of view 1 and its corresponding disparity are processed through the view mapping module Unwrap to obtain the predicted melt pool image of view 2. Similarly, the melt pool image of view 2 and its corresponding disparity are processed through the view mapping module Unwrap to obtain the predicted melt pool image of view 1. Specifically, the principle of the view mapping module Unwrap is to combine one view image with disparity information and obtain another view image through matrix translation and rotation operations. Here, disparity is the pixel position offset of the same object point in the two view images. Further, the error is calculated using a loss function and backpropagated to achieve iterative learning of the network. Specifically, the loss between view 1 and the predicted view 1 is denoted as L. a Let L be the loss between viewpoint 2 and the predicted viewpoint 2. a The parallax smoothness loss for viewpoint 1 and viewpoint 2 is denoted as L, respectively. d L d The parallax consistency loss between viewpoint 1 and viewpoint 2 is denoted as L. c At this point, all losses can be defined as L. t =α(L a +L a ')+ (L d +L d ')+σL c Among them, α, σ represents the weight of each loss component in the total loss, and α represents the weight of each loss component in the total loss. The σ parameter information is obtained through algorithm iteration. Specifically, the algorithm uses Bayesian theory to treat the loss weights as uncertainty estimation parameters, with lower weights for tasks with high uncertainty and higher weights for tasks with low uncertainty. When the algorithm reaches N iterations, the network stops learning.

[0075] Furthermore, the Res-Unet proposed in this invention is used to obtain disparity information of the molten pool dual-view image. Figure 4 is a schematic diagram of the Res-Unet structure. The molten pool dual-view image undergoes feature extraction through a residual module, followed by four rounds of max pooling and residual network to reduce the image size and extract features again. Each time, the image size is halved and the depth is doubled. The downsampled feature image then passes through four rounds of residual and upsampling modules, where the image size is doubled and the depth is halved each time. This upsampled feature image is then concatenated with the features from the downsampling process on the left and input into the subsequent residual and upsampling modules, finally outputting a dual-view disparity image.

[0076] Furthermore, the residual module consists of three convolutional layers: the first layer has a 1×1 kernel, and the following two layers have 3×3 kernels. After the second convolution, batch normalization (BN) and Leaky ReLU (LR) are performed, followed by a third layer where the output features are concatenated with the original input. Then, LR is applied for activation. BN, or batch normalization, prevents overfitting of internal features and acts as a regularization mechanism. LR is an activation function that enhances the model's ability to extract non-linear features.

[0077] Step 3: Use the checkerboard calibration optical monitoring system to obtain the baseline b and focal length f.

[0078] In this embodiment, the calibration method uses a checkerboard pattern to calibrate the binocular vision monitoring system to obtain two parameters: baseline and focal length. The calibration device includes a checkerboard pattern and a binocular vision online monitoring system.

[0079] Figure 5 is a schematic diagram of parallax and depth mapping according to an embodiment of the present invention. The molten pool light signal is focused by the main lens to form images from two different viewpoints. Let the center projections of the two main lenses at viewpoints 1 and 2 be O... l O r Let point A on the molten pool be located at position A in the molten pool images at viewpoints 1 and 2. i +1 A i And A i +1 A i Location and O l O rThe differences in their locations are x1 and x2, respectively. Furthermore, let the distance from object point A to the main lens be z, and the distance from the main lens to the sensor be f. l With O r The spacing is b. Here, f and b are the equivalent focal length and baseline of the binocular system, respectively. Assuming the image center is positive upwards and negative downwards, based on geometric relationships, we obtain:

[0080] (1)

[0081] Where Δx is the disparity of object point A.

[0082] As can be seen from formula (1), parallax is inversely proportional to depth. The greater the parallax, the smaller the distance.

[0083] Figure 6 is a schematic diagram of the calibration of the equivalent baseline b and focal length f in an embodiment of the present invention. Solving for the two parameters b and f separately based on formula (1) would lead to a complex calibration process, and the error of each parameter would increase the system error. In this embodiment of the present invention, the product of b and f is directly recorded as a single parameter for calibration based on formula (1). The specific calibration process is as follows:

[0084] Three identical standard checkerboard grids are placed on the substrate of the laser additive manufacturing equipment. The thickness of the checkerboard grid is d. One checkerboard grid is placed alone as scene A, and the two checkerboard grids are stacked together as scene B. The height difference between the two is d.

[0085] The parallax information Δx' is obtained from the dual-view image of the chessboard in scene B, and the corresponding imaging distance is z'. Similarly, the parallax information Δx'' of the chessboard in scene A is obtained, and the corresponding imaging distance is z'+d.

[0086] Substituting the parallax and corresponding imaging distance of scene A and scene B into the basic geometric relationship, i.e., formula (1), we obtain the baseline and focal length parameter information of the system.

[0087] Based on formula (1), the following formula can be derived:

[0088] (2)

[0089] Specifically, the bf parameter information of the binocular system is obtained by using formula (2) to find the positions of the same feature points of the chessboard grid in scene A and scene B, i.e., the vertices of the chessboard grid cells, in the images of view 1 and view 2.

[0090] Step 4: Based on the mapping relationship between disparity and depth, substitute the parameter information obtained in Step 2 and Step 3, namely disparity information, baseline and focal length, into the mapping relationship to obtain the depth information of the molten pool dual-view image, and reconstruct the three-dimensional shape of the molten pool according to formula (1).

[0091] In one embodiment, the optical monitoring system is calibrated using the scheme proposed in this invention, and the calculated bf is 1150mm. pixel.

[0092] In this embodiment, two standard gauge blocks are lapped together in a flat crystal, forming a height difference of 500 μm. The reconstructed three-dimensional morphology using this invention is shown in Figure 7. The measured depth is 510.5 μm, the lateral error is 10 μm, and the overall three-dimensional morphology error is... .

[0093] In this embodiment of the invention, a coaxial device is used to capture dual-view images of the molten pool. These dual-view images are then input into Res_Unet to obtain molten pool parallax information. Combined with calibration parameters, the three-dimensional morphology of the molten pool is obtained, and the final result is shown in Figure 8. Specifically, the UAWLRU iteration count N is set to 500, resulting in a sequence of nine molten pool three-dimensional morphology images spaced 30ms apart. The length and width of the molten pool are distributed between 0 and 400 μm. With the substrate height as the zero point, the relative height is distributed between -150 and 100 μm. A keyhole in the molten pool can be observed in Figure 8, with a depth distributed between 100 and 150 μm. Irregular protrusions are displayed around the keyhole. Furthermore, Figure 8 marks the maximum and minimum heights of the molten pool. The highest molten pool in the sequence is 63.55 μm, and the deepest is 124.73 μm.

[0094] Figure 9 illustrates a schematic diagram of the physical structure of an electronic device. As shown in Figure 9, the electronic device may include: a processor 810, a communication interface 820, a memory 830, and a communication bus 840. The processor 810, communication interface 820, and memory 830 communicate with each other via the communication bus 840. The processor 810 can call logical instructions in the memory 830 to execute a coaxial online monitoring method for the three-dimensional shape of a molten pool based on binocular vision. This method includes: acquiring dual-view images of the molten pool; inputting the dual-view images of the molten pool into an unsupervised adaptive loss neural network to obtain molten pool disparity information; calibrating the binocular vision monitoring system using a checkerboard pattern to obtain system parameter information; obtaining depth information of the dual-view images of the molten pool based on the disparity information and the parameter information; and establishing the three-dimensional shape of the molten pool based on the depth information of the dual-view images of the molten pool.

[0095] Furthermore, the logical instructions in the aforementioned memory 830 can be implemented as software functional units and, when sold or used as independent products, can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0096] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Those skilled in the art can understand and implement this without any creative effort.

[0097] Through the above description of the embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus necessary general-purpose hardware platforms, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solutions, in essence or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods described in the various embodiments or some parts of the embodiments.

[0098] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for coaxial online monitoring of the three-dimensional morphology of a molten pool based on binocular vision, characterized in that, include: Obtain dual-view images of the molten pool; The dual-view image of the molten pool is input into an unsupervised adaptive loss neural network to obtain molten pool disparity information; The binocular vision monitoring system was calibrated using a checkerboard pattern to obtain binocular system parameter information; Based on the molten pool parallax information and the parameter information, the depth information of the molten pool dual-view image is obtained, and the three-dimensional shape of the molten pool is reconstructed based on the depth information of the molten pool dual-view image.

2. The method for coaxial online monitoring of molten pool three-dimensional topography based on binocular vision according to claim 1, characterized in that, The dual-view images of the molten pool are input into an unsupervised adaptive loss neural network to obtain molten pool disparity information, including: The unsupervised adaptive loss neural network is defined as including the disparity extraction neural network Res_Unet, the view mapping module Unwrap, and the loss function module. The molten pool dual-view image includes the molten pool image of view 1 and the molten pool image of view 2. The molten pool image from viewpoint 1 and the molten pool image from viewpoint 2 are input into the disparity extraction neural network Res_Unet to obtain the molten pool disparity image from viewpoint 1 and the molten pool disparity image from viewpoint 2. The viewpoint 1 molten pool image and the viewpoint 1 molten pool parallax image are used to obtain the predicted viewpoint 2 molten pool image through the viewpoint mapping module Unwrap; the viewpoint 2 molten pool image and the viewpoint 2 molten pool parallax image are used to obtain the predicted viewpoint 1 molten pool image through the viewpoint mapping module Unwrap; the viewpoint mapping module Unwrap includes matrix translation and rotation operations; The loss function module calculates the appearance matching loss between the two viewpoint melt pool images and the predicted melt pool images, the smoothness loss of the two viewpoint melt pool disparity images, and the consistency loss of the two viewpoint melt pool disparity images through the loss function. The loss is backpropagated so that the network can continuously learn to reduce the loss. When the preset number of iterations is reached, the learning stops and the melt pool disparity information is obtained. The appearance matching loss includes the appearance matching loss between the melt pool image of viewpoint 1 and the predicted melt pool image of viewpoint 1, and the appearance matching loss between the melt pool image of viewpoint 2 and the predicted melt pool image of viewpoint 2; the smoothness loss of the two viewpoint melt pool disparity images is the gradient of each disparity image; the consistency loss is the error value obtained by inverting the melt pool disparity image of one viewpoint and subtracting it from the melt pool disparity image of the other viewpoint.

3. The method for coaxial online monitoring of molten pool three-dimensional topography based on binocular vision according to claim 2, characterized in that, The molten pool image from viewpoint 1 and the molten pool image from viewpoint 2 are input into the disparity extraction neural network Res_Unet to obtain the molten pool disparity image from viewpoint 1 and the molten pool disparity image from viewpoint 2, including: The disparity extraction neural network Res_Unet is defined to include a residual module, a max pooling module, and an upsampling module; The residual module includes three convolutional layers. The first convolutional layer has a 1×1 kernel, and the second and third convolutional layers both have 3×3 kernels. The second and third convolutional layers include batch normalization and activation operations. The third convolutional layer includes connection with the original input features and activation operations. The max pooling module halves the image size while keeping the number of layers unchanged, and the upsampling module enlarges the image size while varying the number of layers according to a setting. The molten pool image from viewpoint 1 and the molten pool image from viewpoint 2 are sequentially input into the residual module, the max pooling module, and the upsampling module, and the molten pool parallax image from viewpoint 1 and the molten pool parallax image from viewpoint 2 are output.

4. The method for coaxial online monitoring of molten pool three-dimensional topography based on binocular vision according to claim 3, characterized in that, The loss function includes appearance matching loss, disparity smoothness loss, and disparity consistency loss; The appearance matching loss includes the appearance matching loss between the molten pool image at viewpoint 1 and the predicted molten pool image at viewpoint 1, and the appearance matching loss between the molten pool image at viewpoint 2 and the predicted molten pool image at viewpoint 2. The disparity smoothness loss includes the gradient of the molten pool disparity image at viewpoint 1 and the gradient of the molten pool disparity image at viewpoint 2. The parallax consistency loss includes the error value obtained by subtracting the parallax image of the melt pool from ... The appearance matching loss, the disparity smoothness loss, and the disparity consistency are weighted and summed to obtain the loss function.

5. The method for coaxial online monitoring of molten pool three-dimensional topography based on binocular vision according to claim 2, characterized in that, The binocular vision monitoring system was calibrated using a checkerboard pattern to obtain system parameter information, including: Using the distance from the main lens to the sensor as the focal length, the center distance as the baseline, and the difference between the first and second differences as the parallax, a basic geometric relationship is established. Three identical standard checkerboard grids are placed on a laser additive manufacturing substrate. One checkerboard grid is placed alone as scene A, and the two checkerboard grids are stacked together as scene B. The height difference between the two scenes is the thickness of one checkerboard grid. Determine the parallax information of the checkerboard grid in scene A and scene B; Obtain the distance between the chessboard grid in scene B and the main lens. Add the thickness of the chessboard grid to the distance between the chessboard grid and the main lens to obtain the distance between the chessboard grid in scene A and the main lens. Substituting the parallax and corresponding imaging distance of scene A and scene B into the basic geometric relationship, the baseline and focal length parameters of the system are obtained.

6. The method for coaxial online monitoring of molten pool three-dimensional topography based on binocular vision according to claim 5, characterized in that, Based on the molten pool parallax information and the parameter information, depth information of the molten pool dual-view image is obtained. Based on the depth information of the molten pool dual-view image, a three-dimensional morphology of the molten pool is established, including: Based on the mapping relationship between parallax and depth, the three-dimensional shape of the molten pool is obtained by substituting the depth information of the dual-view image of the molten pool into the basic geometric relationship.

7. A binocular vision-based coaxial online monitoring system for the three-dimensional topography of a molten pool, used to execute the binocular vision-based coaxial online monitoring method for the three-dimensional topography of a molten pool as described in any one of claims 1 to 6, characterized in that, include: Binocular vision optical path and processing optical path to acquire dual-view images of the molten pool in real time; The processing optical path includes a laser, a beam expander, a galvanometer, and a field mirror; The binocular vision optical path includes five mirrors, two right-angle prisms, a lens group, and a camera.

8. The coaxial online monitoring system for the three-dimensional topography of a molten pool based on binocular vision according to claim 7, characterized in that, The processing optical path includes a laser, a beam expander, a long-pass dichroic mirror, a galvanometer, and a field mirror, comprising: The laser sends a laser beam to the beam expander. The laser beam passes through the beam expander and the long-pass dichroic mirror, and is reflected by the galvanometer and focused by the field mirror to melt the metal powder and form a molten pool. The molten pool light signal in the molten pool sequentially travels along the field mirror and the galvanometer, and is reflected by the long-pass dichroic mirror to the binocular vision optical path.

9. The coaxial online monitoring system for the three-dimensional topography of a molten pool based on binocular vision according to claim 8, characterized in that, The binocular vision optical path includes five mirrors, two right-angle prisms, a lens group, and a camera, comprising: The molten pool light signal is reflected by the first reflector to the two right-angle faces of the first right-angle prism in the right-angle prism to form a first molten pool light signal and a second molten pool light signal. The first molten pool light signal and the second molten pool light signal are reflected and then reach the second reflector and the third reflector. The first molten pool optical signal and the second molten pool optical signal are reflected by the second reflector and then reach the fourth reflector. The third reflector reflects the signal and then reaches the fifth reflector, which reflects the signal onto the two right-angled faces of the second right-angled prism in the right-angled prism. The reflected first and second molten pool light signals are focused onto the camera by the lens group to form a dual-view image of the molten pool.

10. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the program, it implements the coaxial online monitoring method for the three-dimensional morphology of the molten pool based on binocular vision as described in any one of claims 1 to 6.

Images