Coaxial on-line monitoring method and system for three-dimensional morphology of molten pool based on binocular vision

Abstract

The application provides a kind of based on binocular vision's molten pool three-dimensional topography coaxial on-line monitoring method and system, comprising: obtaining molten pool double perspective image;The molten pool double perspective image is input into unsupervised adaptive loss neural network, and molten pool parallax information is obtained;Chessboard is used to calibrate the binocular vision monitoring system, and system parameter information is obtained;According to the molten pool parallax information and the calibrated parameter information, molten pool double perspective image depth information is obtained, and the molten pool three-dimensional topography is reconstructed based on the molten pool double perspective image depth information.The three-dimensional reconstruction error of the system is 10.6 μm by standard block verification, and the molten pool three-dimensional topography and spoon hole depth information are obtained.The application uses a single camera for on-line monitoring, and the structure is simple, which can obtain the binocular vision image of the molten pool in the narrow coaxial system of laser additive manufacturing, achieve high-precision parallax prediction, and realize high-precision on-line monitoring of the molten pool three-dimensional topography.

Description

Technical Field

[0001] This invention relates to the field of industrial vision technology, and in particular to a method and system for coaxial online monitoring of the three-dimensional shape of a molten pool based on binocular vision. Background Technology

[0002] In laser additive manufacturing, several key process variables can directly or indirectly represent the quality of the parts. The molten pool is the primary source of dynamic information in the laser additive manufacturing process, and its surface morphology is directly related to the quality of the manufactured parts. Therefore, monitoring the surface morphology characteristics of the molten pool is of great significance for achieving high-quality laser additive manufacturing.

[0003] For online monitoring of molten pool geometry, current methods primarily rely on commercially available equipment such as high-speed cameras and infrared thermal imagers to acquire two-dimensional information about the length and width of the molten pool, failing to capture its three-dimensional morphological features. To obtain these features, the mainstream approach involves using interferometry and binocular vision to determine the relative height of the molten pool for online monitoring. However, coaxial online coherent imaging methods struggle to capture the three-dimensional morphology of the entire molten pool region. While holographic interferometry can acquire the molten pool's three-dimensional morphology, it is difficult to apply to the complex coaxial systems of actual laser additive manufacturing processes. Traditional binocular vision sensing methods always use two independent cameras to detect the target from different angles, making it difficult to ensure good synchronization between the two cameras. Furthermore, using two cameras increases system complexity and equipment cost. Single-camera binocular vision sensing systems based on the principle of biprism refraction can capture the molten pool's three-dimensional morphology in real time and have a simple structure; however, due to the fixed viewing angle, this system is unsuitable for the narrow coaxial systems of laser additive manufacturing.

[0004] In summary, there is an urgent need to develop a simple, high-precision, and coaxial online monitoring method for the three-dimensional morphology of the molten pool that is applicable to laser additive manufacturing. Summary of the Invention

[0005] This invention provides a method and system for coaxial online monitoring of the three-dimensional morphology of a molten pool based on binocular vision, which solves the defects in the prior art. It has a simple structure and can acquire binocular vision images of the molten pool in the narrow coaxial system of laser additive manufacturing, thereby achieving high-precision online monitoring of the three-dimensional morphology of the molten pool.

[0006] In a first aspect, the present invention provides a method for coaxial online monitoring of the three-dimensional topography of a molten pool based on binocular vision, comprising:

[0007] Design a binocular vision system to coaxially acquire dual-view images of the molten pool;

[0008] The dual-view image of the molten pool is input into an unsupervised adaptive loss neural network to obtain molten pool disparity information;

[0009] The binocular vision monitoring system was calibrated using a checkerboard pattern to obtain the system's baseline and focal length parameters.

[0010] Based on the molten pool parallax information and the calibration monitoring system parameters, the depth information of the molten pool dual-view image is obtained, and the three-dimensional shape of the molten pool is quickly reconstructed based on the depth information of the molten pool dual-view image.

[0011] According to the present invention, a coaxial online monitoring method for the three-dimensional topography of a molten pool based on binocular vision is provided, wherein the dual-view images of the molten pool are input into an unsupervised adaptive loss neural network to obtain molten pool disparity information, including:

[0012] An unsupervised adaptive loss neural network is determined, including a disparity extraction neural network Res_Unet, a view mapping module Unwrap, and a loss function module. The melt pool dual-view image includes melt pool images of view 1 and view 2.

[0013] The molten pool images of viewpoint 1 and viewpoint 2 are input into the disparity extraction neural network Res_Unet to obtain the molten pool disparity images of viewpoint 1 and viewpoint 2.

[0014] The melt pool image of viewpoint 1 and the melt pool parallax image of viewpoint 1 are used to obtain the melt pool image of predicted viewpoint 2 through the viewpoint mapping module Unwrap; the melt pool image of viewpoint 2 and the melt pool parallax image of viewpoint 2 are used to obtain the melt pool image of predicted viewpoint 1 through the viewpoint mapping module Unwrap; the viewpoint mapping module Unwrap includes matrix translation and rotation operations;

[0015] The loss function module calculates the appearance matching loss between the two viewpoint melt pool images and the predicted melt pool images, the smoothness loss of the two viewpoint melt pool disparity images, and the consistency loss of the two viewpoint melt pool disparity images using the loss function of the loss function module. The loss is then backpropagated so that the network can continuously learn to reduce the loss. When the preset number of iterations is reached, the learning stops, and the melt pool disparity information is obtained.

[0016] Furthermore, the appearance matching loss between the two viewpoint melt pool images and the predicted two viewpoint melt pool images includes the appearance matching loss between the melt pool image at viewpoint 1 and the predicted melt pool image at viewpoint 1, and the appearance matching loss between the melt pool image at viewpoint 2 and the predicted melt pool image at viewpoint 2; the smoothness loss of the two viewpoint melt pool disparity images is the gradient of each disparity image; the consistency loss is the error value obtained by inverting the melt pool disparity image at one viewpoint and subtracting it from the melt pool disparity image at the other viewpoint.

[0017] According to the present invention, a coaxial online monitoring method for the three-dimensional topography of a molten pool based on binocular vision is provided, wherein the molten pool image at viewpoint 1 and the molten pool image at viewpoint 2 are input into the disparity extraction neural network Res_Unet to obtain the molten pool disparity image at viewpoint 1 and the molten pool disparity image at viewpoint 2, including:

[0018] The disparity extraction neural network Res_Unet is defined to include a residual module, a max pooling module, and an upsampling module;

[0019] The residual module includes three convolutional layers. The first convolutional layer has a 1×1 kernel, and the second and third convolutional layers both have 3×3 kernels. The second and third convolutional layers include batch normalization and activation operations. The third convolutional layer includes connection with the original input features and activation operations.

[0020] The max pooling module halves the image size while keeping the number of layers unchanged, and the upsampling module enlarges the image size while varying the number of layers according to a setting.

[0021] The molten pool image from viewpoint 1 and the molten pool image from viewpoint 2 are sequentially input into the residual module, the max pooling module, and the upsampling module, and the predicted molten pool disparity image from viewpoint 1 and the predicted molten pool disparity image from viewpoint 2 are output.

[0022] According to the present invention, a coaxial online monitoring method for the three-dimensional shape of a molten pool based on binocular vision is provided, wherein the loss function includes appearance matching loss, disparity smoothness loss and disparity consistency loss;

[0023] The appearance matching loss includes the appearance matching loss between the melt pool image at viewpoint 1 and the predicted melt pool image at viewpoint 1, and the appearance matching loss between the melt pool image at viewpoint 2 and the predicted melt pool image at viewpoint 2.

[0024] The disparity smoothness loss includes the gradient of the molten pool disparity image at viewpoint 1 and the gradient of the molten pool disparity image at viewpoint 2.

[0025] The parallax consistency loss is the loss value obtained by subtracting the parallax image of the melt pool from the parallax image of the melt pool from the parallax image of the melt pool from the parallax image of the melt pool in two viewpoints.

[0026] The appearance matching loss, the disparity smoothness loss, and the consistency of the disparity images of the two viewpoints are weighted and summed to obtain the loss function.

[0027] According to the present invention, a coaxial online monitoring method for the three-dimensional topography of a molten pool based on binocular vision is provided, which uses a checkerboard pattern to calibrate the binocular vision monitoring system and obtain system parameter information, including:

[0028] Using the distance from the main lens to the sensor as the focal length, the center distance as the baseline, and the difference between the first and second differences as the parallax, a basic geometric relationship is established.

[0029] Three identical standard checkerboard patterns are placed on the substrate of the laser additive manufacturing equipment. The thickness of the checkerboard patterns is [missing information]. d One chessboard square placed alone constitutes scene A, and two chessboard squares stacked together constitute scene B. The height difference between the two is... d ;

[0030] Disparity information is obtained from the dual-view images of the chessboard pattern in scene B. The imaging distance corresponding to the disparity is: z Similarly, the imaging distance of the chessboard pattern in scene A is obtained as follows: z '+ d ;

[0031] Substituting the parallax and corresponding imaging distance of scene A and scene B into the basic geometric relationship, the baseline and focal length parameter information of the system are obtained;

[0032] According to the present invention, a coaxial online monitoring method for the three-dimensional shape of a molten pool based on binocular vision is provided, which obtains depth information of a molten pool from a dual-view image based on the molten pool disparity information and the parameter information, and reconstructs the three-dimensional shape of the molten pool based on the depth information of the molten pool from the dual-view image, including:

[0033] Based on the mapping relationship between parallax, depth, and three-dimensional coordinates, the three-dimensional shape of the molten pool is obtained by substituting the parallax or depth information of the dual-view image of the molten pool into the basic geometric relationship.

[0034] Secondly, the present invention also provides a coaxial online monitoring system for the three-dimensional topography of a molten pool based on binocular vision, comprising:

[0035] Binocular vision optical path and processing optical path to acquire dual-view images of the molten pool in real time;

[0036] The processing optical path includes a laser, a beam expander, a galvanometer, and a field mirror;

[0037] The binocular vision optical path includes five mirrors, two right-angle prisms, a lens group, and a camera.

[0038] According to the present invention, a coaxial online monitoring system for the three-dimensional topography of a molten pool based on binocular vision is provided. The processing optical path includes a laser, a beam expander, a long-pass dichroic mirror, a galvanometer, and a field mirror, comprising:

[0039] The laser sends a laser beam to the beam expander. The laser beam passes through the beam expander and the long-pass dichroic mirror, and is reflected by the galvanometer and focused by the field mirror to melt the metal powder and form a molten pool.

[0040] The molten pool light signal in the molten pool sequentially travels along the field mirror and the galvanometer, and is reflected by the long-pass dichroic mirror to the binocular vision optical path.

[0041] According to the present invention, a coaxial online monitoring system for the three-dimensional topography of a molten pool based on binocular vision is provided. The binocular vision optical path includes five mirrors, two right-angle prisms, a lens group, and a camera, comprising:

[0042] The molten pool light signal is reflected by the first reflector to the two right-angle faces of the first right-angle prism in the right-angle prism to form a first molten pool light signal and a second molten pool light signal. The first molten pool light signal and the second molten pool light signal are reflected and then reach the second reflector and the third reflector.

[0043] The first molten pool optical signal and the second molten pool optical signal are reflected by the second reflector and then reach the fourth reflector. The third reflector reflects the signal and then reaches the fifth reflector, which reflects the signal onto the two right-angled faces of the second right-angled prism in the right-angled prism.

[0044] The reflected first and second molten pool light signals are focused onto the camera by the lens group to form a dual-view image of the molten pool.

[0045] Thirdly, the present invention also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the coaxial online monitoring method for the three-dimensional topography of a molten pool based on binocular vision as described above.

[0046] The present invention provides a binocular vision-based coaxial online monitoring method and system for 3D molten pool topography, which enables the acquisition of binocular vision images of the molten pool in the narrow coaxial system of laser additive manufacturing for subsequent 3D molten pool reconstruction, and has strong adaptability. High-precision disparity reconstruction is achieved by establishing an unsupervised adaptive weight residual network. This network does not require manually labeled learning labels or set empirical loss weights, and can adaptively learn the disparity information of two viewpoint images. Attached Figure Description

[0047] To more clearly illustrate the technical solutions in this invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this invention. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0048] Figure 1 This is a flowchart illustrating the coaxial online monitoring method for the three-dimensional morphology of a molten pool based on binocular vision provided by the present invention.

[0049] Figure 2 This is a schematic diagram of the processing optical path and adjustable binocular vision optical path structure provided by the present invention;

[0050] Figure 3 This is a schematic diagram of the unsupervised adaptive loss neural network structure provided by the present invention;

[0051] Figure 4 This is a schematic diagram of the Res-Unet structure provided by the present invention;

[0052] Figure 5 This is a schematic diagram of parallax and depth mapping provided by the present invention;

[0053] Figure 6 This is the equivalent baseline provided by the present invention. b and focal length f Calibration diagram;

[0054] Figure 7 The binocular vision system provided by this invention offers high 3D reconstruction accuracy.

[0055] Figure 8 This is a schematic diagram of the three-dimensional morphology of the molten pool provided by the present invention;

[0056] Figure 9 This is a schematic diagram of the structure of the electronic device provided by the present invention.

[0057] Figure label:

[0058] 1: Reflector; 2: Right-angle prism; 3: Lens group; 4: Camera;

[0059] 5: Long-pass dichroic mirror; 6: Beam expander; 7: Laser; 8: Galvanometer;

[0060] 9: Scene shot. Detailed Implementation

[0061] To make the objectives, technical solutions, and advantages of this invention clearer, the technical solutions of this invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this invention. All other embodiments obtained by those skilled in the art based on the embodiments of this invention without creative effort are within the scope of protection of this invention.

[0062] Figure 1 This is a schematic flowchart of the coaxial online monitoring method for the three-dimensional topography of a molten pool based on binocular vision provided in an embodiment of the present invention, as shown below. Figure 1 As shown, it includes:

[0063] Step 100: Obtain dual-view images of the molten pool;

[0064] Step 200: Input the dual-view image of the molten pool into an unsupervised adaptive loss neural network to obtain molten pool disparity information;

[0065] Step 300: Calibrate the binocular vision monitoring system using a checkerboard pattern to obtain system parameter information;

[0066] Step 400: Obtain the depth information of the molten pool dual-view image based on the molten pool parallax information and the parameter information, and reconstruct the three-dimensional shape of the molten pool based on the molten pool dual-view image depth information.

[0067] This invention provides a method and system for online monitoring of the three-dimensional shape of a laser additive manufacturing molten pool based on binocular vision, including hardware and algorithms. The hardware includes two optical paths: a processing optical path and a binocular vision optical path. The algorithm includes an unsupervised adaptive weighted-loss Res_Unet (UAWLRU) and a calibration method, specifically including:

[0068] Step 1: Design of the coaxial binocular vision optical path

[0069] like Figure 2 As shown, the processing optical path and the binocular vision optical path are used to acquire binocular vision images of the molten pool. The processing optical path includes a laser 7, a beam expander 6, a galvanometer 8, and a field mirror 9. The binocular vision optical path includes a long-pass dichroic mirror 5, five reflecting mirrors 1, two right-angle prisms 2, a lens group 3, and a camera 4.

[0070] In this embodiment, laser 7 generates laser light, which passes through beam expander 6 and long-pass dichroic mirror 5, then is reflected by galvanometer 8 and focused by field mirror 9 to melt metal powder and form a molten pool. The molten pool light signal travels along field mirror 9 and galvanometer 8, and is reflected by long-pass dichroic mirror 5 to the binocular vision imaging system. Further, the light signal reaches the two right-angle surfaces of the right-angle prism 2, is reflected, and then reaches two mirrors 1. After passing through these mirrors 1, the light signal enters two other mirrors 1 and is reflected by another right-angle prism 2, outputting two beams of molten pool light signals with dual viewing angles. Finally, these beams are focused by a lens group to the camera sensor to record the dual-view image of the molten pool.

[0071] Step 2: Construct an unsupervised adaptive loss neural network, UAWLRU.

[0072] An unsupervised neural network is a type of neural network trained using unsupervised learning methods. Its goal is to automatically learn the structure and patterns in data by minimizing the loss function, without requiring any manually labeled training samples.

[0073] Figure 3This is a schematic diagram of the unsupervised adaptive loss neural network structure in an embodiment of the present invention, including a disparity extraction neural network Res_Unet, a view mapping module Unwrap, and a loss function module. Res_Unet obtains disparity information of the melt pool dual-view images through feature learning. Res_Unet consists of three parts: a residual module, a max-pooling module, and an upsampling module (Up2× module). Unwrap implements the function of obtaining the predicted view pool images of view 2 and view 1 by matrix shifting and rotation operations on the melt pool images of view 1 and view 2 and the predicted disparities of view 1 and view 2, respectively. The loss function consists of three parts: appearance matching loss, disparity smoothness loss, and disparity consistency loss between the two views. The loss function module implements the functions of appearance matching loss (calculating the error between the predicted view pool images of view 1 and view 2 and the original image), disparity smoothness loss (calculating the gradient of the disparity images of the two views), and disparity consistency loss (calculating the error value by inverting one disparity and subtracting it from the other view).

[0074] The principle of UAWLRU is to obtain disparity information from the input melt pool dual-view images, predict the melt pool dual-view images based on the disparity information, and calculate appearance matching loss, disparity smoothness loss, and disparity consistency loss between the two views through loss functions. The losses are then backpropagated to allow the network to continuously learn and reduce errors. When the algorithm reaches the required number of iterations... N Then learning stops. Specifically, two viewpoint melt pool images are input into Res_Unet to predict the disparity images of the corresponding viewpoints. The viewpoint 1 melt pool image and its corresponding disparity are processed by the viewpoint mapping module Unwrap to obtain the predicted viewpoint 2 melt pool image. Similarly, the viewpoint 2 melt pool image and its corresponding disparity are processed by the viewpoint mapping module Unwrap to obtain the predicted viewpoint 1 melt pool image. Specifically, the principle of the viewpoint mapping module Unwrap is: combining one viewpoint image with disparity information and obtaining another viewpoint image through matrix translation and rotation operations. Here, disparity is the pixel position offset of the same object point in the two viewpoint images. Further, the error is calculated through a loss function and backpropagation is performed to achieve iterative learning of the network. Specifically, the loss between viewpoint 1 and the predicted viewpoint 1 is denoted as... L a Let the loss between viewpoint 2 and the predicted viewpoint 2 be denoted as L a The parallax smoothness loss for viewpoint 1 and viewpoint 2 is denoted as ', respectively. L d , L d The disparity consistency loss between viewpoint 1 and viewpoint 2 is denoted as '. L c At this point, all losses can be defined as L t =α(L a+L a ' )+ (L d +L d ' ) +σL c .in α , σ Assigning weights to each part of the loss relative to the total loss. α , σ Parameter information is obtained through algorithm iteration. Specifically, the algorithm uses Bayesian theory to treat the weights of the loss as parameters for uncertainty estimation, with lower weights for tasks with high uncertainty and higher weights for tasks with low uncertainty. When the final number of algorithm iterations reaches... N The network stopped learning.

[0075] Furthermore, the Res_Unet proposed in this invention is used to obtain parallax information of dual-view images of the molten pool. Figure 4 This is a schematic diagram of the Res-Unet architecture. The molten pool dual-view image undergoes feature extraction via a residual module. It then passes through four rounds of max pooling and residual network to reduce the image size and extract features again, with the image size halved and the depth doubled each time. The downsampled feature image then passes through four rounds of residual and upsampling modules, with the image size doubling and the depth halved each time. This upsampled image is then concatenated with the features from the downsampling process on the left and input into subsequent residual and upsampling modules, ultimately outputting a dual-view disparity image.

[0076] Furthermore, the residual module consists of three convolutional layers: the first layer has a 1×1 kernel, and the following two layers have 3×3 kernels. After the second convolution, batch normalization (BN) and LeakyReLU (LR) activation are performed, followed by a third layer where the output features are concatenated with the original input. Then, LR activation is applied. BN, or batch normalization, prevents overfitting of internal features and acts as a regularization mechanism. LR is an activation function that enhances the model's ability to extract non-linear features.

[0077] Step 3: Use a checkerboard pattern to calibrate the optical monitoring system and obtain a baseline. b With focal length f

[0078] In this embodiment, the calibration method obtains two parameters, baseline and focal length, by calibrating the binocular vision monitoring system using a checkerboard pattern. The calibration device includes a checkerboard pattern and a binocular vision online monitoring system. Based on the mapping relationship between disparity and depth—specifically, disparity is inversely proportional to depth, with a larger disparity indicating a smaller distance—the calibration is performed.

[0079] Figure 5 This is a schematic diagram of parallax and depth mapping according to an embodiment of the present invention. The molten pool optical signal is focused by the main lens to form images from two different viewpoints. Let the centers of the molten pool images at viewpoints 1 and 2 be respectively... O l , O r Let the position of point A on the molten pool in the molten pool images at viewpoints 1 and 2 be... A i+1 , A i ,and A i+1 , A i Location and O l , O r The differences in location are respectively x 1, x 2. Furthermore, let the distance from object point A to the primary lens be... z The distance from the main lens to the sensor is f , O l and O r Spacing is b .in, f and b These represent the equivalent focal length and baseline of the binocular system, respectively. Assuming the image center is positive when upward and negative when downward, based on geometric relationships, we obtain:

[0080] (1)

[0081] in Δx Let be the parallax of object point A.

[0082] As can be seen from formula (1), parallax is inversely proportional to depth. The greater the parallax, the smaller the distance.

[0083] Figure 6 This is the equivalent baseline of the embodiments of the present invention. b and focal length f The calibration diagram is solved based on formula (1). b and f Having two parameters complicates the calibration process, and the error of each parameter increases the system error. In this invention, the example directly records the result based on formula (1). b andf The product is calibrated using a single parameter. The specific calibration process is as follows:

[0084] Three identical standard checkerboard patterns are placed on the substrate of the laser additive manufacturing equipment. The thickness of the checkerboard patterns is [missing information]. d One chessboard square placed alone constitutes scene A, and two chessboard squares stacked together constitute scene B. The height difference between the two is... d ;

[0085] Parallax information Δ is obtained from the dual-view images of the chessboard pattern in scene B. x ', the corresponding imaging distance is z Similarly, the parallax information Δ of the chessboard grid in scene A is obtained. x '', the corresponding imaging distance is z '+ d ;

[0086] Substituting the parallax and corresponding imaging distance of scene A and scene B into the basic geometric relationship, the baseline and focal length parameter information of the system are obtained;

[0087] Based on formula (1), the following formula can be derived:

[0088] (2)

[0089] Specifically, the positions of the same feature points of the checkerboard grid in scene A and scene B, i.e., the vertices of the checkerboard grid cells, in the images of viewpoint 1 and viewpoint 2 are obtained using formula (2) to obtain the position of the binocular system. bf Parameter information.

[0090] Step 4: Based on the mapping relationship between disparity and depth, substitute the parameter information obtained in Step 2 and Step 3, namely disparity information, baseline and focal length, into the mapping relationship to obtain the depth information of the molten pool dual-view image, and reconstruct the three-dimensional shape of the molten pool according to formula (1).

[0091] In one embodiment, the optical monitoring system is calibrated using the scheme proposed in this invention, and the calculated... bf 1150mm pixel.

[0092] In an embodiment, by laminating two standard gauge blocks together in a flat crystal, creating a height difference of 500 μm, the three-dimensional morphology reconstructed by this invention is as follows: Figure 7 As shown, the measured depth was 510.5 μm, the lateral error was 10 μm, and the overall three-dimensional topography error was [missing value]. .

[0093] In this embodiment of the invention, a coaxial device is used to capture dual-view images of the molten pool. These dual-view images are then input into Res_Unet to obtain molten pool parallax information. Combined with calibration parameters, the three-dimensional morphology of the molten pool is obtained. The final result is as follows: Figure 8 As shown. Specifically, set the number of UAWLRU iterations. N With a value of 500, nine three-dimensional morphology sequences of the molten pool were obtained at 30ms intervals. The length and width of the molten pool were distributed between 0 and 400 μm, and with the substrate height as the zero point, the relative height was distributed between -150 and 100 μm. Figure 8 A keyhole in the molten pool can be observed, with a depth ranging from 100 to 150 μm, and irregular protrusions are visible around the keyhole. Furthermore... Figure 8 The maximum and minimum heights of the molten pool are marked in the sequence. The highest molten pool in the sequence is 63.55 μm, and the deepest molten pool is 124.73 μm.

[0094] Figure 9 An example is a schematic diagram of the physical structure of an electronic device, such as... Figure 9 As shown, the electronic device may include: a processor 810, a communication interface 820, a memory 830, and a communication bus 840, wherein the processor 810, the communication interface 820, and the memory 830 communicate with each other through the communication bus 840. The processor 810 can call logical instructions in the memory 830 to execute a coaxial online monitoring method for the three-dimensional shape of a molten pool based on binocular vision. The method includes: acquiring dual-view images of the molten pool; inputting the dual-view images of the molten pool into an unsupervised adaptive loss neural network to obtain molten pool disparity information; calibrating the binocular vision monitoring system using a checkerboard pattern to obtain system parameter information; obtaining molten pool dual-view image depth information based on the molten pool disparity information and the parameter information; and establishing the three-dimensional shape of the molten pool based on the molten pool dual-view image depth information.

[0095] Furthermore, the logical instructions in the aforementioned memory 830 can be implemented as software functional units and, when sold or used as independent products, can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0096] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Those skilled in the art can understand and implement this without any creative effort.

[0097] Through the above description of the embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus necessary general-purpose hardware platforms, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solutions, in essence or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods described in the various embodiments or some parts of the embodiments.

[0098] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for coaxial online monitoring of the three-dimensional morphology of a molten pool based on binocular vision, characterized in that, include: Obtain dual-view images of the molten pool; The dual-view images of the molten pool are input into an unsupervised adaptive loss neural network to obtain molten pool disparity information, including: The unsupervised adaptive loss neural network is defined as including the disparity extraction neural network Res_Unet, the view mapping module Unwrap, and the loss function module. The molten pool dual-view image includes the molten pool image of view 1 and the molten pool image of view 2. The molten pool image from viewpoint 1 and the molten pool image from viewpoint 2 are input into the disparity extraction neural network Res_Unet to obtain the molten pool disparity image from viewpoint 1 and the molten pool disparity image from viewpoint 2. The viewpoint 1 molten pool image and the viewpoint 1 molten pool parallax image are used to obtain the predicted viewpoint 2 molten pool image through the viewpoint mapping module Unwrap; the viewpoint 2 molten pool image and the viewpoint 2 molten pool parallax image are used to obtain the predicted viewpoint 1 molten pool image through the viewpoint mapping module Unwrap; the viewpoint mapping module Unwrap includes matrix translation and rotation operations; The loss function module calculates the appearance matching loss between the two viewpoint melt pool images and the predicted melt pool images, the smoothness loss of the two viewpoint melt pool disparity images, and the consistency loss of the two viewpoint melt pool disparity images through the loss function. The loss is backpropagated so that the network can continuously learn to reduce the loss. When the preset number of iterations is reached, the learning stops and the melt pool disparity information is obtained. The appearance matching loss includes the appearance matching loss between the melt pool image of viewpoint 1 and the predicted melt pool image of viewpoint 1, and the appearance matching loss between the melt pool image of viewpoint 2 and the predicted melt pool image of viewpoint 2; the smoothness loss of the two viewpoint melt pool disparity images is the gradient of each disparity image; the consistency loss is the error value obtained by inverting the melt pool disparity image of one viewpoint and subtracting it from the melt pool disparity image of the other viewpoint. The molten pool image from viewpoint 1 and the molten pool image from viewpoint 2 are input into the disparity extraction neural network Res_Unet to obtain the molten pool disparity image from viewpoint 1 and the molten pool disparity image from viewpoint 2, including: The disparity extraction neural network Res_Unet is defined to include a residual module, a max pooling module, and an upsampling module; The residual module includes three convolutional layers. The first convolutional layer has a 1×1 kernel, and the second and third convolutional layers both have 3×3 kernels. The second and third convolutional layers include batch normalization and activation operations. The third convolutional layer includes connection with the original input features and activation operations. The max pooling module halves the image size while keeping the number of layers unchanged, and the upsampling module enlarges the image size while varying the number of layers according to a setting. The molten pool image from viewpoint 1 and the molten pool image from viewpoint 2 are sequentially input into the residual module, the max pooling module, and the upsampling module, and the molten pool parallax image from viewpoint 1 and the molten pool parallax image from viewpoint 2 are output. A checkerboard pattern was used to calibrate the binocular vision monitoring system and obtain binocular system parameter information; Based on the molten pool parallax information and the parameter information, the depth information of the molten pool dual-view image is obtained, and the three-dimensional shape of the molten pool is reconstructed based on the molten pool dual-view image depth information. The binocular vision monitoring system includes: Binocular vision optical path and processing optical path to acquire dual-view images of the molten pool in real time; The processing optical path includes a laser, a beam expander, a galvanometer, and a field mirror; The binocular vision optical path includes five mirrors, two right-angle prisms, a lens group, and a camera, comprising: The molten pool light signal is reflected by the first reflecting mirror to the two right-angled faces of the first right-angled prism in the right-angled prism to form a first molten pool light signal and a second molten pool light signal. The first molten pool light signal and the second molten pool light signal are reflected and then reach the second reflecting mirror and the third reflecting mirror. The first molten pool optical signal and the second molten pool optical signal are reflected by the second reflector and then reach the fourth reflector. The third reflector reflects the signal and then reaches the fifth reflector, which reflects the signal onto the two right-angled faces of the second right-angled prism in the right-angled prism. The reflected first and second molten pool light signals are focused onto the camera by the lens group to form a dual-view image of the molten pool.

2. The method for coaxial online monitoring of molten pool three-dimensional topography based on binocular vision according to claim 1, characterized in that, The loss function includes appearance matching loss, disparity smoothness loss, and disparity consistency loss; The appearance matching loss includes the appearance matching loss between the molten pool image at viewpoint 1 and the predicted molten pool image at viewpoint 1, and the appearance matching loss between the molten pool image at viewpoint 2 and the predicted molten pool image at viewpoint 2. The disparity smoothness loss includes the gradient of the molten pool disparity image at viewpoint 1 and the gradient of the molten pool disparity image at viewpoint 2. The parallax consistency loss includes the error value obtained by subtracting the parallax image of the melt pool from ... The appearance matching loss, the disparity smoothness loss, and the disparity consistency are weighted and summed to obtain the loss function.

3. The method for coaxial online monitoring of molten pool three-dimensional topography based on binocular vision according to claim 1, characterized in that, The binocular vision monitoring system was calibrated using a checkerboard pattern to obtain system parameter information, including: Using the distance from the main lens to the sensor as the focal length, the center distance as the baseline, and the difference between the first and second differences as the parallax, a basic geometric relationship is established. Three identical standard checkerboard grids are placed on a laser additive manufacturing substrate. One checkerboard grid is placed alone as scene A, and the two checkerboard grids are stacked together as scene B. The height difference between the two scenes is the thickness of one checkerboard grid. Determine the parallax information of the checkerboard grid in scene A and scene B; Obtain the distance between the chessboard grid in scene B and the main lens. Add the thickness of the chessboard grid to the distance between the chessboard grid and the main lens to obtain the distance between the chessboard grid in scene A and the main lens. Substituting the parallax and corresponding imaging distance of scene A and scene B into the basic geometric relationship, the baseline and focal length parameters of the system are obtained.

4. The method for coaxial online monitoring of molten pool three-dimensional topography based on binocular vision according to claim 3, characterized in that, Based on the molten pool parallax information and the parameter information, depth information of the molten pool dual-view image is obtained. Based on the depth information of the molten pool dual-view image, a three-dimensional morphology of the molten pool is established, including: Based on the mapping relationship between parallax and depth, the three-dimensional shape of the molten pool is obtained by substituting the depth information of the dual-view image of the molten pool into the basic geometric relationship.

5. The method for coaxial online monitoring of the three-dimensional morphology of a molten pool based on binocular vision according to claim 1, characterized in that, The processing optical path includes a laser, a beam expander, a long-pass dichroic mirror, a galvanometer, and a field mirror, comprising: The laser sends a laser beam to the beam expander. The laser beam passes through the beam expander and the long-pass dichroic mirror, and is reflected by the galvanometer and focused by the field mirror to melt the metal powder and form a molten pool. The molten pool light signal in the molten pool sequentially travels along the field mirror and the galvanometer, and is reflected by the long-pass dichroic mirror to the binocular vision optical path.

6. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the program, it implements the coaxial online monitoring method for the three-dimensional topography of the molten pool based on binocular vision as described in any one of claims 1 to 5.

Images