Molten pool three-dimensional shape coaxial online monitoring method and system based on binocular vision

Abstract

The invention provides a molten pool three-dimensional shape coaxial online monitoring method and system based on binocular vision. The method comprises the steps that a molten pool double-view-angle image is acquired; inputting the molten pool double-view image into an unsupervised adaptive loss neural network to obtain molten pool parallax information; a checkerboard is adopted to calibrate the binocular vision monitoring system, and system parameter information is obtained; and according to the molten pool parallax information and the calibrated parameter information, molten pool double-view image depth information is obtained, and the three-dimensional shape of the molten pool is reconstructed based on the molten pool double-view image depth information. According to the method, the three-dimensional reconstruction error of the system is verified to be 10.6 mu m through the standard gauge block, and the three-dimensional shape of the molten pool and the depth information of the keyhole are obtained. The single camera is adopted for online monitoring, the structure is simple, binocular vision images of the molten pool can be obtained in a narrow coaxial system for laser additive manufacturing, high-precision parallax prediction is achieved, and high-precision online monitoring of the three-dimensional shape of the molten pool is achieved.

Description

Technical Field

[0001] The present invention relates to the field of industrial vision technology, and particularly relates to a method and system for coaxial online monitoring of the three-dimensional morphology of a molten pool based on binocular vision. Background Art

[0002] During the laser additive manufacturing process, some key process variables can directly or indirectly represent the quality of the part. The molten pool is the main source of dynamic information in the laser additive manufacturing process, and its surface morphology is directly related to the quality of the manufactured part. Therefore, the monitoring of the surface morphology characteristics of the molten pool is of great significance for realizing high-quality laser additive manufacturing.

[0003] For the online monitoring of the geometric morphology of the molten pool, the current main monitoring method is to obtain two-dimensional information of the length and width of the molten pool through commercial devices such as high-speed cameras and infrared thermal imagers, and the three-dimensional morphology characteristics cannot be obtained. In order to obtain the three-dimensional morphology characteristics of the molten pool, the current mainstream method is to obtain the relative height of the molten pool through interference and binocular vision to realize the online monitoring of its three-dimensional morphology. However, it is difficult to obtain the three-dimensional morphology of the entire molten pool area by the coaxial online coherent imaging method. Although the holographic interference technology obtains the three-dimensional morphology of the molten pool, it is difficult to be applied to the complex coaxial system in the actual laser additive manufacturing process. The traditional binocular vision perception method always uses two independent cameras to detect the target from different angles respectively, and it is difficult to ensure the good synchronization of the two cameras. In addition, the use of these two cameras also increases the complexity of the system and the equipment cost. The single-camera binocular vision sensing system based on the double-prism refraction principle can capture the three-dimensional morphology of the molten pool in real time and has a simple structure, but due to the fixed viewing angle, it is difficult for this system to be applied to the narrow coaxial system of laser additive manufacturing.

[0004] In summary, there is an urgent need to develop a method for coaxial online monitoring of the three-dimensional morphology of a molten pool with a simple structure, high precision, and applicable to laser additive manufacturing. Summary of the Invention

[0005] The present invention provides a method and system for coaxial online monitoring of the three-dimensional morphology of a molten pool based on binocular vision to solve the defects existing in the prior art, with a simple structure, capable of obtaining binocular vision images of the molten pool in the narrow coaxial system of laser additive manufacturing, and realizing high-precision online monitoring of the three-dimensional morphology of the molten pool.

[0006] In a first aspect, the present invention provides a method for coaxial online monitoring of the three-dimensional morphology of a molten pool based on binocular vision, including: Designing a binocular vision system to coaxially obtain dual-view images of the molten pool; Inputting the dual-view images of the molten pool into an unsupervised adaptive loss neural network to obtain the disparity information of the molten pool; Calibrating the binocular vision monitoring system with a checkerboard to obtain the baseline and focal length parameter information of the system; Based on the molten pool parallax information and the parameters of the calibration monitoring system, the depth information of the molten pool dual-view image is obtained, and the three-dimensional morphology of the molten pool is quickly reconstructed based on the depth information of the molten pool dual-view image.

[0007] According to a method for coaxial online monitoring of the three-dimensional morphology of a molten pool based on binocular vision provided by the present invention, inputting the molten pool dual-view image into an unsupervised adaptive loss neural network to obtain molten pool parallax information, including: Determine an unsupervised adaptive loss neural network, including a parallax extraction neural network Res_Unet, a perspective mapping module Unwrap, and a loss function module. The molten pool dual-view image includes molten pool images of perspective 1 and perspective 2; Input the molten pool images of perspective 1 and perspective 2 into the parallax extraction neural network Res_Unet to obtain molten pool parallax images of perspective 1 and perspective 2; Input the molten pool image of perspective 1 and the molten pool parallax image of perspective 1 through the perspective mapping module Unwrap to obtain a predicted molten pool image of perspective 2; input the molten pool image of perspective 2 and the molten pool parallax image of perspective 2 through the perspective mapping module Unwrap to obtain a predicted molten pool image of perspective 1; the perspective mapping module Unwrap includes matrix translation and rotation operations; Calculate the appearance matching loss between the molten pool images of the two perspectives and the predicted molten pool images of the two perspectives, the smoothness loss of the molten pool parallax images of the two perspectives, and the consistency loss of the molten pool parallax images of the two perspectives through the loss function of the loss function module. Backpropagate the loss to enable the network to continuously learn to reduce the loss. When the preset number of algorithm iterations is reached, stop learning to obtain the molten pool parallax information.

[0008] Further, the appearance matching loss between the molten pool images of the two perspectives and the predicted molten pool images of the two perspectives includes the appearance matching loss between the molten pool image of perspective 1 and the predicted molten pool image of perspective 1, and the appearance matching loss between the molten pool image of perspective 2 and the predicted molten pool image of perspective 2; the smoothness loss of the molten pool parallax images of the two perspectives is the gradient of each parallax image; the consistency loss is the error value obtained by subtracting the inverted molten pool parallax image of one perspective from the molten pool parallax image of the other perspective.

[0009] According to a method for coaxial online monitoring of the three-dimensional morphology of a molten pool based on binocular vision provided by the present invention, inputting the molten pool image of perspective 1 and the molten pool image of perspective 2 into the parallax extraction neural network Res_Unet to obtain the molten pool image of perspective 1 and the molten pool parallax image of perspective 2, including: Determine that the parallax extraction neural network Res_Unet includes a residual module, a max pooling module, and an upsampling module; The residual module includes three layers of convolution. The first layer of convolution uses a 1×1 convolution kernel, and the second and third layers of convolution both use 3×3 convolution kernels. A batch normalization operation and an activation operation are included between the second and third layers of convolution, and a connection with the original input feature and an activation operation are included after the third layer of convolution; The max pooling module halves the image size and keeps the number of layers unchanged, and the upsampling module enlarges the image size and changes the number of layers according to settings; The molten pool image of view 1 and the molten pool image of view 2 are sequentially input into the residual module, the max pooling module, and the upsampling module, and the predicted molten pool image of view 1 and the predicted molten pool image of view 2 are output.

[0010] According to a coaxial online monitoring method for the three-dimensional morphology of a molten pool based on binocular vision provided by the present invention, the loss function includes an appearance matching loss, a disparity smoothness loss, and a disparity consistency loss; The appearance matching loss includes the appearance matching loss between the molten pool image of view 1 and the predicted molten pool image of view 1, and the appearance matching loss between the molten pool image of view 2 and the predicted molten pool image of view 2; The disparity smoothness loss includes the gradient of the molten pool disparity image of view 1 and the gradient of the molten pool disparity image of view 2; The disparity consistency loss is the loss value obtained by subtracting the inverted molten pool disparity image of one view from the molten pool disparity image of the other view in the molten pool disparity images of the two views; The appearance matching loss, the disparity smoothness loss, and the consistency of the molten pool disparity images of the two views are weighted and summed to obtain the loss function.

[0011] According to a coaxial online monitoring method for the three-dimensional morphology of a molten pool based on binocular vision provided by the present invention, a checkerboard is used to calibrate the binocular vision monitoring system to obtain system parameter information, including: Taking the distance from the main lens to the sensor as the focal length, the center distance as the baseline, and the difference between the first difference and the second difference as the disparity, a basic geometric relationship is established; Three identical standard checkerboards are placed on the substrate of the laser additive manufacturing equipment, and the thickness of the checkerboard is d One of the checkerboards is placed alone as scene A, and two checkerboards are stacked together as scene B, and the height difference formed by the two is d ; The disparity information is obtained from the binocular images of the checkerboard in scene B, and the imaging distance corresponding to the disparity is z ', and similarly, the imaging distance of the checkerboard in scene A is obtained as z '+ d ; Substitute the parallax of the scene A and scene B and the corresponding imaging distances into the basic geometric relationship to obtain the information on the baseline and focal length parameters of the system; According to a method for coaxial on-line monitoring of the three-dimensional morphology of a molten pool based on binocular vision provided by the present invention, the depth information of the double-view images of the molten pool is obtained according to the molten pool parallax information and the parameter information, and the three-dimensional morphology of the molten pool is reconstructed based on the depth information of the double-view images of the molten pool, including: According to the mapping relationship between the parallax and the depth, use the depth information of the double-view images of the molten pool and substitute it into the basic geometric relationship to obtain the three-dimensional morphology of the molten pool.

[0012] In a second aspect, the present invention also provides a coaxial on-line monitoring system for the three-dimensional morphology of a molten pool based on binocular vision, including: A binocular vision optical path and a processing optical path for obtaining double-view images of the molten pool in real time; The processing optical path includes a laser, a beam expander, a galvanometer, and a field lens; The binocular vision optical path includes five reflectors, two right-angle prisms, a lens group, and a camera.

[0013] According to a coaxial on-line monitoring system for the three-dimensional morphology of a molten pool based on binocular vision provided by the present invention, the processing optical path includes a laser, a beam expander, a long-pass dichroic mirror, a galvanometer, and a field lens, including: The laser emits a laser to the beam expander, and the laser passes through the beam expander and the long-pass dichroic mirror in sequence, and is reflected by the galvanometer and focused by the field lens to melt metal powder to form a molten pool; The optical signal of the molten pool in the molten pool sequentially travels along the field lens and the galvanometer, and is reflected by the long-pass dichroic mirror to the binocular vision optical path.

[0014] According to a coaxial on-line monitoring system for the three-dimensional morphology of a molten pool based on binocular vision provided by the present invention, the binocular vision optical path includes five reflectors, two right-angle prisms, a lens group, and a camera, including: The optical signal of the molten pool is reflected by the first reflector to two right-angle surfaces of the first right-angle prism in the right-angle prism to form a first optical signal of the molten pool and a second optical signal of the molten pool, and the first optical signal of the molten pool and the second optical signal of the molten pool reach the second reflector and the third reflector after reflection; The first optical signal of the molten pool and the second optical signal of the molten pool respectively reach the fourth reflector after being reflected by the second reflector, and reach the fifth reflector after being reflected by the third reflector, and are reflected to two right-angle surfaces of the second right-angle prism in the right-angle prism; The reflected first optical signal of the molten pool and the second optical signal of the molten pool are focused by the lens group to the camera to form the double-view images of the molten pool.

[0015] In a third aspect, the present invention further provides an electronic device, including a memory, a processor, and a computer program stored on the memory and executable on the processor. When the processor executes the program, the method for coaxial online monitoring of the three-dimensional morphology of the molten pool based on binocular vision as described in any one of the above is implemented.

[0016] The method and system for coaxial online monitoring of the three-dimensional morphology of the molten pool based on binocular vision provided by the present invention can obtain the binocular vision images of the molten pool in the narrow coaxial system of laser additive manufacturing, so as to perform subsequent three-dimensional reconstruction of the molten pool, and has strong adaptability; by establishing a residual network with unsupervised adaptive weights, high-precision disparity reconstruction is achieved. This network does not require manually marked learning labels and set empirical loss weights, and can adaptively learn the disparity information of the images from two perspectives. Description of the Drawings

[0017] In order to more clearly illustrate the technical solutions in the present invention or the prior art, the following will briefly introduce the drawings required for the description of the embodiments or the prior art. Obviously, the drawings in the following description are some embodiments of the present invention. For those of ordinary skill in the art, without creative efforts, other drawings can also be obtained based on these drawings.

[0018] Figure 1 is a schematic flowchart of the method for coaxial online monitoring of the three-dimensional morphology of the molten pool based on binocular vision provided by the present invention; Figure 2 is a schematic structural diagram of the processing optical path and the adjustable binocular vision optical path provided by the present invention; Figure 3 is a schematic structural diagram of the unsupervised adaptive loss neural network provided by the present invention; Figure 4 is a schematic structural diagram of the Res-Unet provided by the present invention; Figure 5 is a schematic calibration diagram provided by the present invention; Figure 6 is the equivalent baseline provided by the present invention b and focal length f calibration diagram; Figure 7 is the three-dimensional reconstruction accuracy of the binocular vision system provided by the present invention; Figure 8 is a schematic diagram of the three-dimensional morphology of the molten pool provided by the present invention; Figure 9 is a schematic structural diagram of the electronic device provided by the present invention.

[0019] Reference Signs: 1: Reflecting mirror; 2: Right-angle prism; 3: Lens group; 4: Camera; 5: Long-pass dichroic mirror; 6: Beam expander; 7: Laser; 8: Galvanometer; 9: Field lens. Detailed implementation manners

[0020] To make the objectives, technical solutions and advantages of the present invention clearer, the technical solutions in the present invention will be clearly and completely described below with reference to the accompanying drawings in the present invention. Apparently, the described embodiments are some but not all of the embodiments of the present invention. All other embodiments obtained by those of ordinary skill in the art based on the embodiments in the present invention without creative efforts shall fall within the protection scope of the present invention.

[0021] Figure 1 is a schematic flow chart of a method for online monitoring of the three-dimensional morphology of a molten pool based on binocular vision provided by an embodiment of the present invention. As Figure 1 shown, it includes: Step 100: Obtain dual-view images of the molten pool; Step 200: Input the dual-view images of the molten pool into an unsupervised adaptive loss neural network to obtain molten pool disparity information; Step 300: Calibrate the binocular vision monitoring system using a checkerboard to obtain system parameter information; Step 400: Obtain the depth information of the dual-view images of the molten pool according to the molten pool disparity information and the parameter information, and reconstruct the three-dimensional morphology of the molten pool based on the depth information of the dual-view images of the molten pool.

[0022] The present invention provides a method and system for online monitoring of the three-dimensional morphology of a molten pool in laser additive manufacturing based on binocular vision, including hardware and algorithms. The hardware part includes two optical paths, one is a processing optical path and the other is a binocular vision optical path. The algorithm part includes an unsupervised adaptive loss neural network (Unsupervised adaptive weighted-loss Res_Unet, UAWLRU) and a calibration method, specifically including: Step 1. Design of the coaxial binocular vision optical path As Figure 2 shown, the processing optical path and the binocular vision optical path are used to obtain dual-view images of the molten pool. The processing optical path includes a laser 7, a beam expander 6, a galvanometer 8, and a field lens 9. The binocular vision optical path includes a long-pass dichroic mirror 5, five reflectors 1, two right-angle prisms 2, a lens group 3, and a camera 4.

[0023] In this embodiment, the laser 7 generates laser light. The laser light first passes through the beam expander 6 and then the long-pass dichroic mirror 5, and is reflected by the galvanometer 8 and focused by the field lens 9 to melt the metal powder to form a molten pool. The optical signal of the molten pool travels along the field lens 9 and the galvanometer 8, and is reflected by the long-pass dichroic mirror 5 to the binocular vision imaging system. Further, the optical signal reaches the two right-angle surfaces of the right-angle prism 2 reflector respectively, and after being reflected by it, reaches the two reflectors 1 respectively. After passing through the two reflectors 1 and entering the other two reflectors 1, it is reflected by the other right-angle prism 2, outputting two optical signals of the molten pool with double perspectives. Finally, it is focused by the lens group onto the camera sensor to record the double-perspective image of the molten pool.

[0024] Step 2: Construct the unsupervised adaptive loss neural network UAWLRU An unsupervised neural network is a neural network trained by unsupervised learning methods. Its goal is to automatically learn the structure and patterns in the data by minimizing the loss function without any manually labeled training samples.

[0025] Figure 3 It is a schematic diagram of the structure of the unsupervised adaptive loss neural network in the embodiment of the present invention, including a disparity extraction neural network Res_Unet, a perspective mapping module Unwrap, and a loss function module. Res_Unet obtains the disparity information of the double-perspective image of the molten pool through feature learning. Res_Unet includes three parts: a residual module, a max-pooling module, and an upsampling module (Up2× module). Unwrap realizes the function of obtaining the predicted molten pool images of perspective 2 and perspective 1 by matrix translation and rotation operations on the molten pool images of perspective 1 and perspective 2 and the predicted disparities of the molten pool of perspective 1 and perspective 2. The loss function is composed of three parts: appearance matching loss, disparity smoothness loss, and disparity consistency loss between the two perspectives. The loss function module realizes the appearance matching loss, that is, calculating the error between the predicted molten pool images of perspective 1 and perspective 2 and the original images, the disparity smoothness loss calculation, that is, the gradient of the disparity images of the two perspectives, and the disparity consistency loss calculation, that is, obtaining the error value by taking the negative of one disparity and subtracting it from the other perspective image.

[0026] The principle of UAWLRU is to input the double-perspective image of the molten pool to obtain the disparity information, predict the double-perspective image of the molten pool according to the disparity information, calculate the appearance matching loss, disparity smoothness loss, and disparity consistency loss between the two perspectives through the loss function, and perform backpropagation of the loss to make the network continuously learn to reduce the error. When the algorithm iteration times NStop learning afterwards. Specifically, input two perspective molten pool images into Res_Unet to predict the disparity images of the corresponding perspectives. Unwrap the molten pool image of perspective 1 and its corresponding disparity through the perspective mapping module Unwrap to obtain the predicted molten pool image of perspective 2. Similarly, unwrap the molten pool image of perspective 2 and its corresponding disparity through the perspective mapping module Unwrap to obtain the predicted molten pool image of perspective 1. Specifically, the principle of the perspective mapping module Unwrap is as follows: combine a perspective image with disparity information and obtain another perspective image through matrix translation and rotation operations. Here, the disparity is the pixel position offset of the same object point in the two perspective images. Further, calculate the error through the loss function and perform backpropagation to achieve the iterative learning of the network. Specifically, denote the loss between perspective 1 and the predicted perspective 1 as L a , denote the loss between perspective 2 and the predicted perspective 2 as L a ’, denote the disparity smoothness losses between perspective 1 and perspective 2 as L d , L d ’ respectively, and denote the disparity consistency loss between perspective 1 and perspective 2 as L c . At this time, all losses can be defined as All_loss = α(L a +L a ’ )+β (L d +L d ’ ) + σL c . Among them, α and β, σ are the weights of each loss part in the total loss, and α and β, σ parameter information is obtained through algorithm iteration. Specifically, the algorithm uses Bayesian theory and regards the weights of the losses as uncertainty estimation parameters. The tasks with high uncertainty have lower weights, and the tasks with low uncertainty have higher weights. When the final number of algorithm iterations reaches N , the network stops learning.

[0027] Furthermore, the Res_Unet proposed by the present invention is used to obtain the disparity information of the double-perspective molten pool images. Figure 4It is a schematic diagram of the Res-Unet structure. The double-view image of the molten pool undergoes feature extraction through a residual module, and through four max-pooling operations and residual networks, the image size is reduced and feature extraction is performed. Each time, the image size is halved and the depth is doubled. The downsampled feature image passes through four residual modules and upsampling modules. At this time, each time the image size is doubled and the depth is halved, and it is connected to the features in the left-side downsampling process and input into the subsequent residual modules and upsampling modules, finally outputting the double-view disparity image.

[0028] Furthermore, the residual module consists of three convolutional layers. The first layer has a 1×1 convolutional kernel, and the latter two layers have 3×3 convolutional kernels. After the second convolution, batch normalization (Batch Normalization, BN) and activation operation (LeakyRelu, LR) are performed, and then the output features after the third layer are connected to the original input, and then activated through LR. Among them, BN refers to batch normalization, which can prevent overfitting of internal features and play a certain regularization role. LR is an activation function that can increase the non-linear feature extraction ability of the model.

[0029] Step 3: Use a checkerboard to calibrate the optical monitoring system to obtain the baseline b and the focal length f In this embodiment, the calibration method is to obtain two parameters, the baseline and the focal length, by calibrating the binocular vision monitoring system with a checkerboard. The calibration device includes a checkerboard and a binocular vision online monitoring system. According to the mapping relationship between disparity and depth, specifically, disparity is inversely proportional to depth. The larger the disparity, the smaller the distance.

[0030] Figure 5 It is the calibration principle diagram of this embodiment of the present invention. The optical signal of the molten pool is focused by the main lens to form images of two perspectives. Let the centers of the molten pool images of perspectives 1 and 2 be O l , O r respectively. Let the positions of the object point A on the molten pool in the molten pool images of perspectives 1 and 2 be A i+1 , A i respectively. And A i+1 , A i The differences between the positions where they are located and O l are O r respectively. In addition, let the distance from the object point A to the main lens be x 1 , x 2 . z, the distance from the main lens to the sensor is f , O l and O r the spacing is b . Among them, f and b are the equivalent focal length and the baseline of the dual-view system respectively. Assuming that the upward direction of the image center is positive and the downward direction is negative, according to the geometric relationship, we can get: (1) where Δx is the parallax of the object point A.

[0031] It can be seen from formula (1) that the parallax is inversely proportional to the depth. The larger the parallax, the smaller the distance.

[0032] Figure 6 is the equivalent baseline b and the focal length f calibration diagram of the present invention. Based on formula (1), solving b and f respectively will lead to a complex calibration process, and the error of each parameter will increase the system error. In the embodiment of the present invention, on the basis of formula (1), directly record b and f the product as a parameter for calibration. The specific calibration process is as follows: Place three identical standard checkerboards on the substrate of the laser additive manufacturing equipment. The thickness of the checkerboard is d , where one checkerboard is placed alone as scene A, and two checkerboards are stacked together as scene B. The height difference formed by the two is d ; Obtain the parallax information Δ x ' from the dual-view images of the checkerboard in scene B, and the corresponding imaging distance is z '. Similarly, obtain the parallax information Δ x '' of the checkerboard in scene A, and the corresponding imaging distance is z '+ d ; Substitute the parallax and the corresponding imaging distance of the said scene A and scene B into the said basic geometric relationship to obtain the baseline and focal length parameter information of the system; According to formula (1), the following formula is deduced: (2) Specifically, the position of the same feature points of the checkerboard in scene A and scene B, that is, the vertices of the checkerboard cells in the images of view 1 and view 2, is used to obtain the bf parameter information of the binocular system through formula (2).

[0033] Step 4: According to the mapping relationship between parallax and depth, substitute the parameter information obtained in Step 2 and Step 3, i.e., parallax information, baseline, and focal length, into this mapping relationship to obtain the depth information of the double-view image of the molten pool. Then, reconstruct the three-dimensional morphology of the molten pool according to formula (1).

[0034] In one embodiment, the proposed solution of the present invention is used for the calibration of the optical monitoring system, and the calculated bf is 1150mm*pixel.

[0035] In the embodiment, by grinding two standard gauge blocks in a flat crystal, a height difference of 500μm is formed between them. The three-dimensional morphology reconstructed by the present invention is as Figure 7 shown, and the measured depth is 510.5μm, the lateral error is 10μm, and the comprehensive three-dimensional morphology error is .

[0036] In the embodiment of the present invention, a coaxial device is used to capture the double-view image of the molten pool. The double-view image is input into Res_Unet to obtain the parallax information of the molten pool, and the three-dimensional morphology of the molten pool is obtained by combining the calibration parameters. The final result is as Figure 8 shown. Specifically, set the UAWLRU iteration times N to 500, and obtain 6 sequences of the three-dimensional morphology of the molten pool at intervals of 30ms under a laser scanning rate of 30mm / s. The length and width of the molten pool are distributed between 0-400μm. Taking the substrate height as 0 point, its relative height is distributed between -150-100μm. Figure 8 The keyhole of the molten pool can be observed in Figure 8 , and the depth is distributed between 100-150μm. Irregular convex morphologies are shown around the keyhole. In addition

[0037] Figure 9 illustrates a schematic diagram of the physical structure of an electronic device, as Figure 9As shown in the figure, the electronic device may include: a processor 810, a communications interface 820, a memory 830, and a communication bus 840. Among them, the processor 810, the communications interface 820, and the memory 830 communicate with each other through the communication bus 840. The processor 810 may call the logic instructions in the memory 830 to execute the method for coaxial online monitoring of the three-dimensional morphology of the molten pool based on binocular vision. The method includes: acquiring binocular vision images of the molten pool; inputting the binocular vision images of the molten pool into an unsupervised adaptive loss neural network to obtain the disparity information of the molten pool; calibrating the binocular vision monitoring system using a checkerboard to obtain the system parameter information; obtaining the depth information of the binocular vision images of the molten pool according to the disparity information of the molten pool and the parameter information, and establishing the three-dimensional morphology of the molten pool based on the depth information of the binocular vision images of the molten pool.

[0038] In addition, when the logic instructions in the above-mentioned memory 830 are implemented in the form of software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on such an understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or a part of this technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions for causing a computer device (which may be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the methods described in various embodiments of the present invention. The foregoing storage medium includes: various media such as USB flash drives, mobile hard disks, read-only memories (ROM, Read-Only Memory), random access memories (RAM, Random Access Memory), magnetic disks, or optical discs that can store program codes.

[0039] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place, or may be distributed to multiple network units. Some or all of the modules can be selected according to actual needs to achieve the purpose of the solution of this embodiment. Those of ordinary skill in the art can understand and implement it without creative labor.

[0040] Through the description of the above embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus a necessary general hardware platform, and of course, it can also be implemented by hardware. Based on such an understanding, the essence of the above technical solution, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to enable a computer device (which can be a personal computer, server, or network device, etc.) to execute the methods described in each embodiment or some parts of the embodiments.

[0041] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit them. Although the present invention has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that they can still modify the technical solutions recorded in the foregoing embodiments, or perform equivalent replacements for some of the technical features. And these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A coaxial online monitoring method for the three-dimensional morphology of a molten pool based on binocular vision, characterized in that: include: Obtain dual-view images of the molten pool; Inputting the molten pool dual-view image into an unsupervised adaptive loss neural network to obtain molten pool disparity information; Calibrate the binocular vision monitoring system using a chessboard to obtain binocular system parameter information; The dual-view image depth information of the molten pool is obtained according to the molten pool parallax information and the parameter information, and the three-dimensional morphology of the molten pool is reconstructed based on the dual-view image depth information of the molten pool.

2. The method for coaxial online monitoring of the three-dimensional morphology of a molten pool based on binocular vision according to claim 1 is characterized in that: The molten pool dual-view image is input into an unsupervised adaptive loss neural network to obtain molten pool disparity information, including: Determine that the unsupervised adaptive loss neural network includes a disparity extraction neural network Res_Unet, a perspective mapping module Unwrap and a loss function module, and the molten pool dual-perspective image includes a perspective 1 molten pool image and a perspective 2 molten pool image; Inputting the perspective 1 melt pool image and the perspective 2 melt pool image into the disparity extraction neural network Res_Unet to obtain the perspective 1 melt pool disparity image and the perspective 2 melt pool disparity image; The perspective 1 melt pool image and the perspective 1 melt pool parallax image are unwrapped through the perspective mapping module to obtain a predicted perspective 2 melt pool image; the perspective 2 melt pool image and the perspective 2 melt pool parallax image are unwrapped through the perspective mapping module to obtain a predicted perspective 1 melt pool image; the perspective mapping module Unwrap includes matrix movement and rotation operations; The loss function of the loss function module is used to calculate the appearance matching loss of the molten pool images of two perspectives and the predicted molten pool images of two perspectives, the smoothness loss of the molten pool parallax images of two perspectives, and the consistency loss of the molten pool parallax images of two perspectives, and the loss is back-propagated to enable the network to continuously learn to reduce the loss. When the algorithm reaches a preset number of iterations, the learning is stopped to obtain the molten pool parallax information; The appearance matching loss includes the appearance matching loss between the view 1 melt pool image and the predicted view 1 melt pool image, and the appearance matching loss between the view 2 melt pool image and the predicted view 2 melt pool image; the smoothness loss of the two view melt pool parallax images is the gradient of each parallax image; the consistency loss is the error value obtained by subtracting the inverse of the melt pool parallax image of one view from the melt pool parallax image of another view.

3. The method for coaxial online monitoring of the three-dimensional morphology of the molten pool based on binocular vision according to claim 2 is characterized in that: Inputting the perspective 1 melt pool image and the perspective 2 melt pool image into the disparity extraction neural network Res_Unet to obtain the perspective 1 melt pool disparity image and the perspective 2 melt pool disparity image, including: Determine that the disparity extraction neural network Res_Unet includes a residual module, a maximum pooling module and an upsampling module; The residual module includes three layers of convolution, the first layer of convolution is a 1×1 convolution kernel, the second and third layers of convolution are both 3×3 convolution kernels, the second and third layers of convolution include batch normalization operations and activation operations, and the third layer of convolution includes connection with the original input features and activation operations; The maximum pooling module reduces the image size by half and keeps the number of layers unchanged, and the upsampling module enlarges the image size and changes the number of layers according to the settings; The view 1 melt pool image and the view 2 melt pool image are sequentially input into the residual module, the maximum pooling module and the upsampling module, and the view 1 melt pool parallax image and the view 2 melt pool parallax image are output.

4. The method for coaxial online monitoring of the three-dimensional morphology of the molten pool based on binocular vision according to claim 3 is characterized in that: The loss function includes appearance matching loss, parallax smoothness loss and parallax consistency loss; The appearance matching loss includes the appearance matching loss between the view 1 melt pool image and the predicted view 1 melt pool image, and the appearance matching loss between the view 2 melt pool image and the predicted view 2 melt pool image; The parallax smoothness loss includes the gradient of the parallax image of the molten pool at the viewing angle 1 and the gradient of the parallax image of the molten pool at the viewing angle 2; The parallax consistency loss includes, for two perspectives of the molten pool parallax images, an error value obtained by subtracting the inverse of the molten pool parallax image of one perspective from the molten pool parallax image of the other perspective; The appearance matching loss, the disparity smoothness loss and the disparity consistency are weightedly summed to obtain the loss function.

5. The method for coaxial online monitoring of molten pool three-dimensional morphology based on binocular vision according to claim 2 is characterized in that: The binocular vision monitoring system is calibrated using a chessboard to obtain system parameter information, including: The distance from the main lens to the sensor is taken as the focal length, the center distance is taken as the baseline, and the difference between the first difference and the second difference is taken as the parallax, and a basic geometric relationship is established; Three identical standard checkerboards are placed on the laser additive manufacturing substrate, one of which is placed alone as scene A, and two checkerboards are stacked together as scene B. The height difference between the two scenes is the thickness of one checkerboard. Determine the checkerboard disparity information in scene A and scene B; Obtain the distance between the chessboard grid of scene B and the main lens, and obtain the distance between the chessboard grid and the main lens by adding the thickness of the chessboard grid to the distance between the chessboard grid and the main lens; Substituting the parallax of the scene A and the scene B and the corresponding imaging distance into the basic geometric relationship, the baseline and focal length parameter information of the system are obtained.

6. The method for coaxial online monitoring of the three-dimensional morphology of a molten pool based on binocular vision according to claim 5 is characterized in that: Obtaining the molten pool dual-view image depth information according to the molten pool parallax information and the parameter information, and establishing the molten pool three-dimensional morphology based on the molten pool dual-view image depth information, including: According to the mapping relationship between disparity and depth, the depth information of the molten pool dual-view image is used and substituted into the basic geometric relationship to obtain the three-dimensional morphology of the molten pool.

7. A binocular vision-based coaxial online monitoring system for a molten pool three-dimensional morphology, used to execute any binocular vision-based coaxial online monitoring method for a molten pool three-dimensional morphology as claimed in any one of claims 1 to 6, characterized in that: include: Binocular vision optical path and processing optical path, real-time acquisition of dual-view images of the molten pool; The processing optical path includes a laser, a beam expander, a galvanometer and a field lens; The binocular vision optical path includes five reflectors, two right-angle prisms, a lens group and a camera.

8. The binocular vision-based coaxial online monitoring system for molten pool three-dimensional morphology according to claim 7 is characterized in that: The processing optical path includes a laser, a beam expander, a long pass wave dichroic mirror, a galvanometer and a field mirror, including: The laser sends laser light to the beam expander, and the laser light passes through the beam expander and the long-pass-wavelength dichroic mirror in sequence, and is reflected by the galvanometer mirror and focused by the field mirror to melt the metal powder and form a molten pool; The molten pool light signal in the molten pool is reflected to the binocular vision light path along the field mirror and the galvanometer mirror in turn through the long pass wave dichroic mirror.

9. The binocular vision-based coaxial online monitoring system for molten pool three-dimensional morphology according to claim 8 is characterized in that: The binocular vision optical path includes five reflectors, two right-angle prisms, a lens group and a camera, including: The molten pool light signal is reflected by the first reflector to the two right-angle surfaces of the first right-angle prism in the right-angle prism to form a first molten pool light signal and a second molten pool light signal. The first molten pool light signal and the second molten pool light signal are reflected to the second reflector and the third reflector. The first molten pool optical signal and the second molten pool optical signal are respectively reflected by the second reflector and then reach the fourth reflector, and reflected by the third reflector and then reach the fifth reflector, and are reflected to the two right-angle surfaces of the second right-angle prism in the right-angle prism; The reflected first molten pool light signal and the second molten pool light signal are focused to the camera through the lens group to form a dual-viewing angle image of the molten pool.

10. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the program, the coaxial online monitoring method of the three-dimensional morphology of the molten pool based on binocular vision as described in any one of claims 1 to 6 is implemented.

Images