3D printing laser scanning anomaly detection method and device based on machine vision

Abstract

The present invention belongs to the field of 3D printing, and relates to a 3D printing laser scanning anomaly detection method and device based on machine vision, the method comprising: S1, determining the printing sequence number, the light spot moving speed and the model contour boundary according to the layer filling data; S2, acquiring the platform image sequence; S3, classifying the platform image after converting it into a binary image, calibrating the substrate area, detecting the laser spot and converting the laser spot centroid coordinates; S4, abnormal state detection of laser scanning. The device proposed by the present invention comprises an image acquisition module, a communication module, a platform image processing module and an abnormal state detection module. The present invention has a simple structure, does not require additional modification to the existing 3D printing system, replaces manual labor with machine vision, and improves the system's automation and integration.

Description

Technical Field

[0001] The present invention belongs to the field of 3D printing, and in particular relates to a 3D printing laser scanning anomaly detection method and device based on machine vision. Background Art

[0002] At present, lasers are widely used in the industrial field, such as 3D printing, laser scanning welding, laser surface processing, laser quenching and other fields. In the industrial production process, if the laser scanning is in an abnormal state, it will damage the processed products at the least, and even damage the equipment and even endanger the workers in serious cases. Therefore, it is particularly important to monitor the status information of laser scanning, especially the abnormal status.

[0003] At present, in the field of 3D printing, abnormal laser scanning status is generally discovered by workers through manual monitoring on site, and there is little research on how to achieve automatic processing. In the prior art, the patent with authorization announcement number CN209231773U announced a laser galvanometer monitoring circuit for monitoring laser galvanometer alarm signals, but this method requires the monitoring circuit to be integrated and analyzed with the original circuit, which is relatively complicated. Therefore, a more convenient integrated solution is currently needed to realize the monitoring of laser scanning status, especially abnormal status. Summary of the invention

[0004] In view of the shortcomings of the prior art, the present invention provides a 3D printing laser scanning anomaly detection method and device based on machine vision, which realizes the monitoring of the laser scanning state, especially the feedback control of the abnormal state, through image acquisition, image processing, position detection, and feedback control system methods. The device provided by the present invention has a more convenient integration solution, saving time and labor costs in printing monitoring.

[0005] To achieve the above object, the present invention discloses a 3D printing laser scanning anomaly detection method based on machine vision, which comprises the following steps:

[0006] S1, determine the light spot moving speed and printing boundary: obtain the layer filling data of 3D printing, and determine and save the printing sequence number, the model contour boundary coordinate set of the corresponding layer and the light spot moving speed each time the laser is switched on and off according to the layer filling data of 3D printing;

[0007] S2, obtain platform image sequence , and record the print serial number of the laser scan ; In the platform image sequence The platform image collected at each moment is represented as ;

[0008] S3, platform image processing;

[0009] First, the platform image Convert to binary image ,Then the platform image is classified, the substrate area is calibrated, the laser spot is detected, and the laser spot centroid coordinates are transformed;

[0010] S4, abnormal state detection of laser scanning;

[0011] Abnormal state detection of laser scanning includes detecting whether the laser is not turned off during the laser off delay period, detecting whether the laser is not emitting light during the laser on period, detecting whether the galvanometer deflection exceeds the limit, detecting whether the galvanometer does not move, detecting abnormal galvanometer movement, and detecting whether the laser spot exceeds the model boundary contour;

[0012] The abnormal movement of the galvanometer is detected by performing vector calculation on the centroid coordinates of the laser spot in three consecutive binary images containing the laser spot. If the centroid coordinates of the laser spot do not move, a stop alarm signal indicating that the galvanometer has not moved is generated and sent to the industrial computer of the 3D printing system.

[0013] The abnormal movement of the galvanometer is detected as follows: according to the displacement of the center of mass of the laser spot , , calculate the laser spot moving speed in the three binary images , ,If the speed exceeds the set threshold range, it means that the galvanometer movement speed does not meet the printing setting range, and a non-stop alarm signal of abnormal galvanometer movement is generated and sent to the industrial computer of the 3D printing system;

[0014] S5, return to S2, wait for the next laser scan, and repeat S2-S4 until the 3D printing system stops working.

[0015] Preferably, in S4, the centroid coordinates of the laser spot in the binary image are vectorized, which is specifically implemented as follows:

[0016] Assume that the centroid coordinates of the laser spot of the three binary images are , , The recorded times are , , ,and ; Calculate the centroid displacement of the laser spot between two adjacent binary images , , respectively , ,like , If both are 0, it means that the centroid coordinates of the laser spot have not moved.

[0017] Preferably, in S4, the moving speed of the laser spot in the three binary images is calculated. , , and set the threshold range, specifically:

[0018] According to the displacement of the laser spot centroid , , calculate the laser spot moving speed in the three binary images , :

[0019] ;

[0020] ;

[0021] The print serial number of the laser scan obtained by S2 , get the same print number in S1 The corresponding light spot moving speed , according to the moving speed of the light spot , set the laser spot moving speed , Threshold range of

[0022] The range of the light spot moving speed is the light spot moving speed in the layer filling data. Between 0.5 and 1.5 times, the judgment formula is:

[0023] ;

[0024] ;

[0025] in, The print sequence number obtained based on the layer filling data The corresponding light spot moving speed.

[0026] Preferably, in the abnormal state detection of the S4 laser scanning,

[0027] Check whether the laser is turned off during the laser off delay period. Specifically: the time-lapse platform image collected during the laser off delay period If there is a laser spot in the 3D printing system, a shutdown alarm signal indicating that the laser is not turned off abnormally is generated and sent to the industrial computer of the 3D printing system;

[0028] Detect that the laser does not emit light during the on-time period. Specifically: The on-time binary image collected during the laser on-time period If there is no laser spot in the image, a laser light failure alarm signal is generated and sent to the industrial computer of the 3D printing system;

[0029] Detection of galvanometer deflection exceeding the limit, specifically: the binary image of the laser light-on time period collected ,If the coordinate position of the laser spot exceeds the substrate area, a shutdown alarm signal of abnormal galvanometer deflection exceeding the limit is generated and sent to the industrial computer of the 3D printing system;

[0030] Detect that the laser spot exceeds the model boundary contour, specifically: the print sequence number of the laser scan obtained according to S2 , get the same print number in S1 The corresponding model contour boundary coordinate set , determine the centroid of the laser spot in the substrate coordinate system Coordinates and model contour boundary coordinates collection If the centroid of the laser spot is outside the model contour boundary, a non-stop alarm signal of abnormal galvanometer movement range is generated and sent to the industrial computer of the 3D printing system.

[0031] Preferably, in S3, the platform image Convert to binary image , and then classify the platform image, calibrate the substrate area, detect the laser spot, and transform the laser spot centroid coordinates; the specific steps are:

[0032] S31, the platform image Convert to binary image ,In the binary image, the background is black, and the laser spot is white;

[0033] S32, classify the platform image, and classify the binary image according to the laser switching time Classified as laser on time period The collected binary image , or laser off delay period Acquired time-lapse binary images ;

[0034] S33, calibrating the substrate area in the platform image, dividing the acquired image into the substrate area and the peripheral area beyond the substrate by a calibration line or a cropping method;

[0035] S34, laser spot detection, from binary image Extract the laser spot coordinate set from ;According to the laser spot coordinate set , determine the coordinates of the laser spot centroid ;

[0036] S35, laser spot centroid coordinate conversion, convert the laser spot centroid coordinate in the platform pixel coordinate system Converted to the substrate coordinate system, the center of mass of the laser spot in the substrate coordinate system The coordinates are:

[0037] ;

[0038] in, is the substrate area in the substrate coordinate system The maximum value of the coordinates, is the substrate area in the substrate coordinate system The maximum value of the coordinates; , They are the substrate area in the platform pixel coordinate system Minimum and maximum values ​​of coordinates; , They are the substrate area in the platform pixel coordinate system The minimum and maximum values ​​of the coordinates.

[0039] Preferably, the specific steps of obtaining the platform image sequence in step S2 are: when the laser starts laser scanning, the platform image is continuously collected, and the platform image includes the substrate and the peripheral area extended according to the substrate; when the laser scanning is finished, the platform image collection is delayed to end after the laser is turned off, so as to obtain the platform image sequence during this laser scanning.

[0040] The present invention also discloses a device for 3D printing laser scanning anomaly detection based on machine vision, which is used to implement the above-mentioned method for 3D printing laser scanning anomaly detection based on machine vision, and includes an image acquisition module, a communication module, a platform image processing module and an abnormal state detection module, wherein the communication module is respectively connected to the image acquisition module, the platform image processing module and the abnormal state detection module, and the platform image processing module is respectively connected to the image acquisition module and the abnormal state detection module, specifically:

[0041] The image acquisition module is used to obtain a platform image sequence including the substrate area, and send each platform image sequence to the platform image processing module;

[0042] The communication module is used for the transfer and storage of signals and data, and at least includes: receiving the laser light on signal and the laser light off signal from the industrial computer and sending them to the image acquisition module; receiving the layer filling data from the industrial computer, obtaining the printing sequence number, the corresponding light spot moving speed and the model contour boundary coordinate set; the communication module provides data for the abnormal state detection module; the communication module receives the alarm signal from the abnormal state detection module and forwards it to the industrial computer of the 3D printing system;

[0043] The platform image processing module is used to perform image processing on the platform image sequence, and sequentially includes: a binarization submodule, a platform image classification submodule, a platform image substrate calibration submodule, a light spot detection submodule and a coordinate conversion submodule; the platform image processing module sends the platform image sequence with additional information after processing to the abnormal state detection module;

[0044] The abnormal state detection module is used to detect the laser state according to the platform image sequence with additional information obtained from the platform image processing module, through the laser not turning off the light detection submodule, the laser not emitting light detection submodule, the galvanometer deflection over-limit detection submodule, the galvanometer not moving detection submodule, the galvanometer movement abnormality submodule and the super model contour boundary detection submodule.

[0045] Preferably, the laser light-on signal and the laser light-off signal are received from the industrial computer and sent to the image acquisition module, specifically:

[0046] The communication module receives the laser start / stop signal from the industrial computer of the 3D printing system, which is used to indicate the start / stop status of the laser. When the 3D printing system starts the laser, the communication module will simultaneously receive the laser light-on signal from the industrial computer, and the communication module will immediately send a start signal to start the image acquisition module. When the 3D printing system turns off the laser, the communication module will simultaneously receive the laser light-off signal, and the communication module will send a shutdown signal to turn off the image acquisition module after a specified time delay.

[0047] Preferably, the binarization submodule, the platform image classification submodule, the platform image substrate calibration submodule, the light spot detection submodule and the coordinate conversion submodule in the platform image processing module are specifically:

[0048] The binarization submodule is used to perform binarization processing on the platform image to obtain a binary platform image; the platform image classification submodule is used to classify the input platform image into two types: the laser on-time period image and the laser off-time delay period image; the platform image substrate calibration submodule divides the platform image into two parts: the substrate area and the peripheral area; the spot detection submodule is used to determine whether there is a laser spot and obtain the laser spot position; the coordinate conversion submodule is used to convert the laser spot position from the platform pixel coordinate system to the substrate coordinate system.

[0049] Preferably, the laser not turning off detection submodule, the laser not emitting light detection submodule, the galvanometer deflection over-limit detection submodule, the galvanometer not moving detection submodule, the galvanometer movement abnormality submodule and the super model contour boundary detection submodule in the abnormal state detection module are specifically:

[0050] The laser not-off detection submodule is used to send a shutdown alarm signal to the communication module when the platform image is classified as a laser-off delay period image but a light spot is detected; the laser not-emitting light detection submodule is used to send a non-shutdown alarm signal to the communication module when the platform image is classified as a laser-on time period image but no light spot is detected; the galvanometer deflection overlimit detection submodule is used to send a shutdown alarm signal to the communication module when the light spot position is not in the substrate area; the galvanometer non-movement detection submodule is used to perform vector calculations on the light spot positions in three consecutive on-platform images to obtain the light spot position movement value, and send a shutdown alarm signal to the communication module when the light spot position has not moved; the galvanometer movement abnormality submodule is used to calculate the light spot movement speed of the light spot position in three consecutive on-platform images according to the acquisition time of each platform image, and send a non-shutdown alarm signal to the communication module if the speed exceeds the set threshold; the super-model contour boundary detection submodule is used to determine whether the laser spot is located in the set printing area according to the model contour boundary, and send a non-shutdown alarm signal to the communication module if the laser spot is not in the set printing area.

[0051] Compared with the prior art, the present invention has the following beneficial effects:

[0052] 1. The present invention replaces manual labor with machine vision, thus avoiding the harm of laser vision to practitioners. At the same time, during the long printing process, it can continuously monitor the status of the equipment, thereby improving the degree of automation and integration of the system.

[0053] 2. The present invention can detect and process various abnormal laser scanning conditions in the 3D printing process, and realizes the detection of various abnormal conditions such as laser not turning off during the laser delay period, laser not emitting light during the laser on period, galvanometer deflection exceeding the limit, galvanometer not moving, galvanometer movement abnormality, and laser spot exceeding the model boundary, etc. by using one device, and gives abnormal condition prompts. For abnormal conditions that may cause damage to the 3D printing system or endanger the operator, the system can be shut down in time to reduce the harm.

[0054] 3. The present invention can detect 3D printing laser scanning anomalies without requiring additional modification of the existing 3D printing system.

[0055] 4. The present invention innovatively proposes to detect laser scanning anomalies according to the acquired platform image sequence in 3D printing, especially using the laser spot centroid movement displacement and movement speed to detect the galvanometer, which simplifies the laser scanning anomaly detection device.

[0056] 5. The abnormal alarm signal issued by the present invention during detection helps to remind the operator to adjust the equipment status in time, which can improve the success rate of printing parts and save printing time and printing costs. BRIEF DESCRIPTION OF THE DRAWINGS

[0057] Figure 1 It is a flow chart of the 3D printing laser scanning anomaly detection method based on machine vision of the present invention;

[0058] Figure 2 This is a module diagram of a 3D printing laser scanning anomaly detection device based on machine vision of the present invention;

[0059] Figure 3 Schematic diagram of the acquisition range of the image acquisition module of the present invention. DETAILED DESCRIPTION

[0060] The exemplary embodiments, features and aspects of the present invention will be described in detail below with reference to the accompanying drawings. The same reference numerals in the accompanying drawings represent elements with the same or similar functions. Although various aspects of the embodiments are shown in the accompanying drawings, the drawings are not necessarily drawn to scale unless otherwise specified.

[0061] The present invention proposes a 3D printing laser scanning anomaly detection method and device based on machine vision, which realizes 3D printing, especially the abnormal state of laser scanning in the powder bed fusion printing process, based on machine vision, to detect the abnormal state of laser scanning in the process of powder bed fusion printing, and replaces manual work with machine vision to avoid the harm of laser vision to practitioners. At the same time, during the long printing process, the equipment status is continuously monitored, the degree of automation and integration of the system is improved, and the labor cost and time cost are reduced. The abnormal state of laser scanning in the present invention generally includes two types of situations: first, the laser on and off state is abnormal, that is, the laser does not emit light in the laser start state and the laser does not turn off in the laser off state; second, the laser position is abnormal, such as the laser exceeds the printing platform due to the abnormal galvanometer, the laser is located at the same position and keeps emitting light, or the laser scans irregularly. The present invention can detect both of these types of situations.

[0062] The present invention proposes a 3D printing laser scanning anomaly detection method based on machine vision, such as Figure 1 As shown, the specific steps are as follows:

[0063] S1, determine the light spot moving speed and printing boundary.

[0064] Get the layer filling data of 3D printing. The layer filling data of 3D printing contains the layer number. , the model contour boundary coordinate set of the corresponding layer , used to set the laser spot moving coordinate point and the corresponding spot moving speed The layer filling data of 3D printing uses a two-dimensional coordinate system based on the substrate. Usually, the lower left corner of the substrate is the origin of the coordinate system, and the horizontal right is the coordinate system. Axis positive direction, perpendicular to Axial direction The positive direction of the axis is called the substrate coordinate system. 3D printing is performed in units of polylines. Each time the laser is switched on and off, a polyline is printed through a laser scan according to the layer filling data, and the light spot movement speed is 200 μm during each polyline printing. It will not change, so according to the layer filling data of 3D printing, the printing sequence number each time the laser is switched on and off can be predetermined. , the corresponding layer number and the light spot moving speed , print serial number The initial value is 0. The serial number is printed every time the laser is turned on and off. Add 1, and you can get the model contour boundary coordinate set of the corresponding layer according to the layer number . Save the print number obtained based on the layer filling data of 3D printing , corresponding light spot moving speed and the model contour boundary coordinate set .

[0065] S2, obtain the platform image sequence.

[0066] When the laser starts laser scanning, the platform image including the entire printing operation platform area is continuously collected. The platform image includes the substrate and the peripheral area extended according to the substrate. The platform image includes not only the substrate area but also the peripheral area in order to detect the abnormal laser position such as the laser exceeding the printing platform. The specific peripheral area range can be determined according to the actual printer model. For example, when the substrate is 100mm×100mm, the area included in the platform image can preferably be 120mm×120mm.

[0067] When the laser scanning is finished, the laser is turned off and the platform image collection is delayed to obtain the platform image sequence during this laser scanning. , and record the print serial number of this laser scan .

[0068] Platform Image Sequence It consists of a series of platform images arranged in chronological order. The platform image collected at each moment is represented as In 3D printing, the laser light switching time is short, so the collected platform images do not need to be processed in real time. Instead, the platform image sequence collected each time the laser light is switched on and off is processed. Each time the laser light is switched on and off, a platform image sequence is collected. Printing sequence number The initial value is 0. Each time the laser is turned on and off, the last saved print number is printed. Add 1 to get the print sequence number. .

[0069] S3, platform image processing.

[0070] Without changing the order of platform images in the platform image sequence, Each platform image in the sub-platform image sequence The processing includes binarization, classification, substrate area calibration, image spot center position and spot detection, such as Figure 3 As shown, the specific steps include:

[0071] S31, the platform image Converted into a binary image, in which the background is black and the laser spot is white. The specific implementation process is as follows:

[0072] S311, obtain platform image , whose size is M×N, where M is the number of rows of platform image pixels and N is the number of columns of platform image pixels, that is, the total number of pixels of a platform image is M×N; the collected platform image The two-dimensional coordinate system is based on a single pixel. The origin of the pixel coordinate system is set as the lower left pixel, and the horizontal right is Axis positive direction, perpendicular to Axial direction The positive direction of the axis is called the platform pixel coordinate system.

[0073] S312, set the window size S, where S is a positive odd number, such as S=15, and the window size is used to define the size of the local area. The window size determines the neighborhood range considered when calculating the local average.

[0074] S313, define window , for each pixel coordinate , where 1≤ ≤M and 1≤ ≤N, define the window Pixel The S×S area centered is:

[0075] ;

[0076] in, and is the index value within the window.

[0077] S314, for each pixel ( , ), calculation window Average value of pixels within:

[0078] ;

[0079] in, is the number of valid pixels in the window, For Window Inside The pixel value of the pixel.

[0080] S315, calculation window The standard deviation of the pixel values ​​within :

[0081] .

[0082] S316, calculate the local threshold value based on the local mean and standard deviation :

[0083] ;

[0084] in, is a proportional factor used to adjust the sensitivity of the threshold.

[0085] S317, for each pixel Binarization is performed based on local threshold:

[0086] ;

[0087] in, Pixel The pixel value at .

[0088] right After each pixel in is binarized, a binary image is output. .

[0089] S32, classifying the platform image.

[0090] The binary image is transformed according to the laser switching time Classified as laser on time period The collected binary image of the laser on state or the laser off state delay period Acquired time-lapse binary images.

[0091] For each binary image , according to the binary image The acquisition time and the status of the laser are classified as follows:

[0092] ;

[0093] in: The laser-on binary image collected during the laser-on period; The time-lapse binary image collected during the laser light-off delay period; binary image The acquisition time is the platform image The collection time of all The binary image set collected during the laser on time period ,all The time-lapse binary image set collected during the laser off time period .

[0094] S33, calibrating the substrate area in the platform image.

[0095] The captured image is divided into two parts by marking lines or cutting, one part is the substrate area, and the other part is the peripheral area beyond the substrate.

[0096] A clipping-based approach is listed here. is the binary image to be processed, is the calibration area of ​​the substrate area, and the calibration area of ​​the substrate area is:

[0097] ;

[0098] in: , They are the substrate area in the platform pixel coordinate system The minimum and maximum values ​​of the coordinates, similarly , They are the substrate area in the platform pixel coordinate system The minimum and maximum values ​​of the coordinates.

[0099] The segmented area is:

[0100] ;

[0101] ;

[0102] in: is the substrate area, The outer region is beyond the substrate.

[0103] S34, laser spot detection.

[0104] From binary image Extract the laser spot coordinate set from ;

[0105] ;

[0106] The laser spot detection function is: :

[0107] ;

[0108] According to the laser spot detection function, when the binary image When there is no laser spot in the image, the laser spot coordinate set is empty, the laser spot detection function is 0; when the binary image When there is a laser spot in the If it is not empty, the laser spot detection function is 1.

[0109] When the binary image When there is a laser spot in the image, determine the coordinates of the center of mass of the laser spot :

[0110] ;

[0111] ;

[0112] in, is the laser spot coordinate set The total number of pixels contained in are the laser spot coordinate sets The pixels contained in coordinate, are the laser spot coordinate sets The pixels contained in coordinate, The coordinates of the centroid coordinate, The coordinates of the centroid coordinate.

[0113] S35, laser spot centroid coordinate transformation.

[0114] The layer filling data uses a two-dimensional coordinate system based on the substrate. Usually, the lower left corner of the substrate is the origin and the horizontal right is the Axis positive direction, perpendicular to Axial direction The positive direction of the axis is called the substrate coordinate system. The range of the substrate in the substrate coordinate system is (0, 0), ( ,0),(0, ), ( , ) is determined, where For the substrate area The maximum value of the coordinates, For the substrate area The maximum value of the coordinates.

[0115] To convert the centroid coordinates of the laser spot to the substrate coordinate system, it is necessary to introduce the substrate pixel coordinate system. The substrate pixel coordinate system is to move the origin in the platform pixel coordinate system to the pixel point in the lower left corner of the substrate, that is, in the platform pixel coordinate system ( , ) becomes the origin, then the range of the substrate in the substrate pixel coordinate system is (0, 0), ( ,0),(0, ), ( , ) is determined, where , They are the substrate area in the platform pixel coordinate system The minimum and maximum values ​​of the coordinates, similarly , They are the substrate area in the platform pixel coordinate system The minimum and maximum values ​​of the coordinates.

[0116] The laser spot centroid coordinates are converted from the platform pixel coordinate system to the substrate coordinate system as follows: First, the platform pixel coordinate system is converted to the substrate pixel coordinate system, and the coordinates of the laser spot centroid become ; and then converted to the center of mass of the laser spot in the substrate coordinate system The coordinates are:

[0117] ;

[0118] in, Indicates the center of mass of the laser spot in the substrate coordinate system.

[0119] S4, abnormal state detection of laser scanning.

[0120] After the image processing is completed, the platform image sequence is still arranged in chronological order, and abnormal conditions such as the laser not turning off during the delay period, the laser not emitting light during the laser on period, the galvanometer deflection exceeding the limit, the galvanometer not moving, the galvanometer movement abnormality, and the laser spot exceeding the model boundary are detected in turn. Each picture can be detected in turn and then the next picture is detected until all pictures are detected; or one detection can be completed for all pictures before performing the next detection. The detection order of different abnormal state detections can be adjusted as needed, but it is preferred to put the detection of serious abnormal states that require shutdown in front. The abnormal state detection of laser scanning specifically includes:

[0121] S41, detect whether the laser light is turned off during the delay period.

[0122] Time-lapse platform image acquired during the laser off delay period There is a laser spot in the time-lapse binary image set Existence , then a shutdown alarm signal of "laser not turning off abnormal light" is generated; in order to reduce false alarms, a threshold can be set to make a judgment based on the delayed binary image set. Each binary image in The detection result is to accumulate the number of delayed binary images with laser spots, and only when the accumulated number exceeds the specified threshold value will a shutdown alarm signal be generated.

[0123] In this embodiment, the alarm level is subdivided. When the abnormal state of the laser scanning detected causes damage to the 3D printing system or causes injury to the operator, a shutdown alarm signal is generated, otherwise it is a non-shutdown alarm signal. When detecting the abnormal state of the laser scanning, try to give priority to detecting the situation that will generate a shutdown alarm signal. After the alarm signal is generated, it will be immediately sent to the industrial computer of the 3D printing system. When the shutdown alarm signal is received, the industrial computer immediately interrupts the printing work of the 3D printing system and returns to the standby state, and displays the shutdown alarm signal on the display screen of the industrial computer; when a non-shutdown alarm signal is received, the industrial computer will display the non-shutdown alarm signal on the display screen of the industrial computer, and it is manually determined whether it is necessary to stop printing and adjust the 3D printing system.

[0124] S42, detecting that the laser does not emit light during the light-on period.

[0125] The binary image of the laser on time period There is no laser spot in the binary image set. Existence , then a non-stop alarm signal of "abnormal laser light output" is generated; in order to reduce false alarms, a threshold can be set to judge, based on the binary image set of the light Each binary image in The detection result is to accumulate the number of binary images without laser spot. Only when the accumulated number exceeds the specified threshold value, a non-shutdown alarm signal is generated.

[0126] S43, detecting whether the galvanometer deflection exceeds the limit.

[0127] Binary image of the light-on time period collected by the light-on device If the laser spot coordinate position exceeds the substrate area, that is, in the binary image set There is a binary image with light , a shutdown alarm signal of "galvanometer deflection exceeding limit abnormality" is generated.

[0128] S44, detect that the galvanometer mirror has not moved.

[0129] The centroid coordinates of the laser spot in three consecutive binary images containing laser spots are vectorized. The centroid coordinates of the laser spot in the three binary images are set to be , , The recorded times are , , ,and ; Calculate the centroid displacement of the laser spot between two adjacent binary images , , respectively , ,like , If both are 0, a shutdown alarm signal of "galvanometer not moving" is generated. The reason why the galvanometer does not move may be that the galvanometer is stuck or the galvanometer does not respond.

[0130] S45, detecting abnormal movement of the galvanometer.

[0131] According to the laser spot centroid displacement obtained by S44 , , calculate the laser spot moving speed in the three binary images , :

[0132] ;

[0133] ;

[0134] The print serial number of this laser scan obtained according to S2 , get the same print number as S1 The corresponding light spot moving speed , according to the moving speed of the light spot , determine the moving speed of the laser spot in the binary image , Is it within the set threshold range? If the speed exceeds the set threshold range, it means that the galvanometer movement speed does not meet the printing setting range, and a non-stop alarm signal of "abnormal galvanometer movement" is generated.

[0135] Preferably, the light spot moving speed value range is set to the light spot moving speed in the layer filling data If the value is between 0.5 and 1.5 times of , then the optimal judgment formula is:

[0136] ;

[0137] ;

[0138] At this time, according to the judgment formula, Laser spot moving speed during laser scanning , Is it within the set threshold range? If the speed exceeds the set threshold range, it means that the galvanometer trajectory is not smooth, and a "galvanometer movement speed abnormality" non-stop alarm signal is generated.

[0139] In chronological order, from the laser on time period The three binary images containing the laser spot are sequentially selected from the collected binary images, and S44 and S45 are repeatedly executed until the laser is turned on. After all the collected binary images are processed, S46 is continued.

[0140] S46, detecting that the laser spot exceeds the model boundary contour.

[0141] According to the laser centroid in S35 The coordinates in the substrate coordinate system are based on the print sequence number in S2. , get the same print number in S1 The corresponding model contour boundary coordinate set , use the ray method to determine the laser centroid in S35 Coordinates in the substrate coordinate system and the model contour boundary coordinate set If the laser center of mass is outside the model contour boundary, it means that the galvanometer movement trajectory exceeds the printing setting range, and a non-stop alarm signal of "galvanometer movement range abnormality" is generated.

[0142] S5, return to S2, wait for the next laser scan, and repeat S2-S4 until the 3D printing system stops working.

[0143] The present invention also proposes a device for implementing the above-mentioned 3D printing laser scanning anomaly detection method based on machine vision. The device proposed by the present invention is attached to the existing 3D printing equipment and connected to the industrial computer of the 3D printing equipment to automatically detect the abnormal situation of laser scanning, such as Figure 2 As shown, it at least includes an image acquisition module 1, a communication module 2, a platform image processing module 3 and an abnormal state detection module 4.

[0144] The image acquisition module 1 is used to obtain an image including the entire printing operation platform area; in the present invention, the image acquisition module 1 is implemented by a camera with high resolution and high frame rate, preferably an industrial camera, and a camera lens with a wavelength filter is used to filter out the interference of background light. The camera exposure time can be adjusted according to the laser scanning speed. The camera shooting area includes the entire substrate and the surrounding area of ​​the substrate, such as Figure 3 as shown in .

[0145] Communication module 2 is used for signal and data transfer and storage, and its functions include: receiving laser light on and off signals from the industrial computer and sending them to image acquisition module 1; receiving layer filling data from the industrial computer and obtaining the printing sequence number; , corresponding light spot moving speed and the model contour boundary coordinate set , used to provide to the abnormal state detection module 4; the communication module 2 receives the alarm signal from the abnormal state detection module 4 and forwards it to the industrial computer, specifically:

[0146] Communication module 2 obtains layer filling data from the industrial computer of the 3D printing system and processes it to obtain the printing sequence number , corresponding light spot moving speed and the model contour boundary coordinate set , the extracted data is sent to the abnormal state detection module 4 as the judgment threshold of the abnormal state detection module 4.

[0147] Receive the laser start and stop signal indicating the start and stop status of the laser from the industrial computer of the 3D printing system; when the 3D printing system starts the laser, the communication module 2 will simultaneously receive the laser light-on signal from the industrial computer, and the communication module 2 will immediately send a start signal to start the image acquisition module 1; when the 3D printing system turns off the laser, the communication module 2 will simultaneously receive the laser light-off signal, and the communication module 2 will send a shutdown signal to turn off the image acquisition module 1 after a specified time delay. In this way, through the communication module 2, the laser scanning and platform image acquisition are started at the same time, and after the laser scanning is completed, the platform image acquisition is delayed to end.

[0148] The abnormal state detection module 4 sends the alarm signal to the communication module 2, and the communication module 2 forwards the alarm signal to the industrial computer of the 3D printing system. The industrial computer performs different operations according to the received alarm signal of the corresponding level.

[0149] The platform image processing module 3 is used to perform image processing on the platform image sequence, and includes: a binarization submodule 31, a platform image classification submodule 32, a platform image substrate calibration submodule 33, a light spot detection submodule 34 and a coordinate conversion submodule 35. Among them, the binarization submodule 31 is used to perform binarization processing on the platform image to obtain a binary platform image, and the binarization submodule 31 is implemented by step S31; the platform image classification submodule 32 is used to classify the input platform image into two types: a laser light-on time period image and a laser light-off delay time period image, and the platform image classification submodule 32 is implemented by step S32; the platform image substrate calibration submodule 33 divides the platform image into two parts: a substrate area and a peripheral area, and the platform image substrate calibration submodule 33 is implemented by step S33; the light spot detection submodule 34 is used to determine whether there is a laser light spot and obtain the laser light spot position, and the light spot detection submodule 34 is implemented by step S34. The coordinate conversion submodule 35 is used to convert the laser spot position from the platform pixel coordinate system to the substrate coordinate system, and the coordinate conversion submodule 35 is implemented by step S35. Each platform image processed by the image processing module has the following additional information: platform image classification, substrate area range, the position of the laser spot in the platform pixel coordinate system and the substrate coordinate system, and whether the spot is detected.

[0150] The abnormal state detection module 4 is used to detect the laser state according to the platform image with additional information obtained from the platform image processing module 3, and at least includes a laser not turning off the light detection submodule 41, a laser not emitting light detection submodule 42, a galvanometer deflection over-limit detection submodule 43, a galvanometer not moving detection submodule 44, a galvanometer movement abnormality submodule 45 and a super model contour boundary detection submodule 46. The function of the laser non-light-off detection submodule 41 is to send a shutdown alarm signal to the communication module 2 when the platform image is classified as a laser light-off delay period image but a light spot is detected, and the specific implementation adopts step S41; the function of the laser non-light-emitting detection submodule 42 is to send a non-shutdown alarm signal to the communication module 2 when the platform image is classified as a laser light-on time period image but a light spot is not detected, and the specific implementation adopts step S42; the function of the galvanometer deflection over-limit detection submodule 43 is to send a shutdown alarm signal to the communication module 2 when the light spot position is not in the substrate area, and the specific implementation adopts step S43; the function of the galvanometer non-movement detection submodule 44 is to perform a stop alarm signal on the light spot position in three consecutive light-on platform images. The row vector operation obtains the movement value of the light spot position. When the light spot position has not moved, a shutdown alarm signal is sent to the communication module 2, which is specifically implemented by step S44; the function of the galvanometer movement abnormality submodule 45 is to calculate the light spot movement speed for the light spot position in three consecutive opening platform images according to the acquisition time of each platform image. If the speed exceeds the set threshold, a non-shutdown alarm signal is sent to the communication module 2, which is specifically implemented by step S45; the function of the super model contour boundary detection submodule 46 is to determine whether the laser spot is located in the set printing area according to the model contour boundary. If the laser spot is not in the printing set area, a non-shutdown alarm signal is sent to the communication module 2, which is specifically implemented by step S46.

[0151] In order to more clearly prompt the error information, the laser not turning off the light detection submodule sends a shutdown alarm signal displayed as "Laser not turning off the light abnormality"; the laser not emitting light detection submodule sends a non-shutdown alarm signal displayed as "Laser not emitting light abnormality"; the galvanometer deflection over-limit detection submodule sends a shutdown alarm signal displayed as "Galvanometer deflection over-limit abnormality"; the galvanometer not moving detection submodule sends a shutdown alarm signal displayed as "Galvanometer not moving"; the non-shutdown alarm signal sent by the galvanometer movement abnormality submodule is displayed as "Galvanometer movement speed abnormality", and the non-shutdown alarm signal sent by the super model contour boundary detection submodule is displayed as "Galvanometer movement range abnormality".

[0152] The device proposed in the present invention is used to implement a 3D printing laser scanning anomaly detection method based on machine vision. It is connected to the industrial computer of the existing 3D printing equipment through a communication module to detect the laser scanning anomaly of the 3D printing equipment, and realize different processing methods for the laser scanning anomaly through different alarm signals.

[0153] The embodiments described above are only descriptions of the preferred implementation modes of the present invention, and are not intended to limit the scope of the present invention. Without departing from the design spirit of the present invention, various modifications and improvements made to the technical solutions of the present invention by ordinary technicians in this field should all fall within the protection scope determined by the claims of the present invention.

Claims

1. A 3D printing laser scanning anomaly detection method based on machine vision, characterized in that: It includes the following steps: S1, determine the light spot moving speed and printing boundary: obtain the layer filling data of 3D printing, and determine and save the printing sequence number, the model contour boundary coordinate set of the corresponding layer and the light spot moving speed each time the laser is switched on and off according to the layer filling data of 3D printing; S2, obtain platform image sequence , and record the print serial number of the laser scan ; In the platform image sequence The platform image collected at each moment is represented as ; S3, platform image processing; First, the platform image Convert to binary image ,Then the platform image is classified, the substrate area is calibrated, the laser spot is detected, and the laser spot centroid coordinates are transformed; S4, abnormal state detection of laser scanning; Abnormal state detection of laser scanning includes detecting whether the laser is not turned off during the laser off delay period, detecting whether the laser is not emitting light during the laser on period, detecting whether the galvanometer deflection exceeds the limit, detecting whether the galvanometer does not move, detecting abnormal galvanometer movement, and detecting whether the laser spot exceeds the model boundary contour; The abnormal movement of the galvanometer is detected by performing vector calculation on the centroid coordinates of the laser spot in three consecutive binary images containing the laser spot. If the centroid coordinates of the laser spot do not move, a stop alarm signal indicating that the galvanometer has not moved is generated and sent to the industrial computer of the 3D printing system. The abnormal movement of the galvanometer is detected as follows: according to the displacement of the center of mass of the laser spot , , calculate the laser spot moving speed in the three binary images , ,If the speed exceeds the set threshold range, it means that the galvanometer movement speed does not meet the printing setting range, and a non-stop alarm signal of abnormal galvanometer movement is generated and sent to the industrial computer of the 3D printing system; S5, return to S2, wait for the next laser scan, and repeat S2-S4 until the 3D printing system stops working.

2. The 3D printing laser scanning anomaly detection method based on machine vision according to claim 1, characterized in that: In S4, the centroid coordinates of the laser spot in the binary image are vectorized, which is specifically implemented as follows: Assume that the centroid coordinates of the laser spot of the three binary images are , , The recorded times are , , ,and ; Calculate the centroid displacement of the laser spot between two adjacent binary images , , respectively , ,like , If both are 0, it means that the centroid coordinates of the laser spot have not moved.

3. The 3D printing laser scanning anomaly detection method based on machine vision according to claim 1, characterized in that: In S4, the moving speed of the laser spot in the three binary images is calculated. , , and set the threshold range, specifically: According to the displacement of the laser spot centroid , , calculate the laser spot moving speed in the three binary images , : ； ； The print serial number of the laser scan obtained by S2 , get the same print number in S1 The corresponding light spot moving speed , according to the moving speed of the light spot , set the laser spot moving speed , Threshold range of The range of the light spot moving speed is the light spot moving speed in the layer filling data. Between 0.5 and 1.5 times, the judgment formula is: ； ； in, The print sequence number obtained based on the layer filling data The corresponding light spot moving speed.

4. The 3D printing laser scanning anomaly detection method based on machine vision according to claim 1, characterized in that: In the abnormal state detection of the S4 laser scanning, Check whether the laser is turned off during the laser off delay period. Specifically: the time-lapse platform image collected during the laser off delay period If there is a laser spot in the 3D printing system, a shutdown alarm signal indicating that the laser is not turned off abnormally is generated and sent to the industrial computer of the 3D printing system; Detect that the laser does not emit light during the on-time period. Specifically: The binary image of the laser on-time period collected If there is no laser spot in the image, a laser light failure alarm signal is generated and sent to the industrial computer of the 3D printing system; Detection of galvanometer deflection exceeding the limit, specifically: the binary image of the laser light-on time period collected ,If the coordinate position of the laser spot exceeds the substrate area, a shutdown alarm signal of abnormal galvanometer deflection exceeding the limit is generated and sent to the industrial computer of the 3D printing system; Detect that the laser spot exceeds the model boundary contour, specifically: the print sequence number of the laser scan obtained according to S2 , get the same print number in S1 The corresponding model contour boundary coordinate set , determine the centroid of the laser spot in the substrate coordinate system Coordinates and model contour boundary coordinates collection If the centroid of the laser spot is outside the model contour boundary, a non-stop alarm signal of abnormal galvanometer movement range is generated and sent to the industrial computer of the 3D printing system.

5. The 3D printing laser scanning anomaly detection method based on machine vision according to claim 1, characterized in that: The S3 platform image Convert to binary image ,Then the platform image is classified, the substrate area is calibrated, the laser spot is detected, and the laser spot centroid coordinates are transformed; The specific steps are: S31, the platform image Convert to binary image ,In the binary image, the background is black, and the laser spot is white; S32, classify the platform image, and classify the binary image according to the laser switching time Classified as laser on time period The collected binary image , or laser off delay period Acquired time-lapse binary images ; S33, calibrating the substrate area in the platform image, dividing the acquired image into the substrate area and the peripheral area beyond the substrate by a calibration line or a cropping method; S34, laser spot detection, from binary image Extract the laser spot coordinate set from ; According to the laser spot coordinate set , determine the coordinates of the laser spot centroid ; S35, laser spot centroid coordinate conversion, convert the laser spot centroid coordinate in the platform pixel coordinate system Converted to the substrate coordinate system, the center of mass of the laser spot in the substrate coordinate system The coordinates are: ； in, is the substrate area in the substrate coordinate system The maximum value of the coordinates, is the substrate area in the substrate coordinate system The maximum value of the coordinates; , They are the substrate area in the platform pixel coordinate system Minimum and maximum values ​​of coordinates; , They are the substrate area in the platform pixel coordinate system The minimum and maximum values ​​of the coordinates.

6. The 3D printing laser scanning anomaly detection method based on machine vision according to claim 1, characterized in that: The specific steps of obtaining the platform image sequence in step S2 are as follows: when the laser starts laser scanning, the platform image is continuously collected, and the platform image includes the substrate and the peripheral area extended according to the substrate; when the laser scanning is completed, the platform image collection is delayed after the laser is turned off to obtain the platform image sequence during this laser scanning.

7. A device for 3D printing laser scanning anomaly detection based on machine vision, used to implement the method for 3D printing laser scanning anomaly detection based on machine vision as described in any one of claims 1 to 6, characterized in that: It includes an image acquisition module, a communication module, a platform image processing module and an abnormal state detection module, wherein the communication module is respectively connected to the image acquisition module, the platform image processing module and the abnormal state detection module, and the platform image processing module is respectively connected to the image acquisition module and the abnormal state detection module, specifically: The image acquisition module is used to obtain a platform image sequence including the substrate area, and send each platform image sequence to the platform image processing module; The communication module is used for the transfer and storage of signals and data, and at least includes: receiving the laser light on signal and the laser light off signal from the industrial computer and sending them to the image acquisition module; receiving the layer filling data from the industrial computer, obtaining the printing sequence number, the corresponding light spot moving speed and the model contour boundary coordinate set; the communication module provides data for the abnormal state detection module; the communication module receives the alarm signal from the abnormal state detection module and forwards it to the industrial computer of the 3D printing system; The platform image processing module is used to perform image processing on the platform image sequence, and sequentially includes: a binarization submodule, a platform image classification submodule, a platform image substrate calibration submodule, a light spot detection submodule and a coordinate conversion submodule; the platform image processing module sends the platform image sequence with additional information after processing to the abnormal state detection module; The abnormal state detection module is used to detect the laser state according to the platform image sequence with additional information obtained from the platform image processing module, through the laser not turning off the light detection submodule, the laser not emitting light detection submodule, the galvanometer deflection over-limit detection submodule, the galvanometer not moving detection submodule, the galvanometer movement abnormality submodule and the super model contour boundary detection submodule.

8. The device for 3D printing laser scanning anomaly detection based on machine vision according to claim 7, characterized in that: The receiving of the laser light-on signal and the laser light-off signal from the industrial computer and sending them to the image acquisition module is specifically as follows: The communication module receives the laser start / stop signal from the industrial computer of the 3D printing system, which is used to indicate the start / stop status of the laser. When the 3D printing system starts the laser, the communication module will simultaneously receive the laser light-on signal from the industrial computer, and the communication module will immediately send a start signal to start the image acquisition module. When the 3D printing system turns off the laser, the communication module will simultaneously receive the laser light-off signal, and the communication module will send a shutdown signal to turn off the image acquisition module after a specified time delay.

9. The device for 3D printing laser scanning anomaly detection based on machine vision according to claim 7, characterized in that: The binarization submodule, the platform image classification submodule, the platform image substrate calibration submodule, the light spot detection submodule and the coordinate conversion submodule in the platform image processing module are specifically: The binarization submodule is used to perform binarization processing on the platform image to obtain a binarized platform image; the platform image classification submodule is used to classify the input platform image into two types: the laser on-time period image and the laser off-time delay period image; the platform image substrate calibration submodule divides the platform image into two parts: the substrate area and the peripheral area; the spot detection submodule is used to determine whether there is a laser spot and obtain the laser spot position; The coordinate conversion submodule is used to convert the laser spot position from the platform pixel coordinate system to the substrate coordinate system.

10. The device for 3D printing laser scanning anomaly detection based on machine vision according to claim 7, characterized in that: The laser light-off detection submodule, the laser light-off detection submodule, the galvanometer deflection over-limit detection submodule, the galvanometer non-movement detection submodule, the galvanometer movement abnormality submodule and the super model contour boundary detection submodule in the abnormal state detection module are specifically: The laser not-off detection submodule is used to send a shutdown alarm signal to the communication module when the platform image is classified as a laser-off delay period image but a light spot is detected; the laser not-emitting light detection submodule is used to send a non-shutdown alarm signal to the communication module when the platform image is classified as a laser-on time period image but no light spot is detected; the galvanometer deflection over-limit detection submodule is used to send a shutdown alarm signal to the communication module when the light spot position is not in the substrate area; the galvanometer non-movement detection submodule is used to perform vector calculations on the light spot position in three consecutive on-platform images to obtain the light spot position movement value, and send a shutdown alarm signal to the communication module when the light spot position has not moved; The galvanometer movement abnormality submodule is used to calculate the light spot movement speed of the light spot in three consecutive opening platform images according to the acquisition time of each platform image. If the speed exceeds the set threshold, a non-stop alarm signal is sent to the communication module; the super model contour boundary detection submodule is used to determine whether the laser spot is located in the set printing area according to the model contour boundary. If the laser spot is not in the set printing area, a non-stop alarm signal is sent to the communication module.

Images