Laser detection method, electronic device, computer-readable storage medium and computer program product

Abstract

The application provides a laser detection method, an electronic device, a computer readable storage medium and a computer program product, wherein the method can include: obtaining an original image, the original image including a nozzle image area to be detected; based on the original image, performing first-level contour identification on the nozzle to be detected to obtain a first region of interest; the first region of interest containing the nozzle image area to be detected; in a case where it is determined that the first region of interest meets second-level deep learning model identification, performing second-level deep learning model identification on the nozzle to be detected based on the first region of interest to obtain a second region of interest; the second region of interest containing the nozzle image area to be detected; and based on the second region of interest, identifying a laser region of the nozzle to be detected to obtain a laser center detection result. Through the above implementation manner, the detection of the laser center and the nozzle can be more accurately realized.

Description

Technical Field

[0001] This application relates to the field of laser detection technology, and more specifically, to a laser detection method, electronic device, computer-readable storage medium, and computer program product. Background Technology

[0002] The nozzle in a laser processing system is a key component for laser beam transmission, and its geometric accuracy directly affects the quality of laser processing. The physical forms of nozzles in laser processing systems vary considerably. Specifically, while laser processing nozzles typically have a circular inner bore structure, they are affected by factors such as lighting conditions, surface reflection, and edge blurring during actual imaging, posing challenges to accurate detection. Furthermore, the nozzles vary significantly in size and shape, ranging from 1mm to 10mm, with substantial differences in surface texture, material reflectivity, and inner bore depth, posing a significant challenge to the generalization ability of traditional algorithms based on fixed parameters.

[0003] The imaging of a laser spot varies significantly depending on its focal length and position. In practical applications, laser center detection faces even more complex challenges, specifically: different power settings lead to variations in spot brightness and size; different focal lengths affect spot clarity and edge sharpness; and positional offsets cause spot morphology distortion. In precision machining processes such as laser cutting and welding, the coaxiality of the laser beam and nozzle is a key factor affecting processing quality. Traditional laser center detection and alignment methods suffer from the following problems: manual alignment is inefficient, relies on operator experience, is time-consuming (typically 5-10 minutes), has unstable accuracy, requires manual spot firing of the laser, involves high power, and poses safety hazards; traditional visual algorithms have limitations, are sensitive to changes in lighting, have poor environmental adaptability, have a high failure rate when the nozzle shape is irregular, and the laser spot morphology varies greatly under different powers, making identification difficult and requiring manual setting of the region of interest, resulting in low automation. Summary of the Invention

[0004] The purpose of this application is to provide a laser detection method, electronic device, computer-readable storage medium, and computer program product that can improve the detection accuracy of the nozzle of a laser processor, reduce the need for manual intervention, and improve the overall convenience of identification.

[0005] In a first aspect, the present invention provides a laser detection method, comprising: obtaining an original image, the original image including an image region of a nozzle to be detected; performing a first-level contour recognition on the nozzle to be detected based on the original image to obtain a first region of interest; the first region of interest including the image region of the nozzle to be detected; if it is determined that the first region of interest satisfies the recognition of a second-level deep learning model, performing a second-level deep learning model recognition on the nozzle to be detected based on the first region of interest to obtain a second region of interest; the second region of interest including the image region of the nozzle to be detected; and recognizing the laser region of the nozzle to be detected based on the second region of interest to obtain a laser center detection result.

[0006] In an optional implementation, the step of performing first-level contour recognition on the nozzle to be detected based on the original image to obtain a first region of interest includes: converting the original image to obtain first image data in a target format; performing binary classification on each pixel in the first image data to determine a first nozzle region and a first background region; and segmenting the original image based on the first nozzle region to determine the first region of interest.

[0007] In the above implementation method, in the first-level contour recognition, the background area and nozzle area are first clearly distinguished based on binary classification, and then the focus is on the cutting of the first nozzle area. This can make the first region of interest have relatively less background and provide a relatively reliable data foundation for subsequent recognition.

[0008] In an optional implementation, the step of segmenting the original image based on the first nozzle region to determine the first region of interest includes: determining a first segmentation mask based on the first nozzle region; restoring the size of the first segmentation mask to a size matching the original image to obtain a second segmentation mask; segmenting the original image based on the second segmentation mask to obtain a first segmented image; and obtaining the first region of interest based on the first segmented image.

[0009] In an optional implementation, obtaining the first region of interest based on the first segmented image includes: denoising the first segmented image to obtain a first denoised image; extracting contours from the first denoised image to determine one or more first valid contours; and calculating the minimum bounding circle of the largest contour from the one or more first valid contours and determining it as the first region of interest.

[0010] In the above implementation, noise that may affect the accuracy of the image can be removed through noise reduction processing, and the smallest bounding circle of the largest contour is taken as the first region of interest, so that the obtained first region of interest can better and more comprehensively characterize the nozzle to be detected.

[0011] In an optional implementation, the step of denoising the first segmented image to obtain a first denoised image includes: performing closing and opening operations on the first segmented image using elliptical structuring elements to obtain a first denoised image.

[0012] The above implementation combines elliptical structural elements for closing and opening operations to achieve noise reduction, which can process noise of both bright and dark polarities at once.

[0013] In an optional implementation, the step of performing a second-level deep learning model to identify the nozzle to be detected based on the first region of interest to obtain a second region of interest includes: transforming the first region of interest to determine a first tensor corresponding to the first region of interest, wherein the first tensor is a single-channel tensor; inputting the first tensor into an image segmentation model for segmentation to determine a second nozzle region and a second background region; and processing the first region of interest based on the second nozzle region to determine the second region of interest.

[0014] In the above implementation, an image segmentation model is introduced to identify the nozzle region in another way, so that the identified second region of interest can more clearly and accurately represent the nozzle to be detected.

[0015] In an optional implementation, the step of processing the first region of interest based on the second nozzle region to determine the second region of interest includes: segmenting the first region of interest based on the second nozzle region to determine a second initial region of interest; performing morphological enhancement processing on the second initial region of interest to obtain a first enhanced image; and determining the second region of interest based on the first enhanced image.

[0016] In the above implementation, for the second level of recognition, morphological enhancement processing can enhance relevant information and remove noise, so that the obtained second region of interest can better characterize the nozzle to be detected.

[0017] In an optional implementation, determining the second region of interest based on the first enhanced image includes: smoothing the edges of the first enhanced image to obtain a smoothed image; and using a simplified contour algorithm to perform contour simplification processing on the smoothed image to obtain the second region of interest.

[0018] In the above implementation, edge smoothing and contour simplification can be combined to make the area of ​​the nozzle to be detected clearer, and the nozzle to be detected can be more clearly expressed using the determined second region of interest.

[0019] In an optional implementation, before performing a second-level deep learning model recognition on the nozzle to be detected based on the first region of interest to obtain a second region of interest, the method further includes: determining whether the center position and radius of the nozzle to be detected meet a preset standard by using the first region of interest; if both the center position and radius of the nozzle to be detected meet the preset standard, the first region of interest is characterized as meeting the second-level deep learning model recognition; if either the center position or the radius of the nozzle to be detected does not meet the preset standard, the method further includes: recognizing the laser region of the nozzle to be detected based on the first region of interest to obtain a laser center detection result; wherein, if the distance between the center position of the nozzle to be detected and the edge of the first region of interest is not less than a distance threshold, the center position of the nozzle to be detected is characterized as meeting the preset standard; if the radius of the nozzle to be detected is less than a radius threshold, the radius of the nozzle to be detected is characterized as meeting the preset standard.

[0020] In the above implementation, the conditions of the first region of interest can be identified before the second level of recognition to determine whether the first region of interest meets the conditions for further second-level recognition. If the conditions for second-level recognition are met, the subsequent second-level recognition is performed. If the conditions for second-level recognition are not met, the second-level recognition is not required. Through the above implementation logic, the nozzle detection accuracy can be improved, some useless calculations can be avoided, and the overall processing efficiency can be improved.

[0021] In an optional implementation, the step of identifying the laser region of the nozzle to be detected based on the second region of interest to obtain a laser center detection result includes: performing channel separation based on the second region of interest to determine a three-channel tensor; performing binarization processing on the three-channel tensor to determine an intermediate detection result; and determining a laser center detection result based on the intermediate detection result.

[0022] In the above implementation, channel separation is performed based on the determined second region of interest, and binarization is performed to initially highlight the laser center and other intermediate detection results. Then, the detection result of the laser center is further determined based on the intermediate detection results.

[0023] In an optional implementation, determining the laser center detection result based on the intermediate detection result includes: performing morphological enhancement processing on the intermediate detection result to determine a second enhanced image; determining the largest connected region in the second enhanced image; and determining the laser center detection result based on the largest connected region.

[0024] In the above implementation, morphological enhancement processing can be performed on the intermediate detection results to enhance their shape and image, and make it easier to separate the laser center from other parts.

[0025] In an optional implementation, determining the laser center detection result based on the maximum connected region includes: performing polygon approximation calculation and minimum enclosing circle calculation on the maximum connected region to determine the laser center detection result.

[0026] In the above implementation method, the center detection result can be determined by combining polygon approximation calculation and minimum bounding circle calculation. The characteristics of polygon approximation calculation (greatly optimize the data structure and improve the calculation performance with controllable accuracy loss) and minimum bounding circle calculation (find a compact and simple outer boundary for the entire point set) can be combined to achieve complementarity, so that the laser center detection result determined in the short term can be more accurate and reliable.

[0027] In an optional implementation, based on the laser center detection result, the offset between the laser center and the center of the nozzle to be tested is determined; based on the offset between the laser center and the center of the nozzle to be tested, the laser center is adjusted.

[0028] In the above implementation method, the laser center can be adjusted by combining the automatically identified offset between the laser center and the center of the nozzle to be tested, so that the laser center and the center of the nozzle to be tested can be automatically aligned.

[0029] In a second aspect, the present invention provides an electronic device, comprising: a processor and a memory, wherein the memory stores machine-readable instructions executable by the processor, and when the electronic device is running, the machine-readable instructions are executed by the processor to perform the steps of the method described in any of the foregoing embodiments.

[0030] Thirdly, the present invention provides a computer-readable storage medium storing a computer program that, when executed by a processor, performs the steps of the method described in any of the foregoing embodiments.

[0031] Fourthly, the present invention provides a computer program product, the computer program product comprising a computer program, which, when executed by a processor, implements the method described in any one of the foregoing embodiments.

[0032] The laser detection method, electronic device, computer-readable storage medium, and computer program product provided in this application can obtain a first region of interest (ROI) and a second ROI through two-level localization and recognition. This allows the second ROI to more clearly and accurately represent the nozzle. Further laser region identification based on the second ROI results in a more reliable detection of the laser center. Specifically, the first-level localization and recognition uses contour recognition to initially identify the nozzle region in the original image, thus determining the first ROI. The second-level localization and recognition uses a deep learning model and performs identification based on the first ROI. The second-level localization and recognition process involves less interference information (such as background images), and the combination with the deep learning model enables more accurate nozzle identification. After obtaining the second ROI that more accurately represents the nozzle, laser center detection can be more focused and identified within the nozzle region, reducing interference from surrounding background pixels and thus making laser center detection more accurate. Attached Figure Description

[0033] To more clearly illustrate the technical solutions of the embodiments of this application, the accompanying drawings used in the embodiments will be briefly introduced below. It should be understood that the following drawings only show some embodiments of this application and should not be regarded as a limitation of the scope. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort.

[0034] Figure 1 A block diagram illustrating an electronic device provided in an embodiment of this application;

[0035] Figure 2 A flowchart of the laser detection method provided in the embodiments of this application;

[0036] Figure 3 An optional flowchart of step 230 of the laser detection method provided in the embodiments of this application;

[0037] Figure 4 An optional flowchart of step 250 of the laser detection method provided in the embodiments of this application;

[0038] Figure 5 Another flowchart of the laser detection method provided in the embodiments of this application;

[0039] Figure 6 An optional flowchart of step 270 of the laser detection method provided in the embodiments of this application. Detailed Implementation

[0040] The technical solutions in the embodiments of this application will now be described with reference to the accompanying drawings.

[0041] It should be noted that similar reference numerals and letters in the following figures indicate similar items; therefore, once an item is defined in one figure, it does not need to be further defined and explained in subsequent figures. Furthermore, in the description of this application, terms such as "first," "second," etc., are used only to distinguish descriptions and should not be construed as indicating or implying relative importance.

[0042] The embodiments of this application can provide a laser detection method, electronic device, computer-readable storage medium, and computer program product that can more accurately detect the nozzle and laser center of a laser processing system.

[0043] To facilitate understanding of this embodiment, the electronic device that performs the laser detection method disclosed in this application will first be described in detail.

[0044] like Figure 1 The diagram shown is a block illustration of an electronic device. The electronic device 100 may include a memory 111 and a processor 113. Those skilled in the art will understand that... Figure 1 The structure shown is for illustrative purposes only and does not limit the structure of the electronic device 100. For example, the electronic device 100 may also include components that are more... Figure 1 The more or fewer components shown, or having the same Figure 1 The different configurations shown.

[0045] The memory 111 and processor 113 described above are electrically connected to each other directly or indirectly to enable data transmission or interaction. For example, these components can be electrically connected to each other via one or more communication buses or signal lines. The processor 113 described above is used to execute executable modules stored in the memory.

[0046] The memory 111 can be, but is not limited to, Random Access Memory (RAM), Read Only Memory (ROM), Programmable Read-Only Memory (PROM), Erasable Programmable Read-Only Memory (EPROM), Electrically Erasable Programmable Read-Only Memory (EEPROM), etc. The memory 111 stores programs, and the processor 113 executes these programs upon receiving execution instructions. The methods executed by the electronic device 100 as defined in any embodiment of this application can be applied to the processor 113, or implemented by the processor 113.

[0047] The aforementioned processor 113 may be an integrated circuit chip with signal processing capabilities. The processor 113 may be a general-purpose processor, including a Central Processing Unit (CPU), a Network Processor (NP), etc.; it may also be a digital signal processor (DSP), an Application Specific Integrated Circuit (ASIC), a Field Programmable Gate Array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components. It can implement or execute the methods, steps, and logic block diagrams disclosed in the embodiments of this application. The general-purpose processor may be a microprocessor or any conventional processor.

[0048] The electronic device 100 may be a device that can be connected to a laser processing system and can obtain images of the nozzle and laser center of the laser processing system. The electronic device 100 may perform various steps in the laser detection method based on the obtained images of the nozzle and laser center.

[0049] In this embodiment, the electronic device 100 performs model initialization processing before executing each step of the laser detection method. For example, a double-checked locking pattern can be used to achieve thread-safe singleton initialization. In this embodiment, a Compute Unified Device Architecture (CUDA) can be used to execute the program implementing the laser detection method.

[0050] In this embodiment, the electronic device 100 can also create inference sessions based on the ONNX Runtime.

[0051] The electronic device 100 can also be used as a control device in the laser processing system. The output position of the laser in the laser processing system can be adjusted by the result calculated by the electronic device 100.

[0052] A laser processing system may include a laser emitter and a visible light indicator, wherein the laser emitter and the visible light indicator are coaxial, meaning that the laser beam emitted by the laser emitter and the visible light from the visible light indicator are transmitted through an optical fiber, thereby achieving coaxiality. For example, the visible light indicator may be red light.

[0053] The electronic device 100 in this embodiment can be used to execute various steps in the various methods provided in the embodiments of this application. The implementation process of the laser detection method is described in detail below through several embodiments.

[0054] Please see Figure 2 This is a flowchart of a laser detection method provided in an embodiment of this application. The laser detection method provided in this application can be applied to an electronic device, which then executes the steps of the laser detection method. The following will describe... Figure 2 The specific process shown will be explained in detail.

[0055] Step 210: Obtain the original image.

[0056] The original image includes the area of ​​the nozzle to be detected.

[0057] The original image can be an image captured at the nozzle of the laser processing system during the output of the laser beam. The nozzle is used to output the laser beam. Therefore, the image captured during the output of the laser beam by the laser processing system not only includes the image area of ​​the nozzle to be detected, but also a visible light spot coaxial with the laser beam within the image area of ​​the nozzle to be detected. The position of this light spot coincides with the laser beam, and the position of the light spot can represent the center of the laser beam.

[0058] Optionally, real-time images can be continuously acquired from the nozzle of the laser processing system according to a set time pattern. For example, ten images can be acquired per second, or fifteen images per second. Of course, the electronic device can provide a settings interface, allowing users to set the acquisition frequency based on their actual needs.

[0059] Step 230: Based on the original image, perform first-level contour recognition on the nozzle to be detected to obtain the first region of interest.

[0060] The first region of interest contains the area of ​​the nozzle image to be detected.

[0061] The original image contains not only the nozzle image region to be detected, but also the background region. Through the processing and recognition in step 230, the nozzle image region to be detected and the background region can be identified to extract the nozzle image region to be detected and use it as the first region of interest.

[0062] If the first region of interest is determined to satisfy the recognition of the second-level deep learning model, proceed to step 250.

[0063] Step 250: Based on the first region of interest, perform a second-level deep learning model recognition on the nozzle to be detected to obtain the second region of interest.

[0064] The second region of interest contains the area of ​​the nozzle image to be detected.

[0065] In this embodiment, the recognition method used by the second-level deep learning model may differ from the recognition method used by the first-level contour recognition. For example, the first-level contour recognition may use contour recognition to identify the nozzle region and the background region; the second-level deep learning model may use a deep learning model to identify and locate the nozzle region.

[0066] Step 270: Based on the second region of interest, identify the laser region of the nozzle to be detected and obtain the laser center detection result.

[0067] The laser center is located inside the area of ​​the nozzle to be tested. The laser center appears as a bright spot.

[0068] Based on the laser center detection results, the offset between the laser center and the center of the nozzle to be tested can be determined; then, based on the offset between the laser center and the center of the nozzle to be tested, the laser center can be adjusted.

[0069] The adjusted laser center can be aligned with the center of the nozzle to be tested. This achieves laser center alignment.

[0070] The offset between the laser center and the center of the nozzle to be tested can be expressed as the distance between the coordinates of the laser center and the coordinates of the center of the nozzle to be tested.

[0071] For example, the emission direction and position of the laser beam emitter can be adjusted to achieve the adjustment of the laser center.

[0072] The above implementation method first identifies the nozzle region through two-level localization and recognition, and then further identifies the second region of interest (ROI) containing the nozzle region to determine the detection result of the laser center. Based on hierarchical localization and recognition, the recognition error can be gradually reduced, improving the accuracy of laser center detection. Specifically, the first-level localization and recognition uses contour recognition to initially identify the nozzle region in the original image, determining the first ROI. The second-level localization and recognition uses a deep learning model and performs recognition based on the first ROI. The second-level localization and recognition process involves relatively less interference information (such as background images), and the combination with the deep learning model enables more accurate nozzle identification. After obtaining the second ROI that more accurately represents the nozzle, laser center detection can be more focused and identified within the nozzle region, reducing interference from surrounding background pixels, thus making laser center detection more accurate.

[0073] Furthermore, experiments were conducted to verify the detection method provided in this application. It was found that the cross-union ratio (CUI) of the nozzle detection results obtained through only the first-level contour recognition is approximately 89%. However, if the first-level contour recognition and the second-level deep learning model are combined, the CUI of the nozzle detection results can reach over 98%. In the CUI, the intersection refers to the region where the bounding box obtained from the nozzle contour recognition overlaps with the actual bounding box; the union refers to the sum of all covered areas of the predicted box and the actual box obtained from the nozzle contour recognition. A CUI closer to 1 indicates a more accurate recognition result. Through two levels of recognition, the accuracy of nozzle recognition can be improved, and a better data foundation is provided for subsequent laser center recognition, making the inspection results of laser centers more accurate and reliable.

[0074] Regarding the first level of recognition, a binary classification approach can be used to distinguish between the nozzle and the background. Based on this, such as... Figure 3 As shown, step 230 above may include steps 231 to 233.

[0075] Step 231: Convert the original image to obtain the first image data in the target format.

[0076] To facilitate standardized processing, the image dimensions can be uniformly adjusted to a set size before processing. In one example, the set size can be the standard 384×384. Of course, depending on the needs of subsequent processing, the standard size can also be other sizes, such as 256×256, 1024×1024, etc.

[0077] Optionally, the original image can be directly scaled to obtain a standard-sized image. For example, the original image can be scaled based on a first transformation relationship to obtain a standard-sized image. Alternatively, a crop of the original image can also be taken to obtain a standard-sized image.

[0078] The target format can be a four-dimensional tensor data.

[0079] The original image can be BGR color space image data, which can be converted to RGB format image data first.

[0080] The pixel value of each pixel in the image can be converted into a 32-bit floating-point number and normalized to [0,1].

[0081] Alternatively, the pixel values ​​of each pixel can be standardized using mean=[0.485,0.456,0.406] and std=[0.229,0.224,0.225].

[0082] For example, standardization can be performed using the formula: Standardized data = (Original data - Mean of data) / Standard deviation of data.

[0083] Where, mean=[0.485,0.456,0.406] represents the mean of each of the three channels. 0.485 represents the average pixel value of the R (red) channel. 0.456 represents the average pixel value of the G (green) channel. 0.406 represents the average pixel value of the B (blue) channel. std=[0.229,0.224,0.225] represents the standard deviation of each of the three channels. 0.229 represents the standard deviation of the R (red) channel; 0.224 represents the standard deviation of the G (green) channel; and 0.225 represents the standard deviation of the B (blue) channel.

[0084] Based on this, the formula for standardizing the pixel values ​​of each pixel based on mean=[0.485,0.456,0.406] and std=[0.229,0.224,0.225] can be expressed as: r_norm=(r-0.485) / 0.229; g_norm=(g-0.456) / 0.224; b_norm=(b-0.406) / 0.225.

[0085] Where r_norm, g_norm, and b_norm represent the pixel values ​​of the three channels after normalization; r, g, and b represent the pixel values ​​of the three channels before normalization.

[0086] After the above processing, the obtained data can be combined into a four-dimensional tensor of size [1,3,384,384]. Here, in [1,3,384,384], 1 represents the batch size (processing one image at a time); 3 represents the number of channels (RGB color channels); 384 represents the image height; and 384 represents the image width. Therefore, the first image data can be a four-dimensional tensor of size [1,3,384,384].

[0087] Step 232: Perform binary classification on each pixel in the first image data to determine the first nozzle region and the first background region.

[0088] Optionally, forward inference and argmax operations can be performed to achieve binary classification of each pixel in the first image data.

[0089] In one instance, the classification result can be 1 representing the nozzle region and 0 representing the background region. Given that the first image data is a four-dimensional tensor of [1,3,384,384], the binary classification output of step 232 can be a four-dimensional tensor of [1,1,384,384].

[0090] For example, after binary classification of each pixel, binarized image data can be obtained. Contour recognition is then performed based on the binarized image data to determine the first nozzle region and the first background region.

[0091] Step 233: The original image is segmented based on the first nozzle region to determine the first region of interest.

[0092] Considering that in the above implementation, the original image is converted to a 384×384 image, the first nozzle region can be converted to the size corresponding to the original image based on the conversion ratio. Then, the original image is segmented based on the first nozzle region to obtain the first region of interest.

[0093] In this embodiment, the success of nozzle detection can be determined by identifying whether the nozzle was successfully detected within the first region of interest. If successful detection is confirmed, the first region of interest is then used for subsequent calculations. For example, the criterion could be whether the nozzle to be detected was successfully detected, or whether the size of the detected nozzle area is within a threshold range. The threshold range could be 1-10 mm.

[0094] The above implementation method can distinguish the nozzle from the background based on binary classification, and the nozzle and background can be separated.

[0095] Optionally, step 233 above may include steps 2331 to 2334.

[0096] Step 2331: Determine the first segmentation mask based on the first nozzle region.

[0097] In this embodiment, the first segmentation mask of the first nozzle region can be obtained based on the binary classification result of step 232 above.

[0098] Step 2332: Restore the size of the first segmentation mask to match the size of the original image to obtain the second segmentation mask.

[0099] Considering that the first image data was converted to a standard-sized image upon acquisition, the processing in step 232 is based on this standard-sized image data. Therefore, the first nozzle region and the first background region obtained in step 232 are regions based on the standard-sized image data. Thus, if the original image needs to be segmented based on the first segmentation mask obtained from the first nozzle region, it is necessary to first restore the first segmentation mask to the size of the original image.

[0100] Optionally, the first segmentation mask can be restored from 384×384 to the original image size based on the second transformation relationship. The second transformation relationship can be the opposite of the first transformation relationship. For example, the original image can be converted to a 384×384 image using the first transformation relationship, and the 384×384 image can be converted back to the original image size using the second transformation relationship.

[0101] Optionally, if the standard-sized image is obtained by cropping it in the aforementioned calculation, the size or position of the first segmentation mask can be adjusted according to the orientation of the cropped image to determine a second segmentation mask that corresponds to the original image. For example, the position of the cropped standard-sized image in the original image can be determined based on the coordinates of the four vertices of the cropped image, thus determining the relative position of the first segmentation mask in the original image.

[0102] Step 2333: Cut the original image based on the second segmentation mask to obtain the first segmented image.

[0103] Step 2334: Based on the first segmented image, obtain the first region of interest.

[0104] The minimum bounding circle of the maximum contour is determined from the image data obtained after denoising the first segmented image, and the minimum bounding circle of the maximum contour is used as the first region of interest.

[0105] Optionally, the first segmented image can be denoised using elliptical structuring elements to perform closing and opening operations.

[0106] Both closing and opening operations are based on combinations of the two basic operations: dilation and erosion.

[0107] The erosion operation shrinks or "erodes" the foreground area of ​​the first cut image. It causes object boundaries to shrink inward, eliminating small, isolated bright spots and thus removing noise. The dilation operation enlarges or "expands" the foreground area of ​​the first cut image. It causes object boundaries to expand outward, filling in small holes and cracks inside objects.

[0108] The opening operation first performs erosion and then dilation to remove bright noise (pepper noise) smaller than the structuring element in the image, smooth the boundaries of objects, and break narrow connections. The closing operation first performs dilation and then erosion to fill dark noise (salt noise) and small holes smaller than the structuring element inside the object, smooth the boundaries of the object, and connect narrow breaks.

[0109] The outer contours of the denoised image are extracted to determine one or more valid contours. Valid contours are defined as having an area greater than a set area threshold. For example, this set area threshold could be 100 pixels, 120 pixels, etc.

[0110] In this embodiment, considering that the nozzle is generally circular, the effective contour can also satisfy the requirement that the shape is circular, and non-circular contours can also be filtered through shape filtering.

[0111] In this embodiment, after determining the effective contours, the minimum bounding circle of the largest contour is calculated from one or more first effective contours, and this circle is determined as the first region of interest.

[0112] To improve the reliability of the first region of interest (ROI), a roundness index can be calculated based on the area-to-perimeter ratio, and then verified using ellipse fitting. If the roundness index passes the verification, the region is designated as the first ROI for subsequent calculations.

[0113] Based on the above implementation logic, when determining the first region of interest, multiple processes such as denoising, contour extraction, and roundness index verification can be integrated to enable the determined first region of interest to better characterize the nozzle.

[0114] The second level of recognition can be achieved by combining image recognition models. Based on this, such as... Figure 4 As shown, step 250 above may include steps 251 to 253.

[0115] Step 251: Transform the first region of interest to determine the first tensor corresponding to the first region of interest.

[0116] In this embodiment, the second-level deep learning model recognition can be based on grayscale image processing. The first tensor can be a single-channel tensor.

[0117] Therefore, the first region of interest is first converted into a 384×384 image according to the first transformation relationship, and then converted into a grayscale image. Alternatively, a screenshot of the first region of interest can be taken to capture a 384×384 image.

[0118] To improve calculation accuracy, the pixel value of each pixel can be converted into a 32-bit floating-point number and normalized to [-1, 1]. For example, the pixel value can be normalized using the formula: (pixel / 255.0 - 0.5) / 0.5, where pixel represents the pixel value of each pixel.

[0119] Based on the above processing, a single-channel tensor of [1,1,384,384] can be obtained.

[0120] Step 252: Segment the first tensor input image using the segmentation model to determine the second nozzle region and the second background region.

[0121] Optionally, the image segmentation model is a deep learning model. For example, the deep learning model could be a UNet model.

[0122] The UNet model is used to perform pixel-level segmentation of the first tensor, thereby determining the second nozzle region and the second background region.

[0123] Step 253: Process the first region of interest based on the second nozzle region to determine the second region of interest.

[0124] For example, a third segmentation mask can be determined based on the segmentation result of step 252, and the first region of interest can be segmented based on the third segmentation mask to determine a temporary region of interest.

[0125] In this embodiment, the success of nozzle detection can be determined by identifying whether the second region of interest (ROI) successfully detects the nozzle. If successful detection is confirmed, the second ROI is then used for subsequent calculations. For example, the criterion could be whether the nozzle to be detected was successfully detected, or whether the size of the detected nozzle area is within a threshold range. The threshold range could be 1-10 mm.

[0126] Optionally, the temporary region of interest can be processed by noise reduction or other methods to obtain a second region of interest.

[0127] Optionally, step 253 may include steps 2531 to 2533.

[0128] Step 2531: Cut the first region of interest based on the second nozzle region to determine the second initial region of interest.

[0129] Optionally, a third segmentation mask can be determined based on the second nozzle region, and the first region of interest can be segmented based on the third segmentation mask to determine the second initial region of interest.

[0130] Step 2532: Morphological enhancement is performed on the second initial region of interest to obtain the first enhanced image.

[0131] Alternatively, this morphological enhancement process can combine closing and opening operations to achieve noise reduction. In one example, a 7×7 elliptic kernel closing operation can be used to fill holes, and a 5×5 elliptic kernel opening operation can be used to remove noise.

[0132] Step 2533 determines the second region of interest based on the first enhanced image.

[0133] To make the identified region of interest clearer and more accurate, the edges of the first enhanced image can be smoothed and re-binarized to achieve a sharp boundary. For example, a Gaussian filter can be used to smooth the edges to obtain a smoothed image. For instance, a 5×5 Gaussian filter can be used to smooth the edges and then re-binarized to achieve a sharp boundary.

[0134] To further clarify and simplify the outline of the identified second region of interest, the outline can be simplified. For example, a simplified outline algorithm can be used to process a smooth image to obtain the second region of interest. For example, the simplified outline algorithm could be the Douglas-Peucker algorithm. For instance, the simplified outline algorithm could be a Douglas-Peucker algorithm with a 2.0 pixel tolerance.

[0135] Based on the aforementioned second-level localization, further nozzle identification is performed on the grayscale image, enabling the determined second region of interest to more accurately represent the nozzle. The first-level localization and recognition is based on the original image, providing a relatively large recognition range, thus expanding the field of view. Furthermore, the deep learning model used in the second-level localization and recognition can be relatively small, allowing for faster computation and rapid nozzle range capture. Moreover, since two-level recognition eliminates the need for directly using a relatively complex large model to perform high-precision processing on the original image, the computational resource requirements are relatively low, achieving a balance between computational resource requirements and computational accuracy. This allows the laser detection method provided in this application embodiment to be used on industrial control computers without GPUs.

[0136] like Figure 5 As shown, before step 250, the laser detection method may also include step 240.

[0137] Step 240: Determine the center position of the nozzle to be tested and whether the radius of the nozzle to be tested meets the preset standard by analyzing the first region of interest.

[0138] If the center position and radius of the nozzle to be detected in the first region of interest both meet the preset standards, the first region of interest is characterized as meeting the recognition criteria of the second-level deep learning model.

[0139] If either the center position or the radius of the nozzle to be tested does not meet the preset standard, proceed to step 260; if both the center position and the radius of the nozzle to be tested meet the preset standard, proceed to step 250.

[0140] Step 260: Based on the first region of interest, identify the laser region of the nozzle to be detected and obtain the laser center detection result.

[0141] Specifically, if the distance between the center position of the nozzle to be tested in the first region of interest and the edge of the first region of interest is not less than a distance threshold, it indicates that the center position of the nozzle to be tested meets a preset standard; if the radius of the nozzle to be tested in the first region of interest is less than a radius threshold, it indicates that the radius of the nozzle to be tested meets a preset standard.

[0142] Optionally, the distance threshold can be half of the first region of interest. The radius threshold can be determined based on the standard size used in image processing. For example, if the standard size used in image processing is 384×384, the radius threshold can be a value smaller than half of 384, such as 150. As another example, if the standard size is 256×256, the radius threshold could be 100, etc.

[0143] In an alternative implementation, such as Figure 6 As shown, step 270 above may include steps 271 to 273.

[0144] Step 271: Channel separation is performed based on the second region of interest to determine the three-channel tensor.

[0145] For example, the standard image size used in the laser center detection process can be the same as or different from the standard sizes used in the aforementioned steps. For instance, the standard size in the laser center detection process can be 256×256. A cropped image of the second region of interest can be obtained, resulting in a standard 256×256 image. This size image can crop only the edges of the nozzle, allowing for more accurate laser identification in subsequent processes.

[0146] The image of the second region of interest can be BGR, which can be converted to RGB image data.

[0147] The pixel values ​​of each pixel are first converted to 32-bit floating-point numbers, and then normalized to [0,1].

[0148] A three-channel tensor can be a four-dimensional tensor with three channels. The normalized data is separated into three channels, and the RGB three channels are separated and recombined into a four-dimensional tensor with three channels [1,3,256,256].

[0149] Step 272: Binarize the three-channel tensor to determine the intermediate detection results.

[0150] To prevent data overflow, the three-channel tensor can be pruned using the logit value, clipping it to [-500, 500] to prevent numerical overflow.

[0151] Alternatively, the sigmoid function can be used to convert the data into probability values, and then a 0.5 threshold can be used for binarization to obtain intermediate detection results.

[0152] Alternatively, an FP32 model can be used to perform the processing in step 272. Before using the FP32 model, CPU environment optimization can be performed: enable memory pool, memory mode, and full graph optimization; then thread configuration can be performed: dynamically configure the number of inference threads based on the number of CPU cores.

[0153] An FP32 model can be a Unet model with a precision of 32 float bits. Specifically, a Unet model with a precision of 32 float bits means that the weights and activation values ​​of the Unet model are stored and calculated in a 32-bit float format.

[0154] Step 273: Based on the intermediate detection results, determine the detection result of the laser center.

[0155] For example, the intermediate detection results can be enhanced and denoised to locate the laser center.

[0156] Optionally, step 273 may include steps 2731 to 2733.

[0157] Step 2731: Perform morphological enhancement processing on the intermediate detection results to determine the second enhanced image.

[0158] Optionally, morphological enhancement processing may include a hybrid of closing and opening operations. For example, the closing operation may be a 5×5 elliptic kernel closing operation; the opening operation may be a 3×3 elliptic kernel opening operation.

[0159] Step 2732: Identify the largest connected region in the second enhanced image.

[0160] To make the image smoother and clearer, the second enhanced image can be Gaussian smoothed, and then contour filtering can be performed based on the smoothed image data to determine the largest connected region.

[0161] For example, Gaussian smoothing can be a slightly blurred Gaussian smoothing with 3×3 kernels and 0.5 standard deviation.

[0162] For example, contour filtering methods may include area threshold filtering and selecting the largest connected component.

[0163] Step 2733: Determine the laser center detection result based on the largest connected region.

[0164] For example, polygon approximation calculation and minimum enclosing circle calculation can be performed on the largest connected region to determine the laser center detection result.

[0165] For example, polygon approximation calculation and minimum bounding circle calculation are performed on the largest connected region to determine the laser center, and then the laser center is mapped to the coordinate system of the original image to determine the laser center detection result.

[0166] In the overall implementation logic described above, the nozzle is first coarsely located using a first-level contour recognition, then finely located using a second-level deep learning model recognition, and finally the laser center is detected based on the nozzle center determined by the two-level positioning. Each stage of processing has dedicated preprocessing, inference, and post-processing steps, achieving a good balance between detection accuracy and processing efficiency.

[0167] In this embodiment, the nozzle identification achieved through first-level contour recognition and second-level deep learning model recognition can achieve a nozzle center positioning accuracy of 0.05mm, meeting the precision requirements of laser cutting. The laser center detection employs a high-precision algorithm, meeting the requirements of industrial laser processing. The detection process incorporates a roundness fitting algorithm, which has good adaptability and supports nozzles of different shapes and sizes. The method in this embodiment employs a three-level cascaded deep learning architecture, achieving high-precision automatic centering through a progressive refinement strategy of coarse positioning, fine positioning, and laser center detection.

[0168] Furthermore, embodiments of this application also provide a computer-readable storage medium storing a computer program, which, when executed by a processor, performs the steps of the laser detection method described in the above method embodiments.

[0169] The computer program product of the laser detection method provided in this application includes a computer-readable storage medium storing program code. The instructions included in the program code can be used to execute the steps of the laser detection method described in the above method embodiments. For details, please refer to the above method embodiments, which will not be repeated here.

[0170] In the several embodiments provided in this application, it should be understood that the disclosed methods can also be implemented in other ways. The method embodiments described above are merely illustrative. For example, the flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of methods and computer program products according to various embodiments of this application. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions marked in the blocks may occur in a different order than those marked in the drawings. For example, two consecutive blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in a block diagram and / or flowchart, and combinations of blocks in block diagrams and / or flowcharts, can be implemented using a dedicated hardware-based system that performs the specified function or action, or using a combination of dedicated hardware and computer instructions.

[0171] In addition, the method steps in the various embodiments of this application can be integrated together to form an independent part for execution, or each method step can be executed by a separate module, or two or more steps can be formed into an independent part for execution.

[0172] If the aforementioned functions are implemented as software functional modules and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks. It should be noted that in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0173] The above description is merely a preferred embodiment of this application and is not intended to limit this application. Various modifications and variations can be made to this application by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the protection scope of this application. It should be noted that similar reference numerals and letters in the following figures indicate similar items; therefore, once an item is defined in one figure, it does not need to be further defined and explained in subsequent figures.

[0174] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the technical scope disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A laser detection method, characterized in that, include: Obtain the original image, which includes the nozzle image region to be detected; Based on the original image, a first-level contour recognition is performed on the nozzle to be detected to obtain a first region of interest; the first region of interest includes the image region of the nozzle to be detected. If the first region of interest satisfies the recognition of the second-level deep learning model, the nozzle to be detected is recognized by the second-level deep learning model based on the first region of interest to obtain the second region of interest; the second region of interest includes the image region of the nozzle to be detected. Based on the second region of interest, the laser region of the nozzle to be detected is identified to obtain the laser center detection result.

2. The method according to claim 1, characterized in that, The step of performing first-level contour recognition on the nozzle to be detected based on the original image to obtain a first region of interest includes: The original image is converted to obtain the first image data in the target format; For each pixel in the first image data, perform binary classification to determine the first nozzle region and the first background region; The original image is segmented based on the first nozzle region to determine the first region of interest.

3. The method according to claim 2, characterized in that, The step of segmenting the original image based on the first nozzle region to determine the first region of interest includes: A first segmentation mask is determined based on the first nozzle region; The size of the first segmentation mask is restored to match the size of the original image to obtain the second segmentation mask; The original image is segmented based on the second segmentation mask to obtain a first segmented image; Based on the first segmented image, a first region of interest is obtained.

4. The method according to claim 3, characterized in that, The step of obtaining the first region of interest based on the first segmented image includes: The first segmented image is denoised to obtain a first denoised image; Contour extraction is performed on the first denoised image to determine one or more first valid contours; From one or more of the first valid contours, calculate the minimum bounding circle of the largest contour and determine it as the first region of interest.

5. The method according to claim 4, characterized in that, The step of denoising the first segmented image to obtain a first denoised image includes: The first segmented image is denoised by performing closing and opening operations on the elliptical structuring element to obtain the first denoised image.

6. The method according to claim 1, characterized in that, The step of performing a second-level deep learning model recognition on the nozzle to be detected based on the first region of interest to obtain a second region of interest includes: The first region of interest is transformed to determine the first tensor corresponding to the first region of interest. The first tensor is a single-channel tensor. The first tensor input image segmentation model is used to segment the image to determine the second nozzle region and the second background region. The first region of interest is processed based on the second nozzle region to determine the second region of interest.

7. The method according to claim 6, characterized in that, The step of processing the first region of interest based on the second nozzle region to determine the second region of interest includes: The first region of interest is segmented based on the second nozzle region to determine the second initial region of interest; Morphological enhancement processing is performed on the second initial region of interest to obtain the first enhanced image; Based on the first enhanced image, a second region of interest is determined.

8. The method according to claim 7, characterized in that, The step of determining the second region of interest based on the first enhanced image includes: The edges of the first enhanced image are smoothed to obtain a smoothed image; A simplified contour algorithm is used to simplify the contours of a smooth image to obtain a second region of interest.

9. The method according to any one of claims 1-8, characterized in that, Before performing a second-level deep learning model recognition on the nozzle to be detected based on the first region of interest to obtain the second region of interest, the method further includes: By analyzing the first region of interest, it is determined whether the center position and radius of the nozzle to be detected meet a preset standard; if both the center position and radius of the nozzle to be detected meet the preset standard, it indicates that the first region of interest satisfies the recognition of the second-level deep learning model. If either the center position of the nozzle to be tested or the radius of the nozzle to be tested does not meet the preset standard, the method further includes: identifying the laser region of the nozzle to be tested based on the first region of interest to obtain a laser center detection result; Wherein, if the distance between the center position of the nozzle to be tested and the edge of the first region of interest is not less than a distance threshold, it indicates that the center position of the nozzle to be tested meets a preset standard; if the radius of the nozzle to be tested is less than a radius threshold, it indicates that the radius of the nozzle to be tested meets a preset standard.

10. The method according to claim 1, characterized in that, The step of identifying the laser region of the nozzle to be detected based on the second region of interest to obtain the laser center detection result includes: Channel separation is performed based on the second region of interest to determine the three-channel tensor; The three-channel tensor is binarized to determine the intermediate detection results; Based on the intermediate detection results, the laser center detection result is determined.

11. The method according to claim 10, characterized in that, The determination of the laser center detection result based on the intermediate detection results includes: The intermediate detection results are subjected to morphological enhancement processing to determine the second enhanced image; The largest connected region is identified in the second enhanced image; Based on the largest connected region, the laser center detection result is determined.

12. The method according to claim 11, characterized in that, The determination of the laser center detection result based on the maximum connected region includes: Polygon approximation calculation and minimum enclosing circle calculation are performed on the largest connected region to determine the laser center detection result.

13. The method according to any one of claims 1-8, characterized in that, The method further includes: Based on the laser center detection results, the offset between the laser center and the center of the nozzle to be detected is determined; The laser center is adjusted based on the offset between the laser center and the center of the nozzle to be tested.

14. An electronic device, characterized in that, include: The processor and memory, wherein the memory stores machine-readable instructions executable by the processor, wherein when the electronic device is running, the machine-readable instructions are executed by the processor to perform the steps of the method as described in any one of claims 1 to 13.

15. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, performs the steps of the method as described in any one of claims 1 to 13.

16. A computer program product, characterized in that, The computer program product includes a computer program that, when executed by a processor, implements the method described in any one of claims 1 to 13.

Images