Tool state detection method, device, electronic device and readable storage medium

Abstract

The present invention discloses a tool state detection method, device, electronic device and readable storage medium, and relates to the technical field of numerical control machining. The method comprises the following steps: acquiring image data of the tool and the machining area, acquiring point cloud data of the machining workpiece and / or acquiring point cloud data of the tool; preprocessing the image data; acquiring feature information of the tool through an image processing algorithm, and judging the defect state of the tool based on the feature information of the tool; determining the machining error of the machining workpiece based on the point cloud data of the machining workpiece and the 3D model of the machining workpiece; preloading the machining program based on the point cloud data of the tool and the point cloud data of the machining workpiece, and judging the interference state of the tool and the machining workpiece; generating an alarm signal or a control instruction based on the defect state of the tool, the machining error of the machining workpiece and / or the interference state of the tool and the machining workpiece. The method has the advantages of real-time and high reliability of detection results.

Description

Technical Field

[0001] The present invention relates to the technical field of numerical control machining, and in particular to a tool state detection method, device, electronic equipment and readable storage medium. Background Art

[0002] In the manufacturing industry, CNC machine tools are commonly used processing equipment. During the use of CNC machine tools, the tool status will have a great impact on the CNC processing process. Abnormal tool status will lead to processing errors, resulting in a large number of workpieces being scrapped. Therefore, when performing CNC processing, it is particularly important to monitor the tool and its processing status of the CNC machine tool in real time.

[0003] In the prior art, the tool status is detected by the operator visually or by the operator using a measuring tool. The detection method in the prior art has the following problems: (1) When the operator detects the tool status, the machine tool is in a shutdown state and cannot be detected in real time, resulting in low machine tool production efficiency. Moreover, when the tool is damaged, using the damaged tool for processing will result in a large number of workpieces being scrapped. (2) The operator needs to have certain experience and skills, which results in high labor costs. Moreover, the detection depends on the operator, which may result in subjective errors. The accuracy of the tool status judgment result cannot be guaranteed, resulting in poor product quality stability.

[0004] Therefore, providing a method for detecting the tool status online with high accuracy is a technical problem that needs to be solved urgently by those skilled in the art. Summary of the invention

[0005] The present invention discloses a tool state detection method, device, electronic device and readable storage medium to solve the technical problems in the related art of detecting the tool state by an operator, resulting in low machine tool production efficiency and poor accuracy of tool state judgment results.

[0006] In order to solve the above problems, the present invention adopts the following technical solutions:

[0007] A first aspect of the present invention provides a tool state detection method.

[0008] The tool state detection method of the present invention comprises the following steps:

[0009] During the operation of the machine tool, image data of the tool and the processing area are obtained, point cloud data of the processed workpiece is obtained, and / or point cloud data of the tool is obtained;

[0010] Preprocessing the image data to obtain denoised and enhanced image data;

[0011] Based on the denoised and enhanced image data, the characteristic information of the tool is obtained through an image processing algorithm, and the defect state of the tool is determined based on the characteristic information of the tool;

[0012] Determine a machining error of the machined workpiece based on point cloud data of the machined workpiece and a 3D model of the machined workpiece;

[0013] Based on the point cloud data of the tool and the workpiece, the machining program is preloaded to determine the interference status between the tool and the workpiece;

[0014] Based on the defect state of the tool, the machining error of the machined workpiece and / or the interference state between the tool and the machined workpiece, an alarm signal or a control instruction is generated.

[0015] Furthermore, the image data is preprocessed, which at least includes eliminating one or more of water mist information, water drop information, dust information and waste residue information in the image data.

[0016] Furthermore, when the image data is preprocessed, at least the following steps are included:

[0017] Extract image edge: Based on the image data, convolution is performed with a 3*3 horizontal edge detection operator and a 3*3 vertical edge detection operator to obtain the image edge. The calculation formula is as follows:

[0018] ,

[0019] ,

[0020] Among them, f(x,y) is the original two-dimensional image matrix at the (x,y) coordinate point, each element contains 3 values, namely RGB red, green and blue brightness values, x and y are coordinates, g x (x, y) is the horizontal edge of the image extracted at the (x, y) coordinate point, g y (x,y) is the vertical edge of the image extracted at the (x,y) coordinate point;

[0021] Calculate the brightness map of the image edge and analyze the connected domain: Calculate the brightness map of the image edge based on the extracted horizontal edge and vertical edge of the image, and extract the edge area with brightness greater than 30. The calculation formula is as follows:

[0022] ,

[0023] ,

[0024] Among them, E(x,y) is the brightness map of the edge of the image at the (x,y) coordinate point, and ER(x,y) is the edge area at the (x,y) coordinate point;

[0025] Based on the brightness map of the image edge, the connected region analysis algorithm is used to obtain the connected region labels of the image edge contour. The connected region analysis algorithm is expressed as follows:

[0026] ,

[0027] Among them, P(x,y) is the label of the contour connected area at the (x,y) coordinate point;

[0028] Filter contours based on contour roundness: Calculate the contour roundness of the connected area and calculate the contour roundness using the following formula:

[0029] ,

[0030] ,

[0031] ,

[0032] ,

[0033] ,

[0034] ,

[0035] ,

[0036] in, , are the coordinates of the center point of the region, is the contour area, which refers to the number of contour pixels. is the average distance from the pixel point on the contour to the center, is the deviation between the distance from the contour pixel to the center and the average distance; is the contour roundness, is the pixel point belonging to the connected area i;

[0037] The part with contour roundness greater than the preset value is used as the background image. The expression of the background image is:

[0038] ,

[0039] Among them, B(x,y) is the background image at the coordinate point (x,y);

[0040] Image extraction: Based on the filtered background image, remove the image of water mist, water droplets, dust and waste residue Expressed as follows:

[0041] ,

[0042] Among them, S(x,y) is the image at the (x,y) coordinate point with water mist, water droplets, dust and waste residue removed;

[0043] Image difference and fusion: Subtract the images obtained after the sequence matting to get the difference image, and perform image fusion to get the denoised image. The image difference and fusion are calculated using the following formula:

[0044] ,

[0045] Among them, M is the fused image, N is the number of images, is the kth image, is the fusion factor, =1 / N;

[0046] Image contrast stretching: Based on the pixel values ​​of the fused image, each pixel value is linearly transformed to stretch the contrast, and the pixel value is linearly transformed using the following formula:

[0047] ,

[0048] Where I(x,y) is the pixel value of the xth row and yth column on the stretched image, M(x,y) is the pixel value of the xth row and yth column on the fused image, Mmin is the minimum pixel value of the fused image, and Mmax is the maximum pixel value of the fused image.

[0049] Furthermore, based on the denoised and enhanced image data, characteristic information of the tool is obtained through an image processing algorithm, and the defect state of the tool is determined based on the characteristic information of the tool, which at least includes the following steps:

[0050] Based on the brightness of the edge of the image, the distance between the tool contour point and the normal contour line of the tool is calculated, and the tool defect is detected based on the difference between the distance and a set threshold.

[0051] Furthermore, the distance between the actual contour line of the tool and the normal contour line of the tool is calculated as follows:

[0052] ,

[0053] Among them, (x, y) is the coordinate of the actual contour point of the tool, (x′, y′) is the coordinate of the normal contour point of the tool, is the actual contour label value of the tool, where is a conditional term, != means not equal, 0 is a point in a non-contour position, is the normal contour label value of the tool. When (x, y) is the non-contour point position of the tool, set ,

[0054] min means the minimum value, To traverse all normal contour points (x′, y′) of the tool, find the minimum distance between the actual contour line of the tool and the normal contour line of the tool,

[0055] When the minimum distance between the actual contour line of the tool and the normal contour line of the tool is greater than the set threshold, it is judged that the tool has a defect at (x, y); when the minimum distance between the actual contour line of the tool and the normal contour line of the tool is less than or equal to the set threshold, it is judged that the tool does not have a defect at (x, y).

[0056] Further, determining the machining error of the machined workpiece based on the point cloud data of the machined workpiece and the 3D model of the machined workpiece includes at least the following steps:

[0057] Point cloud registration: Use the iterative closest point algorithm to register the collected 3D point cloud data with the 3D model data of the processed workpiece;

[0058] Machining error analysis: Use one or more methods of distance comparison, surface fitting, feature extraction and comparison, topological relationship comparison and statistical analysis to determine the error between the machined workpiece and the 3D model of the machined workpiece.

[0059] Furthermore, based on the point cloud data of the tool and the point cloud data of the workpiece being processed, a processing program is preloaded to determine the interference state between the tool and the workpiece being processed, which at least includes the following steps:

[0060] Construct tool point cloud: construct tool point cloud based on the acquired point cloud data of the tool; construct machining workpiece point cloud: construct machining workpiece point cloud based on the acquired point cloud data of the machining workpiece;

[0061] Determine the interference status between the tool and the workpiece: preload the machining program, calculate the point cloud intersection of the tool point cloud and the workpiece point cloud. When the point cloud intersection is empty, there is no interference between the tool and the workpiece. When the point cloud intersection is not empty, there is interference between the tool and the workpiece.

[0062] A second aspect of the present invention provides a tool state detection device.

[0063] The tool state detection device of the present invention is used to implement the tool state detection method described in any technical solution of the present invention, and the tool state detection device includes:

[0064] Acquisition module: used to obtain image data of the tool and the processing area, obtain point cloud data of the processed workpiece and / or obtain point cloud data of the tool during the operation of the machine tool;

[0065] Preprocessing module: used to preprocess the image data and obtain denoised and enhanced image data;

[0066] The first processing module is used to obtain the characteristic information of the tool through the image processing algorithm based on the denoised and enhanced image data, and to judge the defect state of the tool based on the characteristic information of the tool;

[0067] A second processing module: used for determining a machining error of the machined workpiece based on the point cloud data of the machined workpiece and the 3D model of the machined workpiece;

[0068] The third processing module is used to preload the processing program based on the point cloud data of the tool and the point cloud data of the processed workpiece, and to determine the interference state between the tool and the processed workpiece;

[0069] Output module: used to generate an alarm signal or a control instruction based on the defect state of the tool, the machining error of the machined workpiece and / or the interference state between the tool and the machined workpiece.

[0070] A third aspect of the present invention provides an electronic device.

[0071] The electronic device of the present invention comprises a memory and a processor, wherein the memory stores a program or instruction that can be run on the processor, and when the program or instruction is executed by the processor, the steps of the tool state detection method as described in any technical solution of the present invention are implemented.

[0072] A fourth aspect of the present invention provides a readable storage medium.

[0073] The readable storage medium of the present invention stores a program or instruction on the readable storage medium, and when the program or instruction is executed by a processor, the steps of the tool state detection method as described in any technical solution of the present invention are implemented.

[0074] The technical solution adopted by the present invention can achieve the following beneficial effects:

[0075] First, the tool state detection method of the present invention can detect the state of the tool in real time through a visual system, and generate an alarm signal or control command when the tool is abnormal. Compared with the traditional method of detecting the tool state through the operator's visual inspection or the operator using a measuring tool, this detection method is real-time, does not require manual operation or offline analysis, improves detection efficiency, saves labor costs, and also improves the production efficiency of machine tools. It can detect tool abnormalities in time, reduce interruptions in the production process and reduce the amount of scrapped workpieces; in addition, this detection method can reduce manual intervention in processing results, reduce dependence on operators, make the judgment results highly reliable, and improve product quality stability; this detection method also has the advantage of wide applicability and can be applied to various tools and processing scenarios.

[0076] Secondly, the tool state detection method of the present invention can simultaneously judge the three indicators of the defect state of the tool, the processing error of the workpiece, and the interference state of the tool and the workpiece. It can simultaneously detect the states of the tool and the workpiece and the relationship between the two, realize comprehensive monitoring of the processing process, and can more comprehensively and accurately judge the processing quality. At the same time, the detection method of the present invention is used to simultaneously detect the three indicators, which can not only improve the detection efficiency, but also avoid the misjudgment or missed judgment that may be caused by the detection of a single indicator. BRIEF DESCRIPTION OF THE DRAWINGS

[0077] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0078] Figure 1 is a flow chart of a tool state detection method according to an embodiment of the present application;

[0079] Figure 2 is a schematic diagram of the installation position of the camera in the embodiment of the present application;

[0080] Figure 3 It is a module diagram of a tool state detection device according to an embodiment of the present application;

[0081] Figure 4 It is a module diagram of the electronic device according to an embodiment of the present application.

[0082] In the figure: 11, acquisition module; 12, pre-processing module; 13, first processing module; 14, second processing module; 15, third processing module; 16, output module; 21, memory; 22, processor; 31, machine tool spindle; 32, processing workpiece; 33, camera; 34, PC. DETAILED DESCRIPTION

[0083] To make the purpose, technical solution and advantages of the present invention clearer, the technical solution of the present invention will be described in detail below. Obviously, the described embodiments are only part of the embodiments of the present invention, rather than all the embodiments. Based on the embodiments of the present invention, all other implementation methods obtained by ordinary technicians in this field without creative work belong to the scope of protection of the present invention.

[0084] The terms "first", "second", etc. in the specification and claims of the present application are used to distinguish similar objects, and are not used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable under appropriate circumstances, so that the embodiments of the present application can be implemented in an order other than those illustrated or described here, and the objects distinguished by "first", "second", etc. are generally of one type, and the number of objects is not limited. For example, the first object can be one or more. In addition, "and / or" in the specification and claims represents at least one of the connected objects, and the character " / " generally indicates that the objects associated with each other are in an "or" relationship.

[0085] The following is combined with Figures 1 to 4 , the tool state detection method, device, electronic device and readable storage medium provided in the embodiments of the present application are described in detail through specific embodiments and their application scenarios.

[0086] The first aspect of this embodiment describes the tool status detection method in detail.

[0087] Figure 1 FIG. 2 shows a flow chart of the tool state detection method of this embodiment. Figure 1 As shown, the tool state detection method of this embodiment includes the following steps:

[0088] S100: During the operation of the machine tool, image data of the tool and the processing area are obtained, point cloud data of the processed workpiece is obtained, and / or point cloud data of the tool is obtained.

[0089] For example, the image data of the tool and the processing area can be obtained by taking photos or recording videos with a 2D camera, and the point cloud data of the processed workpiece and the point cloud data of the tool can be obtained with a 3D camera.

[0090] The acquired image data may include the tool and the workpiece, or may only include the tool. Preferably, the image data may be collected at regular intervals or after a specific operation is performed. Exemplarily, the image data of the tool and the processing area is collected after operations such as drilling or tapping are performed. The acquired image data can not only be used to subsequently determine the defect state of the tool, but also to detect whether the drilling or tapping operation is performed in place. By collecting data once, two levels of detection can be performed, which helps to improve detection efficiency and also reduces the detection system's requirements for data storage.

[0091] Figure 2 FIG. 1 is a schematic diagram of the installation position of the camera in this embodiment. Figure 2As shown, the tool is installed under the machine tool spindle 31, the tool is used to process the workpiece 32, and the camera 33 is installed on the machine tool. The distance between the camera 33 and the machine tool spindle 31 and / or the workpiece 32 is 10-20 cm, so that a clear image containing the tool and the workpiece 32 can be captured. The camera 33 can also send the collected image to the PC 34 for data processing.

[0092] S200: Preprocess the image data and obtain denoised and enhanced image data.

[0093] Preferably, the image data is preprocessed, at least including eliminating one or more of water mist information, water droplet information, dust information and waste residue information in the image data. The environment in which the tool and the workpiece are located is complex, and water mist, water droplets, dust and waste residue may adhere to the tool. By removing water mist, water droplets, dust and waste residue, the image clarity can be improved, the shape of the tool and the workpiece can be enhanced, and the water mist, water droplets, dust and waste residue can be prevented from interfering with the extraction and recognition of the feature area, thereby improving the reliability of the tool state detection result.

[0094] S300: Based on the denoised and enhanced image data, characteristic information of the tool is obtained through an image processing algorithm, and the defect state of the tool is determined based on the characteristic information of the tool.

[0095] The characteristic information of the tool mainly refers to the basic information of the tool. For example, the characteristic information of the tool includes: whether the tool is in the processing position, whether the tool is intact, the degree of tool wear, etc.

[0096] The defective state of the tool directly affects the product accuracy. Detecting the defective state of the tool can ensure product accuracy, improve production efficiency, and reduce the scrap rate of processed workpieces. The tool is a key component in the processing process. The defect of the tool directly affects the product accuracy. When the tool has a defect, an alarm signal is issued, which helps to reduce the scrap rate of the processed workpiece and reduce production costs.

[0097] S400: Determine a machining error of the machined workpiece based on the point cloud data of the machined workpiece and the 3D model of the machined workpiece.

[0098] Detecting machining errors of workpieces can timely identify problems in the machining process and make adjustments or corrections, which helps to reduce machining errors and improve product quality consistency.

[0099] When an error occurs in the processed workpiece, it indicates that the processed workpiece is a defective product. When an error occurs in the processed workpiece, an alarm signal is issued, which helps to reduce the scrap rate of the processed workpiece and reduce production costs.

[0100] S500: Based on the point cloud data of the tool and the point cloud data of the workpiece being processed, a processing program is preloaded to determine the interference state between the tool and the workpiece being processed.

[0101] Detecting the interference status between the tool and the workpiece and sending out an alarm signal when interference occurs between the tool and the workpiece helps to protect the integrity of the tool and the workpiece and increase the service life of the tool. It can also avoid equipment damage and processing errors caused by interference and further ensure the quality of the processed workpiece.

[0102] S600: Generate an alarm signal or a control instruction based on a defect state of the tool, a machining error of the workpiece, and / or an interference state between the tool and the workpiece.

[0103] The tool state detection method of this embodiment detects three indicators: the defect state of the tool, the processing error of the workpiece, and the interference state of the tool and the workpiece. The three indicators can be calculated and judged simultaneously. As long as any one of the three indicators does not meet the preset requirements, an alarm signal is generated; or when any two or more of the three indicators do not meet the preset requirements, an alarm signal is generated.

[0104] The tool state detection method of the present embodiment simultaneously detects three indicators: the defect state of the tool, the machining error of the workpiece, and the interference state of the tool and the workpiece. It can simultaneously detect the states of the tool and the workpiece and the relationship between the two, thereby achieving comprehensive monitoring of the machining process and making a more comprehensive and accurate judgment of the machining quality. Using the detection method of the present invention to simultaneously detect the three indicators can not only improve the detection efficiency, but also avoid misjudgment or missed judgment that may be caused by single indicator detection.

[0105] It can be known that the three indicators of the defect state of the tool, the machining error of the workpiece, and the interference state between the tool and the workpiece can also be calculated and judged one by one, that is: first calculate the defect state of the tool, generate an alarm signal when the tool has defects, and when the tool has no defects, calculate the machining error of the workpiece; generate an alarm signal when the machining error of the workpiece is greater than a preset value, and when the machining error of the workpiece is less than a preset value, calculate the interference state between the tool and the workpiece; generate an alarm signal when interference occurs between the tool and the workpiece, and generate a control instruction to continue processing when there is no interference between the tool and the workpiece.

[0106] In the tool state detection method of the present embodiment, three indicators, namely the defect state of the tool, the machining error of the workpiece, and the interference state of the tool and the workpiece, are calculated and judged one by one. The next indicator is judged only when the previous indicator meets the preset requirements. This can reduce the calculation amount of the detection system and the demand for data storage of the detection system. In addition, the calculation of a single indicator has the advantage of high calculation efficiency and reduces the hardware requirements for the detection device.

[0107] The tool status detection method of the present embodiment can detect the status of the tool in real time through the visual system, and generate an alarm signal or control command when the tool is abnormal. Compared with the traditional method of detecting the tool status through the operator's visual inspection or the operator using a measuring tool, this detection method is real-time, does not require manual operation or offline analysis, improves detection efficiency, saves labor costs, and also improves the production efficiency of machine tools. It can detect tool abnormalities in time, reduce interruptions in the production process and reduce the amount of workpiece scrap; in addition, this detection method can reduce manual intervention in processing results, reduce dependence on operators, make the judgment results more reliable, and improve product quality stability; this detection method also has the advantage of wide applicability and can be applied to various tools and processing scenarios.

[0108] According to an optional implementation, preprocessing the image data includes at least the following steps:

[0109] S210: Extract image edge. Based on the image data, convolve with a 3*3 horizontal edge detection operator and a 3*3 vertical edge detection operator to obtain the image edge. The calculation formula is as follows:

[0110] ,

[0111] ,

[0112] Among them, f(x,y) is the original two-dimensional image matrix at the (x,y) coordinate point, each element contains 3 values, namely RGB red, green and blue brightness values, x and y are coordinates, g x (x, y) is the horizontal edge of the image extracted at the (x, y) coordinate point, g y (x,y) is the vertical edge of the image extracted at the (x,y) coordinate point.

[0113] S220: Calculate the brightness map of the image edge and analyze the connected domain. Calculate the brightness map of the image edge based on the extracted horizontal edge and vertical edge of the image, and extract the edge area with brightness greater than 30. The calculation formula is as follows:

[0114] ,

[0115] ,

[0116] The threshold of edge brightness is not limited to 30, but may be other values, such as 20, 40, 60, etc.

[0117] Among them, E(x,y) is the brightness map of the edge of the image at the (x,y) coordinate point, and ER(x,y) is the edge area at the (x,y) coordinate point.

[0118] Based on the brightness map of the image edge, a connected region analysis algorithm is used to obtain the connected region labels of the image edge contour. Specifically, a four-neighborhood connected region analysis is performed here. More specifically, a Two-Pass algorithm is used. The Two-Pass algorithm is an existing technology. Specifically, the Two-Pass algorithm needs to undergo two scans.

[0119] First scan:

[0120] Access the current pixel ER(x,y), if ER(x,y)=1;

[0121] (1) If all pixel values ​​in the domain of ER(x,y) are 0, a new label i is assigned to ER(x,y);

[0122] I=i+1, ER(x,y)=i.

[0123] (2) If there are pixels with values ​​> 1 in the domain of ER(x,y):

[0124] The minimum value among the neighborhood pixels is assigned to ER(x,y), and the equality relationship between the label values ​​i in the neighborhood pixels is recorded, that is, these label values ​​belong to the same connected area.

[0125] Second scan:

[0126] Visit the current pixel ER(x,y). If ER(x,y)>1, find the minimum label value that belongs to the same connected region as ER(x,y) and assign it to ER(x,y). After the scan is completed, the pixels with the same label value in the image form the same connected region.

[0127] The connected region analysis algorithm is expressed as follows:

[0128] ,

[0129] Among them, P(x,y) is the label of the contour connected area at the (x,y) coordinate point.

[0130] S230: Screen the contours based on the contour roundness. Calculate the contour roundness of the connected area, and calculate the contour roundness by the following formula:

[0131] ,

[0132] ,

[0133] ,

[0134] ,

[0135] ,

[0136] ,

[0137] ,

[0138] in, , are the coordinates of the center point of the region, is the contour area, which refers to the number of contour pixels. is the average distance from the pixel point on the contour to the center, is the deviation between the distance from the contour pixel to the center and the average distance; is the contour roundness, are the pixels belonging to the connected region i.

[0139] The part with contour roundness greater than the preset value is used as the background image. The expression of the background image is:

[0140] ,

[0141] Among them, B(x,y) is the background image at the (x,y) coordinate point.

[0142] The closer it is to 1, the closer the contour is to a circle, which can be considered as interference such as water mist, water drops, dust and waste residue. Therefore, the threshold of contour roundness is set to 0.8. It can be seen that the threshold of contour roundness can also be other values, such as 0.85, 0.9, etc.

[0143] S240: Image extraction. Based on the filtered background image, remove the image of water mist, water droplets, dust and waste residue Expressed as follows:

[0144] ,

[0145] Among them, S (x, y) is the image at the (x, y) coordinate point, with water mist, water droplets, dust and waste residue removed.

[0146] It can be seen from the above expression that the images of areas such as water mist, water drops, dust and waste residue in the image are filled with 0, and the images of other areas are retained.

[0147] S250: Image difference and fusion. Subtract the images obtained after the sequence matting to obtain a difference image, and perform image fusion to obtain a denoised image. The image difference is calculated and fused using the following formula:

[0148] ,

[0149] Among them, M is the fused image, N is the number of images, is the kth image, is the fusion factor, =1 / N. For example, N is 10, 5 images are taken per second, and the difference images of 2 seconds are fused into one image. =1 / 10.

[0150] S260: Image contrast stretching. Based on the fused image pixel values, each pixel value is linearly transformed to stretch the contrast, and the pixel values ​​are linearly transformed using the following formula:

[0151] ,

[0152] Where I(x,y) is the pixel value of the xth row and yth column on the stretched image, M(x,y) is the pixel value of the xth row and yth column on the fused image, Mmin is the minimum pixel value of the fused image, and Mmax is the maximum pixel value of the fused image.

[0153] The tool state detection method of the preferred technical solution of this embodiment performs preprocessing on image data including two parts: denoising and enhancement. First, the water mist, water droplets, dust and waste residue in the image are removed by the roundness of the image edge, thereby improving the image clarity, enhancing the shape of the tool and the processed workpiece, and avoiding the interference of water mist, water droplets, dust and waste residue in the extraction and recognition of the feature area, thereby improving the reliability of the tool state detection result; secondly, the image clarity is enhanced by two steps of image difference fusion and image contrast stretching. Specifically, image difference fusion can enhance image details, reduce noise, and make the image clearer; after image difference fusion, image contrast stretching is performed to further enhance the contrast of the image, making the brightness difference in the image more obvious, thereby further enhancing the clarity of the image, making the details of the image more prominent, which is beneficial to the subsequent feature recognition and extraction of the tool and the processed workpiece.

[0154] According to an optional implementation, based on the denoised and enhanced image data, the characteristic information of the tool is obtained through an image processing algorithm, and the defect state of the tool is judged based on the characteristic information of the tool, and the characteristic information of the tool is, for example, the contour information of the tool. Judging the defect state of the tool includes at least the following steps: based on the brightness of the edge of the image, calculating the distance between the contour point of the tool and the normal contour line of the tool, and detecting the defect of the tool based on the size of the distance and the set threshold.

[0155] Preferably, the distance between the actual contour line of the tool and the normal contour line of the tool is calculated as follows:

[0156] ,

[0157] Among them, (x, y) is the coordinate of the actual contour point of the tool, (x′, y′) is the coordinate of the normal contour point of the tool, is the actual contour label value of the tool, where is a conditional term, != means not equal, 0 is a point in a non-contour position, is the normal contour label value of the tool. When (x, y) is the non-contour point position of the tool, set .

[0158] min means the minimum value, To traverse all normal contour points (x′, y′) of the tool, find the minimum distance between the actual contour line of the tool and the normal contour line of the tool.

[0159] When the minimum distance between the actual contour line of the tool and the normal contour line of the tool is greater than the set threshold, it is judged that the tool has a defect at (x, y); when the minimum distance between the actual contour line of the tool and the normal contour line of the tool is less than or equal to the set threshold, it is judged that the tool does not have a defect at (x, y).

[0160] The threshold here can be set based on user needs. If the user has high requirements for the precision of the workpiece being processed, the threshold can be set smaller, such as 0.1 mm; if the user has low requirements for the precision of the workpiece being processed, the threshold can be set larger, such as 1 mm.

[0161] The tool state detection method of the preferred technical solution of this embodiment performs binarization processing on the image before obtaining the tool contour, which not only helps to remove background noise and highlight the bright details in the image, but also simplifies the image and reduces the amount of data, thereby improving the subsequent image processing speed; then, based on the extracted tool contour, the distance between the tool contour point and the normal contour line of the tool is calculated to determine whether the tool is defective, thereby determining the defective state of the tool.

[0162] The detection and analysis of the tool contour line can be completed in a relatively short time, and the tool contour line can provide high-precision geometric information of the tool. The distance between the tool contour point and the normal contour line of the tool can be used to determine whether the tool is defective. Tiny defects or deformations of the tool can be detected. It has the advantages of high detection accuracy and fast detection speed. At the same time, this detection method is non-contact detection, which will not cause additional damage to the tool and will not affect the production process.

[0163] According to an optional embodiment, determining a machining error of a machined workpiece based on point cloud data of the machined workpiece and a 3D model of the machined workpiece comprises at least the following steps:

[0164] S410: Point cloud registration: The collected 3D point cloud data is registered with the 3D model data of the workpiece by using an iterative closest point algorithm.

[0165] The iterative closest point algorithm is a prior art and will not be described in detail here.

[0166] S420: Processing error analysis: using one or more methods of distance comparison, surface fitting, feature extraction and comparison, topological relationship comparison and statistical analysis to determine the error between the processed workpiece and the 3D model of the processed workpiece.

[0167] Distance comparison: For each point on the workpiece, calculate the distance to the nearest model point. By statistically analyzing the distribution of these distances, we can understand the overall distance deviation between the workpiece and the model, and thus evaluate the machining accuracy.

[0168] Surface fitting: Use a surface fitting algorithm (such as least squares fitting) to fit the point cloud data of the machined workpiece and the model into a surface, and then compare the geometric features of the two surfaces, such as curvature, normal direction, etc., to evaluate the shape error between the machined workpiece and the model.

[0169] Feature extraction and comparison: Extract feature points or feature areas (such as local surface curvature extreme points, corner points, etc.) of the processed workpiece and model, and then compare the correspondence and attribute differences between them to evaluate the processing error.

[0170] Topological relationship comparison: Analyze the topological relationship between the workpiece and the model, and compare whether their topological structures are consistent, such as boundary curves, facet connections, etc., so as to evaluate possible shape deformation or topological structure changes during the processing process.

[0171] Statistical analysis: Statistical analysis is performed on various comparative data between the workpiece and the model, such as mean, variance, distribution, etc., to fully understand the distribution and characteristics of the machining error.

[0172] The above methods can all be methods in the prior art and will not be described in detail here.

[0173] Without limitation thereto, after the collected 3D point cloud data is registered with the 3D model data of the workpiece being processed, the processing error of the workpiece may also be determined by other methods.

[0174] The tool state detection method of the preferred technical solution of this embodiment detects the processing error of the model after point cloud registration. This detection method is a non-contact detection and will not cause additional damage to the processed workpiece. It is also suitable for the detection of complex surfaces or internal structures. This detection method has high detection accuracy and can effectively detect tiny processing errors, and can achieve sub-millimeter or even sub-micron accuracy. This detection method also has a global perspective, which can not only detect local subtle errors of the processed workpiece, but also detect changes or distortions in the overall shape of the processed workpiece.

[0175] According to an optional implementation, based on the point cloud data of the tool and the point cloud information of the workpiece being processed, a processing program is preloaded, and the interference state between the tool and the workpiece being processed is determined, which at least includes the following steps:

[0176] S510: Constructing a tool point cloud. Constructing a tool point cloud based on the acquired point cloud data of the tool.

[0177] S520: Constructing a point cloud of the workpiece. Constructing a point cloud of the workpiece based on the acquired point cloud data of the workpiece.

[0178] S530: Determine the interference state between the tool and the workpiece: preload the machining program, and during the tool movement, calculate the point cloud intersection of the tool point cloud and the workpiece point cloud. When the point cloud intersection is empty, there is no interference between the tool and the workpiece. When the point cloud intersection is not empty, there is interference between the tool and the workpiece.

[0179] Specifically, all the data in the tool point cloud are traversed to check whether the corresponding workpiece point cloud exists. If so, it indicates that the tool point cloud and the workpiece point cloud have overlapping points, the intersection of the two is non-empty, and there is interference between the tool and the workpiece; conversely, if it does not exist, it indicates that the tool point cloud and the workpiece point cloud do not have overlapping points, the intersection of the two is empty, and there is no interference between the tool and the workpiece.

[0180] The tool state detection method of the preferred technical solution of this embodiment detects the interference state of the tool and the workpiece by constructing a point cloud. This detection method can predict the collision state between the tool, the workpiece and the machine tool body in advance, and can timely discover potential collision or interference problems, reduce the occurrence of accidents, and avoid downtime caused by interference problems; at the same time, it can also protect the machine tool, tool and workpiece to avoid equipment damage or workpiece quality problems caused by collision, thereby ensuring the processing quality of the workpiece.

[0181] The second aspect of this embodiment describes the tool state detection device in detail.

[0182] The tool state detection device of this embodiment is used to implement the tool state detection method of any technical solution of this embodiment. The tool state detection device of this embodiment and the aforementioned tool detection method can be referred to each other.

[0183] Figure 3 FIG. 2 shows a module diagram of the tool state detection device of this embodiment. Figure 3 As shown, the tool state detection device of this embodiment includes a collection module 11 , a pre-processing module 12 , a first processing module 13 , a second processing module 14 , a third processing module 15 and an output module 16 .

[0184] like Figure 3As shown, the acquisition module 11 is connected to the preprocessing module 12, the first processing module 13 is connected to the preprocessing module 12, the second processing module 14 is connected to the acquisition module 11, the third processing module 15 is connected to the acquisition module 11 and the first processing module 13, and the first processing module 13, the second processing module 14 and the third processing module 15 are also connected to the output module 16.

[0185] Preferably, the acquisition module 11 is used to acquire image data of the tool and the processing area, acquire point cloud data of the processed workpiece and / or acquire point cloud data of the tool during the operation of the machine tool.

[0186] Preferably, the preprocessing module 12 is used to preprocess the image data and obtain denoised and enhanced image data.

[0187] Preferably, the first processing module 13 is used to obtain characteristic information of the tool through an image processing algorithm based on the denoised and enhanced image data, and to determine the defect state of the tool based on the characteristic information of the tool.

[0188] Preferably, the second processing module 14 is used to determine a machining error of the machined workpiece based on the point cloud data of the machined workpiece and the 3D model of the machined workpiece.

[0189] Preferably, the third processing module 15 is used to preload the processing program based on the point cloud data of the tool and the point cloud information of the processed workpiece, and to determine the interference state between the tool and the processed workpiece.

[0190] Preferably, the output module 16 is used to generate an alarm signal or a control instruction based on a defect state of the tool, a machining error of the machined workpiece, and / or an interference state between the tool and the machined workpiece.

[0191] The specific functions of the acquisition module 11, the preprocessing module 12, the first processing module 13, the second processing module 14, the third processing module 15 and the output module 16 can be referred to the specific description in the method embodiment, which will not be repeated here.

[0192] The tool state detection device of this embodiment is used to implement the tool state detection method of any technical solution in this embodiment, so that the tool state detection has the advantages of real-time and high reliability.

[0193] The third aspect of this embodiment describes the electronic device in detail.

[0194] The electronic device of this embodiment includes a memory 21 and a processor 22, and the memory 21 and the processor 22 are connected, such as Figure 4The memory 21 stores programs or instructions that can be run on the processor 22. When the programs or instructions are executed by the processor 22, the steps of the tool state detection method of any technical solution in this embodiment are implemented. The memory 21 can exist independently or be integrated with the processor 22.

[0195] The memory 21 may include a random access memory (RAM) or a read-only memory (ROM). The memory 21 may be used to store instructions, programs, codes, code sets or instruction sets. The memory 21 may include a program storage area and a data storage area, wherein the program storage area may store instructions for implementing an operating system, instructions for implementing at least one function (such as a touch function, a sound playback function, an image playback function, etc.), instructions for implementing the above-mentioned various method embodiments, etc. The data storage area may also store data (such as audio and video data, chat record data) created by the electronic device during use.

[0196] The processor 22 may include one or more processing cores. The processor 22 may use various interfaces and lines to connect various parts of the entire electronic device, and may execute various functions of the electronic device and process data by running or executing instructions, programs, code sets or instruction sets stored in the memory 21, and calling data stored in the memory 21. Optionally, the processor 22 may be implemented in at least one hardware form of digital signal processing (DSP), field programmable gate array (FPGA), and programmable logic array (PLA). The processor 22 may integrate one or a combination of a central processing unit (CPU), a graphics processing unit (GPU), and a modem. Among them, the CPU mainly processes the operating system, user interface, and application programs; the GPU is responsible for rendering and drawing display content; and the modem is used to process wireless communications. It is understandable that the above-mentioned modem may not be integrated into the processor 22, but may be implemented separately through a communication chip.

[0197] When the program or instruction stored in the memory 21 is executed, the processor 22 can be used to perform various operations in the above method embodiments to implement the steps of the tool state detection method of any technical solution in this embodiment. The specific implementation of these operations can be referred to the previous method embodiments, and will not be repeated here.

[0198] The fourth aspect of this embodiment provides a detailed description of the readable storage medium.

[0199] The readable storage medium of this embodiment stores a program or instruction on the readable storage medium, and when the program or instruction is executed by the processor, the steps of the tool state detection method of any technical solution in this embodiment are implemented. The implementation of the above operations can refer to the previous method embodiments, and will not be repeated here.

[0200] The readable storage medium may be an electronic memory such as a flash memory, an electrically erasable programmable read only memory (EEPROM), an erasable programmable read only memory (EPROM), a hard disk or a ROM. Alternatively, the readable storage medium may include a non-transitory computer-readable storage medium. The computer-readable storage medium has storage space for program codes for executing any method steps in the above method. These computer program codes may be read from or written to one or more computer program products. The computer program codes may be compressed, for example, in an appropriate form.

[0201] It should be noted that, in this article, the terms "comprise", "include" or any other variant thereof are intended to cover non-exclusive inclusion, so that the process, method, article or device including a series of elements includes not only those elements, but also includes other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprise one..." do not exclude the presence of other identical elements in the process, method, article or device including the element. In addition, it should be pointed out that the scope of the methods and devices in the embodiments of the present application is not limited to performing functions in the order shown or discussed, and may also include performing functions in a substantially simultaneous manner or in reverse order according to the functions involved, for example, the described method may be performed in an order different from that described, and various steps may also be added, omitted, or combined. In addition, the features described with reference to certain examples may be combined in other examples.

[0202] The above description is only a specific implementation mode of the present invention, but the protection scope of the present invention is not limited thereto. Any technician familiar with the technical field can easily think of changes or substitutions within the technical scope disclosed by the present invention, which should be covered by the protection scope of the present invention.

Claims

1. A tool state detection method, characterized in that: The steps include: During the operation of the machine tool, image data of the tool and the processing area are obtained, point cloud data of the processed workpiece is obtained, and / or point cloud data of the tool is obtained; Preprocessing the image data to obtain denoised and enhanced image data; Based on the denoised and enhanced image data, the characteristic information of the tool is obtained through an image processing algorithm, and the defect state of the tool is determined based on the characteristic information of the tool; Determine a machining error of the machined workpiece based on point cloud data of the machined workpiece and a 3D model of the machined workpiece; Based on the point cloud data of the tool and the workpiece, the machining program is preloaded to determine the interference status between the tool and the workpiece; Generate an alarm signal or a control command based on a defect state of the tool, a machining error of the workpiece, and / or an interference state between the tool and the workpiece; in, When the image data is preprocessed, at least the following steps are included: Extract image edge: Based on the image data, convolution is performed with a 3*3 horizontal edge detection operator and a 3*3 vertical edge detection operator to obtain the image edge. The calculation formula is as follows: ， ， Among them, f(x,y) is the original two-dimensional image matrix at the (x,y) coordinate point, each element contains 3 values, namely RGB red, green and blue brightness values, x and y are coordinates, g x (x, y) is the horizontal edge of the image extracted at the (x, y) coordinate point, g y (x,y) is the vertical edge of the image extracted at the (x,y) coordinate point; Calculate the brightness map of the image edge and analyze the connected domain: Calculate the brightness map of the image edge based on the extracted horizontal edge and vertical edge of the image, and extract the edge area with brightness greater than 30. The calculation formula is as follows: ， ， Among them, E(x,y) is the brightness map of the edge of the image at the (x,y) coordinate point, and ER(x,y) is the edge area at the (x,y) coordinate point; Based on the brightness map of the image edge, the connected region analysis algorithm is used to obtain the connected region labels of the image edge contour. The connected region analysis algorithm is expressed as follows: ， Among them, P(x,y) is the label of the contour connected area at the (x,y) coordinate point; Filter contours based on contour roundness: Calculate the contour roundness of the connected area and calculate the contour roundness using the following formula: ， ， ， ， ， ， ， in, , are the coordinates of the center point of the region, is the contour area, which refers to the number of contour pixels. is the average distance from the pixel point on the contour to the center, is the deviation between the distance from the contour pixel to the center and the average distance; is the contour roundness, is the pixel point belonging to the connected area i; The part with contour roundness greater than the preset value is used as the background image. The expression of the background image is: ， Among them, B(x,y) is the background image at the coordinate point (x,y); Image extraction: Based on the filtered background image, remove the image of water mist, water droplets, dust and waste residue Expressed as follows: ， Among them, S(x,y) is the image at the (x,y) coordinate point with water mist, water droplets, dust and waste residue removed; Image difference and fusion: Subtract the images obtained after the sequence matting to get the difference image, and perform image fusion to get the denoised image. The image difference and fusion are calculated using the following formula: ， Among them, M is the fused image, N is the number of images, is the kth image, is the fusion factor, =1 / N; Image contrast stretching: Based on the pixel values ​​of the fused image, each pixel value is linearly transformed to stretch the contrast, and the pixel value is linearly transformed using the following formula: ， Where I(x,y) is the pixel value of the xth row and yth column on the stretched image, M(x,y) is the pixel value of the xth row and yth column on the fused image, Mmin is the minimum pixel value of the fused image, and Mmax is the maximum pixel value of the fused image; Based on the denoised and enhanced image data, the characteristic information of the tool is obtained by an image processing algorithm, and the defect state of the tool is judged based on the characteristic information of the tool, which at least includes the following steps: Based on the brightness of the edge of the image, the distance between the contour point of the tool and the normal contour line of the tool is calculated, and the tool defect is detected based on the difference between the distance and a set threshold value; Based on the point cloud data of the tool and the point cloud data of the workpiece, the machining program is preloaded and the interference state between the tool and the workpiece is determined, which at least includes the following steps: Construct tool point cloud: construct tool point cloud based on the acquired point cloud data of the tool; construct machining workpiece point cloud: construct machining workpiece point cloud based on the acquired point cloud data of the machining workpiece; Determine the interference status between the tool and the workpiece: preload the machining program, calculate the point cloud intersection of the tool point cloud and the workpiece point cloud. When the point cloud intersection is empty, there is no interference between the tool and the workpiece. When the point cloud intersection is not empty, there is interference between the tool and the workpiece.

2. The tool state detection method according to claim 1, characterized in that: The image data is preprocessed, including at least eliminating one or more of water mist information, water drop information, dust information and waste residue information in the image data.

3. The tool state detection method according to claim 2, characterized in that: The distance between the actual contour of the tool and the normal contour of the tool is calculated as follows: ， Among them, (x, y) is the coordinate of the actual contour point of the tool, (x′, y′) is the coordinate of the normal contour point of the tool, is the actual contour label value of the tool, where is a conditional term, != means not equal, 0 is a point in a non-contour position, is the normal contour label value of the tool. When (x, y) is the non-contour point position of the tool, set , min means the minimum value, To traverse all normal contour points (x′, y′) of the tool, find the minimum distance between the actual contour line of the tool and the normal contour line of the tool, When the minimum distance between the actual contour line of the tool and the normal contour line of the tool is greater than the set threshold, it is judged that the tool has a defect at (x, y); when the minimum distance between the actual contour line of the tool and the normal contour line of the tool is less than or equal to the set threshold, it is judged that the tool does not have a defect at (x, y).

4. The tool state detection method according to claim 3, characterized in that: Determining a machining error of a machined workpiece based on point cloud data of the machined workpiece and a 3D model of the machined workpiece includes at least the following steps: Point cloud registration: Use the iterative closest point algorithm to register the collected 3D point cloud data with the 3D model data of the processed workpiece; Machining error analysis: Use one or more methods of distance comparison, surface fitting, feature extraction and comparison, topological relationship comparison and statistical analysis to determine the error between the machined workpiece and the 3D model of the machined workpiece.

5. A tool state detection device, characterized in that: Used to implement the tool state detection method according to any one of claims 1 to 4, the tool state detection device comprises: Acquisition module: used to obtain image data of the tool and the processing area, obtain point cloud data of the processed workpiece and / or obtain point cloud data of the tool during the operation of the machine tool; Preprocessing module: used to preprocess the image data and obtain denoised and enhanced image data; The first processing module is used to obtain the characteristic information of the tool through the image processing algorithm based on the denoised and enhanced image data, and to judge the defect state of the tool based on the characteristic information of the tool; A second processing module: used for determining a machining error of the machined workpiece based on the point cloud data of the machined workpiece and the 3D model of the machined workpiece; The third processing module is used to preload the processing program based on the point cloud data of the tool and the point cloud data of the processed workpiece, and to determine the interference state between the tool and the processed workpiece; Output module: used to generate an alarm signal or a control instruction based on the defect state of the tool, the machining error of the machined workpiece and / or the interference state between the tool and the machined workpiece.

6. An electronic device, characterized in that: The method comprises a memory and a processor, wherein the memory stores a program or instruction that can be run on the processor, and when the program or instruction is executed by the processor, the steps of the tool state detection method according to any one of claims 1 to 4 are implemented.

7. A readable storage medium, characterized in that: The readable storage medium stores a program or instruction, and when the program or instruction is executed by a processor, the steps of the tool state detection method according to any one of claims 1 to 4 are implemented.

Images