A machine vision-based numerical control machine tool operation fault detection system and method thereof

Abstract

The application discloses a kind of based on machine vision's numerical control machine tool operation fault detection system and method thereof, comprising: image acquisition system, for collecting the running state image of each part equipment in numerical control machine tool processing process;Edge computing system, for detecting whether the image collected by image acquisition system appears equipment failure condition;Fault alarm system, for the running failure information of numerical control machine tool detected by edge computing system is fed back to local area network system.Edge computing system uses target detection algorithm for fault detection, and the target detection algorithm uses improved two-stage target detection convolutional neural network.The application realizes all-weather real-time detection of numerical control machine tool operation fault by using non-invasive machine vision method, can automatically alarm for the fault occurred, with the advantages of easy implementation and high accuracy.

Description

TECHNICAL FIELD

[0001] The present application belongs to the field of industrial machine vision, and particularly relates to a numerical control machine running fault detection system and method based on machine vision. BACKGROUND

[0002] In order to further promote the rapid development of intelligent manufacturing, how to use intelligent methods for industrial detection, ensure the continuous normal operation of numerical control machine tools, and improve the production efficiency of numerical control machine tools, has gradually become an important problem in the field of intelligent manufacturing.

[0003] Real-time state monitoring and fault detection of numerical control machine tools are important problems that need to be solved in numerical control machining, especially in the production mode of "no factory workers" or "one person operating multiple machines". If unpredictable faults occur during the machining process of numerical control machine tools, the production schedule may be delayed or even serious safety accidents may occur.

[0004] Existing numerical control machine tools mainly use in-situ monitoring methods for fault detection, such as using proximity switch sensors to complete displacement control and monitoring of transmission devices.

[0005] For example, a Chinese patent document with publication number CN110614539A discloses a numerical control machine tool spindle state online real-time monitoring and analysis method, which includes the following steps: a, performing online real-time monitoring system parameter setting; b, testing and collecting machine tool spindle state information; c, a three-way vibration sensor and an inductive proximity switch directly transmit signals to a state monitoring and analysis device, a current sensor and a temperature sensor first input signals to a multi-channel signal collector, and after digital processing, the signal data is transmitted to the state monitoring and analysis device; d, through state analysis database comparison, complete spindle state monitoring data analysis. This monitoring method can real-time master the working state of the machine tool spindle and timely discover machine tool spindle faults.

[0006] A Chinese patent document with publication number CN203853858U discloses an alarm system for a numerical control lathe, which includes a man-machine operation interface and a PLC controller, the PLC controller is connected with the man-machine operation interface; the PLC controller is connected with a detection element, an alarm element and a moving part; the detection element includes a spindle end proximity switch, a middle slide plate position detection sensor, a tool holder impact force sensor, an air inlet pipe pressure sensor, a machine box internal temperature sensor and a lubricating oil level sensor; the alarm element is an audible and visual alarm; the moving part includes a slide plate movement driving motor, a spindle motor and a cooling liquid circulating pump; the spindle end proximity switch is provided with multiple, and its installation position is adjustable; the middle slide plate position detection sensor is installed on the large slide plate of the numerical control lathe. The alarm system for the numerical control lathe solves the problems of the tool and tool holder abnormally colliding or hitting the spindle and workpiece, the middle slide plate running position "out of range", difficulty in resetting, and the machine running in a sick state under the conditions of high temperature and lack of lubrication.

[0007] However, the in-situ monitoring method can achieve good detection effect under certain conditions, but its installation is complex, the detection accuracy is not high, and the numerical control machine tool needs to be invasively modified, so it is only suitable for special occasions. SUMMARY

[0008] The application provides a numerical control machine tool operation fault detection system based on machine vision, which realizes all-weather real-time detection of numerical control machine tool operation faults by using non-invasive machine vision means, can automatically alarm the occurring faults, and has the advantages of easy implementation and high accuracy.

[0009] A numerical control machine tool operation fault detection system based on machine vision, comprising:

[0010] An image acquisition system for acquiring the running state images of each part of the numerical control machine tool during processing;

[0011] An edge computing system for detecting whether a device fault occurs in the images acquired by the image acquisition system;

[0012] A fault alarm system for feeding back the operation fault information of the numerical control machine tool detected by the edge computing system to a local area network system.

[0013] Further, the image acquisition system is an embedded on-board camera arranged at the end of the industrial robot arm, the camera has an automatic focusing function, is fixed to an aluminum alloy shell by a metal connecting piece, and the main shaft is parallel to the end clamp of the robot arm.

[0014] The image acquisition system adopts an aluminum alloy shell to protect the internal camera and accessories, and an optical glass coated with an optical film is used as a perspective protection window of the camera, which is located in front of the camera lens.

[0015] Further, the edge computing system comprises a hardware module and a software module; the hardware module is placed in the robot control cabinet and comprises an industrial control computer provided with a graphic processing unit, and the computer is used to provide the calculation capacity required by the target detection algorithm; the software module is a computer software program running on the edge computing system, which is used to receive the images acquired by the image acquisition system, detect faults by using the target detection algorithm, and send the result to the fault alarm system for processing after obtaining the result.

[0016] The improved two-stage target detection convolutional neural network is used as the target detection algorithm, and the positioning and identification process of the convolutional neural network is as follows:

[0017] The preprocessed running state image is input into a backbone network to obtain a high-dimensional feature map, a region proposal network based on K-means clustering is used to estimate the position of a candidate frame, a projection head based on a large-size spatial separable convolution is used to reduce and project the high-dimensional feature map, and target region alignment is used to intercept the target region in the feature map after the reduction and projection to a specified size, and the intercepted target region is used as the input of a subsequent classification and regression subnetwork;

[0018] In one aspect, a full connection layer is used to directly regress the coordinate encoding of the target object, and then the position of the object in the actual image is calculated based on the coordinate encoding mode; in another aspect, a prototype vector corresponding to each category is used, a regression subnetwork is used to regress a category regression vector, the cosine similarity between the category regression vector and the prototype vector is calculated, a soft-max function is used to convert all similarity scores into category probabilities, and the category corresponding to the maximum probability is taken as the category predicted by the convolutional neural network.

[0019] Further, the fault alarm system comprises a local area network system and a control terminal, if a device fault occurs, the fault detection system broadcasts a fault signal to the control terminal through the local area network for processing, and if no device fault occurs, the fault detection system broadcasts a normal reply signal to the local area network.

[0020] The application also provides a machine vision-based numerical control machine tool running fault detection method, comprising the following steps:

[0021] (1) using an image acquisition system to acquire a numerical control machine tool running state image;

[0022] (2) using an image processing algorithm to preprocess the running state image;

[0023] (3) using a target detection algorithm to locate and identify the feature target of the preprocessed running state image; wherein the target detection algorithm adopts an improved two-stage target detection convolutional neural network, and the locating and identifying process of the convolutional neural network is as follows:

[0024] The preprocessed running state image is input into a backbone network to obtain a high-dimensional feature map, a region proposal network based on K-means clustering is used to estimate the position of a candidate frame, a projection head based on a large-size spatial separable convolution is used to reduce and project the high-dimensional feature map, and target region alignment is used to intercept the target region in the feature map after the reduction and projection to a specified size, and the intercepted target region is used as the input of a subsequent classification and regression subnetwork;

[0025] On the one hand, the coordinate encoding of the target object is obtained by direct regression using a fully connected layer, and then the position of the object in the actual image is calculated based on the coordinate encoding method. On the other hand, each category corresponds to a prototype vector, and the category regression vector is obtained by regression using a regression sub-network. The cosine similarity between the category regression vector and the prototype vector is calculated, and all similarity scores are converted into category probabilities using a soft maximum function. The category corresponding to the maximum probability is taken as the category predicted by the convolutional neural network.

[0026] (4) Compare the target detection results with the standard operating status of the equipment. If the results deviate from the standard operating status, a fault alarm will be triggered.

[0027] In step (1), the image acquisition system extracts image frames from the real-time video stream of the camera as CNC machine tool running status images based on a single trigger event.

[0028] In step (2), the preprocessing includes grayscale transformation of the running state image, target region cropping, and image normalization.

[0029] In step (3), for different features of the CNC machine tool, a separate convolutional neural network is trained to locate and identify the feature targets.

[0030] The improved two-stage object detection convolutional neural network uses manually collected data as input and manually labeled data as supervision during training. It constructs the training dataset using class-balanced sampling and the test dataset using sequential sampling.

[0031] The training and test datasets were collected using the following methods:

[0032] (a) Using a CNC machine tool, the feature target is sequentially controlled to move to the normal position and the fault position, and an image acquisition system is used to acquire an image containing the feature target;

[0033] (b) Sequentially change the ambient lighting conditions of the CNC machine tool, change the posture of the feature target by a small angle, and change the position of the feature target by a small amount, and use the method in step (a) to acquire an image containing the feature target;

[0034] (c) Combining the methods of steps (a) and (b), collect more than 1,000 image samples as the total dataset.

[0035] In step (4), the standard running state is the average state of the numerical control machine tool without fault, which is determined by the following experimental method: the numerical control machine tool is controlled to recover from the fault state to the normal running state, at this time the characteristic target is reset to the normal position, more than 50 experiments are repeated and the normal position of the characteristic target is recorded, and the average value of all experimental record positions is taken as the standard running state;

[0036] The deviation of the standard running state is determined by the following experimental method: the numerical control machine tool is controlled to run the characteristic target from the normal state to the fault state, more than 50 experiments are repeated and the fault state position of the characteristic target is recorded, the average value of all experimental record positions is taken as the standard fault state, and the average value of the standard fault state position and the standard running state position is taken as the critical position of the deviation of the standard running state.

[0037] Compared with the prior art, the present application has the following beneficial effects:

[0038] 1, the technical scheme of the present application adopts machine vision method, compared with other fault detection methods, has the advantages of non-invasive, simple device and easy to install, high detection accuracy.

[0039] 2, the fault detection method of the present application uses the improved two-stage target detection convolutional neural network, compared with other image processing algorithms, has the advantages of high accuracy and strong robustness.

[0040] 3, the technical scheme of the present application adopts edge computing information processing scheme, uses high-performance graphics processing unit as core computing equipment, solves the functional limitation caused by the lack of computing ability of numerical control machine tool, and the edge computing scheme can be extended to the calculation of other functional modules of numerical control machine tool, effectively controls the total system investment cost. BRIEF DESCRIPTION OF DRAWINGS

[0041] Figure 1 It is a schematic diagram of a numerical control machine tool running fault detection system based on machine vision of the present application;

[0042] Figure 2 It is a flow chart of a numerical control machine tool running fault detection method based on machine vision of the present application;

[0043] Figure 3 It is a schematic diagram of tool head cascade detection in an embodiment of the present application;

[0044] Figure 4 It is a schematic diagram of spindle fault detection in an embodiment of the present application;

[0045] Figure 5 It is a schematic diagram of the two-stage target detection convolutional neural network model used in the present application;

[0046] Figure 6The schematic diagram of data collection and labeling of the present application;

[0047] Figure 7 The effect diagram of real-time detection of the embodiment of the present application on a numerical control machine tool. DETAILED DESCRIPTION

[0048] The present application will be further described in detail below in conjunction with the drawings and embodiments, it should be pointed out that the following described embodiments are intended to facilitate the understanding of the present application, and do not have any limiting effect on it.

[0049] As Figure 1 shown, a numerical control machine tool operation fault detection system based on machine vision includes an image acquisition system for acquiring the running state of each part of the numerical control machine tool during processing; an edge computing system for detecting whether a device failure occurs in the collected image; a fault alarm system for feeding back the operation fault information of the numerical control machine tool to the local area network system.

[0050] Specifically, the image acquisition system includes a board camera disposed at the end of the industrial robot arm, wherein the board camera uses a CMOS sensor, carries a camera module with an automatic focusing function, has a field of view angle greater than 90°, and can realize clear collection of images within the width and depth range of the numerical control machine tool.

[0051] Among them, the board camera is fixed to an aluminum alloy shell using a sandblasting anodizing process using a metal connecting piece, which is used to protect the camera and its internal electrical parts; the camera lens is placed in front of a circular quartz optical glass as a protective window, and the protective window is coated with a waterproof, oil-proof and corrosion-resistant optical film to prevent cutting fluid from adhering to the lens and affecting the camera imaging quality.

[0052] The edge computing system includes hardware modules and software modules, and the hardware modules are located in the control cabinet on the side of the robot, wherein an industrial control computer containing an NVIDIA graphics processing unit is contained, and the graphics processing unit is used to provide the computing power required by the machine vision algorithm. The calculation includes two parts of training and reasoning, wherein the training process is based on NVIDIA CUDA parallel computing, and is characterized by requiring at least 4G of display memory; the reasoning process is based on NVIDIA TensorRT high-performance deep learning inference framework, which can provide low latency and high throughput for deep learning applications. In addition, the industrial control computer has two or more Ethernet interfaces for providing the ability to access the local area network system for inter-device communication.

[0053] The software module is a computer software program running on the edge computing system, which is written based on the computer language C++, and can run on the edge computing system all day; the software module is used for receiving images collected by the image collection system, and using a target detection algorithm for fault detection, and sending the result to the fault alarm system for processing after calculation.

[0054] The fault alarm system includes a local area network system mainly composed of a router and a switch, wherein the edge computing system and the industrial management control system are connected to the local area network system, which can provide a method for transmitting data between devices such as cameras, numerical control machine tools and edge computing systems; if the numerical control machine tool fails, the image collection system collects the image of the failure, the edge computing system calculates the result and forwards it to the fault alarm system, and the fault alarm system broadcasts the fault signal through the local area network, and each device connected to the local area network system acquires device fault information as needed to respond to the fault.

[0055] More specifically, the fault alarm system includes a control terminal with a user interaction interface, which stops the numerical control machine tool in an emergency when the machine tool fails, and provides the authority for the staff to operate the robot system and the numerical control system of the machine tool to restore it to a normal operating state.

[0056] As shown in Figure 2 A numerical control machine tool running fault detection method based on machine vision, comprising the following steps:

[0057] S1, using an image collection system to collect numerical control machine tool running state images.

[0058] S2, using an image processing algorithm to pre-process the running state images, and the pre-processing result is used as the input of S3.

[0059] S3, using a target detection algorithm to locate and identify the feature targets in the numerical control machine tool.

[0060] S4, comparing the target detection result with the standard running state of the equipment, and if it deviates from the standard running state, a fault alarm is given.

[0061] Specifically, the industrial management control system sends a fault detection signal to the edge computing system, and the image collection system extracts image frames from the camera real-time video stream to determine the running state of the numerical control machine tool as a trigger event.

[0062] The original image obtained by the image acquisition system needs to be preprocessed to highlight the feature target, and the image preprocessing algorithm uses a high-performance graphics processing unit as a processor; for the image preprocessing method, the original image is subjected to automatic white balance through a gray world transformation algorithm, then the target region is cropped according to the region range of the specific detection task, and finally the color three-channel image is subjected to ImageNet normalization processing and used as the input of the convolutional neural network.

[0063] Specifically, a two-stage target detection convolutional neural network algorithm is used to locate and identify the features in the numerical control machine tool. For different features of the numerical control machine tool, a separate convolutional neural network is trained to locate and identify the feature target.

[0064] The numerical control machine tool uses a cascade detection method for feature target recognition, as shown in Figure 3 , which is a specific embodiment for detecting the failure state of the tool head. The tool head and the spindle have a containment relationship, and the spindle feature is more prominent. A separate convolutional neural network algorithm is trained to first locate the position of the spindle feature, which is more prominent. Based on the coarse positioning of the spindle position, the tool head region is located, and then the convolutional neural network algorithm is used to precisely locate the tool head position and identify the failure state of the tool head.

[0065] Further, the machine vision-based numerical control machine tool failure detection method is also based on the comparison of the standard running state, as shown in Figure 4 , which is a specific embodiment for detecting the failure state of the spindle. When the spindle fails, as shown in Figure 4 (a), the spindle cannot be withdrawn to both sides of the numerical control machine tool, and the mechanical arm operating the numerical control machine tool will interfere with the spindle; when the spindle does not fail, as shown in Figure 4 (b), the spindle normally withdraws to both ends of the numerical control machine tool; the main basis for determining whether the spindle has failed is the position of the spindle; in the present application, the two-stage target detection convolutional neural network algorithm is used to locate the precise position of the spindle, and the detection result is compared with the standard running position to determine whether the spindle has failed. If it is detected that the spindle has failed, a fault alarm is given through the local area network system.

[0066] The standard running state is the average state of the numerical control machine tool without failure, which is determined by the following experimental method: control the numerical control machine tool to restore it from the failure state to the normal running state, at which time the feature target is reset to the normal position, repeat the experiment more than 50 times and record the normal position of the feature target, and take the average of all experimental records as the standard running state.

[0067] The deviation standard operating state is determined by the following experimental method: controlling the numerical control machine tool to make the characteristic target from the normal state to the fault state, repeating more than 50 experiments and recording the fault state position of the characteristic target, taking the average value of all experimental record positions as the standard fault state, and taking the average value of the standard fault state position and the standard operating state position as the critical position of the deviation standard operating state.

[0068] The convolutional neural network adopts the following process for data collection: manually controlling the numerical control machine tool and the image acquisition system to collect samples of the characteristic target at different positions to form a training data set and a test data set, and changing the environmental conditions of image acquisition to collect multiple groups of characteristic target images under different environments. Finally, a rectangular box is used as a label to label the targets in the training set and the test set.

[0069] As shown in Figure 5 , the improved two-stage target detection convolutional neural network algorithm inputs the preprocessed image into a convolutional neural network with ResNet50 as the backbone to obtain a high-dimensional feature map, and uses a region proposal network to estimate the position of the candidate box. As an improvement, to avoid missing detection and false detection of targets in complex scenes, the anchor box scale of the region proposal network is estimated using the K-means clustering algorithm. Specifically, the length and width dimensions of each label in the training data set can be represented as a two-dimensional vector v i =(w i ,h i ), i = 1, 2, …, N, where N is the total number of labels in the training data set, w i is the width of the label, and h i is the height of the label. All label vectors v i , i = 1, 2, …, N are input to the K-means clustering, where K is the number of target categories to be detected. Finally, the centers of the K-means clustering are used as the preset anchor boxes.

[0070] As an improvement, to reduce the computational load of the fully connected layer, a spatial separable convolution with a kernel size of 15 is used as a projection head to project the feature map to a lower dimension, reducing the feature map from 2048 channels to 490 channels, and increasing the inference speed of the fully connected layer by 3 times. The target region alignment is used to crop the target region in the feature map to a specified size, and the cropped target region is expanded and fed into the fully connected layer to obtain the original features of the target object.

[0071] On the one hand, the original features are directly subjected to fully connected regression to obtain the coordinate encoding of the target object, and finally the coordinate position of the object in the actual image is calculated based on the coordinate encoding; on the other hand, as an improvement, to ensure the scalability of the convolutional neural network for categories, the original features are subjected to fully connected regression to obtain a category regression vector v cls of the target object, and the category regression vector v cls and the prototype vector pj Calculate the cosine distance similarity between j = 1, 2, ..., K. Use the softmax function to convert all similarities into class probabilities P(j) = softmax. j {γs(v cls ,p j )}, where γ is the category-related hyperparameter, and finally, the category j corresponding to the maximum probability is taken as arg max. j P(j) represents the category predicted by the convolutional neural network.

[0072] Taking spindle fault detection as a specific example, such as Figure 6 As shown in the figure. Operate the CNC machine tool spindle to the normal position shown by the solid line, and use the image acquisition system to acquire images of the CNC machine tool including the spindle position. Slightly change the lighting conditions of the CNC machine tool and the posture of the image acquisition system to acquire images of the CNC machine tool under multiple environmental conditions. Operate the CNC machine tool spindle again to the fault position shown by the dashed line, changing the environmental conditions to acquire images of the CNC machine tool. Over several days of continuous operation of the CNC machine tool, change the environmental conditions and spindle position using the above method, acquiring more than 1000 image samples as the total dataset.

[0073] The training process for convolutional neural networks (CNNs) is as follows: Images in the training dataset are preprocessed; batch samples are generated from the training dataset using class-balanced sampling; these batch samples are then fed into the CNN for forward propagation to obtain predicted values; the predicted values ​​and labels are used to calculate the loss function; the loss values ​​are then backpropagated to obtain the parameter gradients of the CNN; and a first-order gradient optimization algorithm is used to update the CNN parameters. In each training round, samples in the training dataset are sampled without repetition until all samples have been sampled once. Training a single CNN requires a total of 100 rounds.

[0074] The following testing procedure is used for convolutional neural networks: freeze the parameters and batch normalization layers in the convolutional neural network, preprocess the samples in the test dataset, input them into the convolutional neural network for forward propagation to obtain the predicted values, and use the predicted values ​​and labels to calculate the mean average precision (mAP) for all classes.

[0075] Currently, the technology described in this invention has been successfully applied in Jiangxi Changhe Aviation Industry Co., Ltd. Figure 7 These are a set of effect diagrams showing the real-time testing of this invention on the Shenyang Zhongjie SUC1200 machine tool. Figure 7Fig. (a) and (b) are effect pictures of the detection of the machine tool door, wherein the machine tool door is normally opened and no fault occurs; Fig. (c) is an effect picture of the detection of the spindle of the machine tool, wherein the spindle can normally retreat and no fault occurs; and Fig. (d) is an effect picture of the detection of the workpiece of the machine tool, wherein there is a mechanical part finished on the workpiece table of the machine tool, no abnormality and fault occur, and the mechanical arm can normally clamp the workpiece.

[0076] The above-described embodiments have described the technical solutions and beneficial effects of the present application in detail, and it should be understood that the above-described is only a specific embodiment of the present application and is not used to limit the present application, and any modification, supplement and equivalent replacement made within the principle range of the present application should be included in the protection range of the present application.

Claims

1. A machine vision-based CNC machine tool operation fault detection system, characterized in that, include: Image acquisition system is used to acquire images of the operating status of various parts of the equipment during CNC machine tool processing; Edge computing systems are used to detect whether equipment malfunctions occur in images acquired by image acquisition systems. The fault alarm system is used to feed back the operating fault information of the CNC machine tool detected by the edge computing system to the local area network system; The edge computing system includes a hardware module and a software module. The hardware module is placed in the robot control cabinet and contains an industrial control computer equipped with a graphics processing unit. The computer is used to provide the computing power required for the target detection algorithm. The software module is a computer software program that runs on the edge computing system. It is used to receive images acquired by the image acquisition system, use the target detection algorithm to perform fault detection, and send the results to the fault alarm system for processing. The object detection algorithm employs an improved two-stage object detection convolutional neural network, which performs the localization and recognition process as follows: The preprocessed running state image is input into the backbone network to obtain a high-dimensional feature map. The location of the candidate box is estimated using a region candidate network based on K-means clustering. The high-dimensional feature map is dimensionality reduced by a projection head based on large-size spatially separable convolution. The target region is cropped to a specified size in the dimensionality-reduced feature map using target region alignment. The cropped target region is used as the input for subsequent classification and regression sub-networks. On the one hand, the coordinate encoding of the target object is obtained by direct regression using a fully connected layer, and then the position of the object in the actual image is calculated based on the coordinate encoding method. On the other hand, each category corresponds to a prototype vector, and the category regression vector is obtained by regression using a regression sub-network. The cosine similarity between the category regression vector and the prototype vector is calculated, and all similarity scores are converted into category probabilities using a soft maximum function. The category corresponding to the maximum probability is taken as the category predicted by the convolutional neural network.

2. The CNC machine tool operation fault detection system based on machine vision according to claim 1, characterized in that, The fault alarm system includes a local area network (LAN) system and a control terminal. If a device fault occurs, the fault detection system will broadcast the fault signal to the control terminal for processing via the LAN; if no device fault occurs, the fault detection system will broadcast a normal response signal to the LAN.

3. A method for detecting operational faults in CNC machine tools based on machine vision, characterized in that, Includes the following steps: (1) Use an image acquisition system to acquire images of the CNC machine tool's operating status; (2) Use image processing algorithms to preprocess the running status image; (3) The target detection algorithm is used to locate and identify the feature targets in the preprocessed running state image; the target detection algorithm adopts an improved two-stage target detection convolutional neural network, and the localization and recognition process of the convolutional neural network is as follows: The preprocessed running state image is input into the backbone network to obtain a high-dimensional feature map. The location of the candidate box is estimated using a region candidate network based on K-means clustering. The high-dimensional feature map is dimensionality reduced by a projection head based on large-size spatially separable convolution. The target region is cropped to a specified size in the dimensionality-reduced feature map using target region alignment. The cropped target region is used as the input for subsequent classification and regression sub-networks. On the one hand, the coordinate encoding of the target object is obtained by direct regression using a fully connected layer, and then the position of the object in the actual image is calculated based on the coordinate encoding method; on the other hand, each category corresponds to a prototype vector, and the category regression vector is obtained by regression using a regression sub-network. The cosine similarity between the category regression vector and the prototype vector is calculated, and all similarity scores are converted into category probabilities using a soft maximum function. The category corresponding to the maximum probability is taken as the category predicted by the convolutional neural network. (4) Compare the target detection results with the standard operating status of the equipment. If the results deviate from the standard operating status, a fault alarm will be triggered.

4. The method for detecting operational faults in CNC machine tools based on machine vision according to claim 3, characterized in that, In step (1), the image acquisition system extracts image frames from the real-time video stream of the camera as CNC machine tool running status images based on a single trigger event.

5. The method for detecting operational faults in CNC machine tools based on machine vision according to claim 3, characterized in that, In step (2), the preprocessing includes grayscale transformation of the running state image, target region cropping, and image normalization.

6. The method for detecting operational faults in CNC machine tools based on machine vision according to claim 3, characterized in that, In step (3), for different features of the CNC machine tool, a separate convolutional neural network is trained to locate and identify the feature targets.

7. The method for detecting operational faults in CNC machine tools based on machine vision according to claim 3, characterized in that, The improved two-stage object detection convolutional neural network uses manually collected data as input and manually labeled data as supervision during training. It constructs the training dataset using class-balanced sampling and the test dataset using sequential sampling.

8. The method for detecting operational faults in CNC machine tools based on machine vision according to claim 7, characterized in that, The training and test datasets were collected using the following methods: (a) Using a CNC machine tool, the feature target is sequentially controlled to move to the normal position and the fault position, and an image acquisition system is used to acquire an image containing the feature target; (b) Sequentially change the ambient lighting conditions of the CNC machine tool, change the posture of the feature target by a small angle, and change the position of the feature target by a small amount, and use the method in step (a) to acquire an image containing the feature target; (c) Combining the methods of steps (a) and (b), collect more than 1,000 image samples as the total dataset.

9. The method for detecting operational faults in CNC machine tools based on machine vision according to claim 3, characterized in that, In step (4), the standard operating state is the average state in which the CNC machine tool does not malfunction. It is determined by the following experimental method: control the CNC machine tool to restore it from the fault state to the normal operating state. At this time, the feature target is reset to the normal position. Repeat the experiment more than 50 times and record the normal position of the feature target. Take the average value of all experimental recorded positions as the standard operating state. The deviation from the standard operating state is determined by the following experimental method: control the CNC machine tool to make the feature target run from the normal state to the fault state, repeat the experiment more than 50 times and record the fault state position of the feature target, take the average of all experimental recorded positions as the standard fault state, and take the average of the standard fault state position and the standard operating state position as the critical position of deviation from the standard operating state.

Images