A protection system and protection method for a plate forming machine tool based on machine vision

Abstract

The application discloses a kind of based on machine vision's plate forming processing machine tool's protection system, it includes image acquisition component, mounting bracket component, image processing module, relay control module;Mounting bracket component is fixedly connected with the side wallboard of machine tool, image acquisition component is installed on bracket component, image processing module is electrically connected with image acquisition component, one end of relay control module is electrically connected with image processing module, other end is electrically connected with the electric control box of machine tool.The protection system of the application realizes high-precision all-around monitoring to its working area when plate is processed by machine tool, can respond quickly when foreign matter appears in working area, improve the security in the machining process of machine tool.Simultaneously, the application also provides a kind of based on machine vision's plate forming processing machine tool's protection method.

Description

TECHNICAL FIELD

[0001] The present application belongs to the field of industrial automation and numerical control machine tool processing, and particularly relates to a protection system and method for a plate forming machine tool based on machine vision. BACKGROUND

[0002] With the continuous development of industrial manufacturing level, safety problems in industrial production are increasingly valued by people. Plate forming machine tools, as important equipment in the field of mechanical processing, are widely used in industrial production. Plate shearing machines and bending machines are typical plate forming processing equipment, and their working process is as follows: the worker places the plate to be processed between the upper die and the lower die, and through the reciprocating movement of the upper die relative to the lower die, the plate is processed into the required shape. During the operation process, the worker may stretch his body parts or tools into the upper and lower dies, which poses a risk of personal injury and machine damage. In order to avoid human-machine failure during production and protect the safety of workers and equipment, a safety protection system is essential.

[0003] The existing safety protection schemes mostly use infrared devices installed at different positions of the machine tool to achieve protection. There are two common schemes: one is Chinese patent CN216802715U (publication date 2022.06.24), which discloses a protection device for a numerical control plate shearing machine. It forms a plane light path in the front end area of the upper and lower dies of the machine tool by installing an infrared grating at the front end of the machine tool workbench. If an object blocks the light path in this area, the infrared device triggers the machine tool to stop working. Another Chinese patent is CN219684111U (publication date 2023.09.15), which discloses a new plate shearing machine with a protection structure. It installs an infrared device on the machine tool frame, which moves with the upper die when the machine tool is working, forming a plane light path in the area below the upper die. When an object enters the area below the upper die and blocks the light path, the machine tool stops working. The infrared sensors in the above two patents act as point detectors, which have the following shortcomings in actual application: 1. There is a gap between the different transmitting and receiving ends of the infrared sensor, so when the height of a small-size object or obstacle does not exceed the gap, the object cannot be detected even if it exists, especially when the foreign object is placed or falls into the groove of the lower die; 2. The vibration during the operation of the plate shearing machine may interfere with the infrared device, significantly increasing the false alarm rate and affecting the stability of the device, which is time-consuming and laborious to debug and maintain. SUMMARY

[0004] The present application provides a protection system for a plate forming machine tool based on machine vision, which can achieve high-precision and all-around monitoring of the working area of the machine tool during plate processing, and quickly respond when foreign objects appear in the working area, thereby improving the safety of the machine tool during processing. Meanwhile, the present application also provides a protection method for a plate forming machine tool based on machine vision.

[0005] The application discloses a protection system of a plate forming machining tool based on machine vision, which comprises an image acquisition assembly, a mounting bracket assembly, an image processing module and a relay control module.

[0006] Further, the image acquisition assembly comprises a plurality of image acquisition units, each of which comprises a camera and an image transmission cable, and the mounting bracket assembly comprises left and right L-shaped side brackets, a horizontal rod and left and right L-shaped top brackets; one end of the L-shaped side bracket is fixedly connected to the outer side of the side wall plate of the tool, and the other end is connected to the camera; the L-shaped top bracket is fixedly connected to the upper surface of the side wall plate of the tool, and the two ends of the horizontal rod are fixedly connected to the left and right L-shaped top brackets; a plurality of cameras are installed on the horizontal rod at equal intervals; and the camera is connected to the image processing module through the image transmission cable.

[0007] Further, each image acquisition unit further comprises a positioning plate and a positioning lock, the positioning plate is connected to the rear of the camera, the positioning lock is arranged on the positioning plate, and the positioning lock is connected to the L-shaped side bracket or the horizontal rod.

[0008] Further, the image processing module comprises a camera serial interface, an embedded operation unit and input and output pins, and the relay control module comprises a Dupont wire, relay pins, a relay internal circuit, relay contacts and a control cable; the camera serial interface is connected to the image transmission cable, the two ends of the Dupont wire are connected to the input and output pins and the relay pins respectively, and the two ends of the control cable are connected to the relay contacts and the control switch of the tool respectively.

[0009] The application discloses a protection method of a plate forming machining tool based on machine vision, which comprises the following steps: ① a plurality of image acquisition units acquire image data of a working area in real time; ② an image processing module processes the image through an optimized deep learning model in the module, judges whether there is a foreign matter or not, outputs a high level if there is a foreign matter, and outputs a low level if there is no foreign matter; ③ a relay control module receives the high and low level signals, controls the shutdown or normal work of the tool, and if the signal is a high level, the relay coil is powered on, the normally closed contact is disconnected, the main circuit of the tool is disconnected and stopped working, the foreign matter is removed manually, the tool is reset, the protection system is initialized, the output signal becomes an initial low level, and the system judges the next frame of image of the working area; if the signal is a low level, the relay coil is powered off, the normally closed contact is closed, the main circuit of the tool is connected and normally works.

[0010] Further, the determination method of the number and specific position of the image acquisition unit in step 1 is as follows: I, determining the detection range according to the dangerous area during the machining of the machine tool: for the machine tool with an upper tool length L and a lower tool thickness W, the dangerous area is divided into the following parts: an upper tool edge front end area with a length of L and a width of 200 mm; an upper tool edge rear end area with a length of L and a width of 200 mm; two left and right side areas with a length of 50 mm and a width of W+400 mm; a lower tool area with a length of L and a width of W; the above areas form a rectangular area with a length of L+100 mm and a width of W+400 mm; II, determining the number of cameras with adjustable angles: the length of the detection area covered by a single camera is D, and the width is W+400 mm, and the number of cameras is the integer part of (L+100) / D; III, arranging the cameras horizontally above the machine tool and on the two outer sides of the machine tool, so that the field of view of the cameras covers the rectangular area with a length of L+100 mm and a width of W+400 mm.

[0011] Further, in step 1: the camera captures a video image, and uses a high-speed interface and a suitable data transmission protocol to transmit the image to the image processing module; in order to ensure that each camera acquires images at the same time point, the system uses a software clock synchronization mechanism and sets a unified frame rate, so that each camera acquires images at the same speed, avoiding inconsistent data or information loss; in the image transmission, a suitable bandwidth is set according to the number of cameras, so that the data of the cameras can be transmitted smoothly; in order to cope with the synchronous reception of multiple data images, the embedded device uses a multi-thread processing technology, that is, each thread receives the data stream from a specific camera, thereby improving the efficiency of overall data reception.

[0012] Further, the processing flow of the optimized deep learning model in step ② is as follows: I. Preprocessing: under the premise of maintaining the aspect ratio of the image, the image is scaled to a specific size, the image pixel value is normalized from 0-255 to the range of 0-1, the image is randomly transformed, and the data diversity is increased through rotation, translation, scaling, flipping and other data enhancement methods to improve the generalization ability of the model; II. Feature extraction: using convolution kernels to slide in the image, the basic features in the image are extracted through weighted summation, including low-level features such as image edges and corner points; then the features are processed using the activation function (RELU function) to change negative values to 0 and keep positive values unchanged, so that important detail features in the image are more obvious; finally, the pooling operation is performed to simplify the image, the maximum value and the average value of the local region are extracted to reduce the data amount and retain the main features; III. Object detection: the processed image is divided into multiple grids, and each grid is responsible for detecting the target within its range. In each grid, the model generates multiple bounding boxes surrounding the object based on the size and shape of the target object in the training data set, calculates the confidence of each bounding box and the probability of the belonging class, and generates multiple possible prediction results; IV. Generating prediction results: the image features extracted by the feature extraction are flattened into a one-dimensional vector in order, and input into the fully connected layer. The features are further processed through weighted summation and activation function to extract more complex feature representations such as shape and color combination features; the output layer receives the feature representation of the fully connected layer, uses the classification task activation function to convert the feature value into a probability distribution, and selects the class with the highest probability as the final prediction result; V. Post-processing: adjust the bounding box to match the actual target, remove the overlapping prediction boxes through non-maximum suppression (NMS), and only keep the box with the highest confidence; sum the probabilities defined as foreign object class to get the total probability of foreign object, set a threshold, if the probability is greater than the threshold, it is judged that there is a foreign object, if it is less than the threshold, it is judged that there is no foreign object.

[0013] Further, the method for optimizing the deep learning model in step ② is as follows: import the divided data set into the model for training through the deep learning framework to obtain a preliminary model; fuse the compression and excitation (SE) attention mechanism in the backbone network to obtain a dangerous area focusing module; add a multi-size target capture unit after the dangerous area focusing module to obtain a dangerous object recognition module; the dangerous object recognition module classifies and recognizes the captured target area to determine whether the target belongs to a dangerous object and outputs the class label and confidence; based on the output of the dangerous object recognition module, the dangerous assessment output layer further analyzes the recognition result to evaluate the risk level of the detected target in the current scene, and finally outputs a control signal to guide the operation logic of the machine tool.

[0014] Furthermore, the preliminary model construction method for the deep learning model in step ② is as follows: Ⅰ. When the machine tool is not working, simulate the occurrence of sheet metal and various foreign objects in different scenarios in the dangerous area, and collect these images to create a dataset; Ⅱ. After completing the collection of dataset samples, use image annotation tools to classify and label the processed sheet metal and various foreign objects, and randomly divide them into training set, validation set and test set in a ratio of 7:1:2; Ⅲ. Import the divided dataset into the built-in model of the image processing module through the deep learning framework for training, and obtain the preliminary model of the deep learning model.

[0015] The advantages of the protection system of this invention are: 1. By setting the dangerous area range (including the front and rear positions of the upper and lower die-cutting molds, as well as the space between the upper and lower die-cutting molds), designing a reasonable camera arrangement scheme, and adjusting the cameras to appropriate angles, all-round monitoring of the dangerous area is achieved, improving adaptability to different machine tool equipment, avoiding blind spots, and thus improving machine tool safety; 2. By optimizing the deep learning model, adding an attention mechanism and an improved pyramid pooling module after the model backbone network, the detection speed and recognition accuracy are improved, enabling the model to intelligently analyze complex scenes and accurately detect abnormal situations where small foreign objects enter the dangerous area; 3. The relay control module can receive signals to realize the emergency stop and reset of the machine tool, achieving rapid response, improving the automation level of the machine tool, and reducing safety hazards caused by operator error. In summary, when machine tools process sheet metal, the protection system of this invention provides high-precision all-round monitoring of its working area, and can quickly respond to form an efficient protection scheme when foreign objects appear in the working area, improving the safety and production efficiency of the machine tool processing process. Attached Figure Description

[0016] Figure 1 This is a perspective view of the protection system of the present invention.

[0017] Figure 2 This is a 3D view of the image acquisition component arranged on the mounting bracket assembly.

[0018] Figure 3 It is a stereoscopic image acquisition unit Figure 1 .

[0019] Figure 4 It is a stereoscopic image acquisition unit Figure 2 .

[0020] Figure 5 This is a schematic diagram of the image processing module.

[0021] Figure 6 This is a schematic diagram of a relay control module.

[0022] Figure 7 This is a schematic diagram of the field of view and detection area of ​​the camera in the vision system.

[0023] Figure 8 This is an optimized deep learning model processing flowchart.

[0024] Figure 9 This is a schematic diagram illustrating the working principle of the protection method of the present invention. Detailed Implementation

[0025] To enable those skilled in the art to better understand the technical solution of the present invention, the present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments.

[0026] Example 1

[0027] from Figure 1 , Figure 2 As can be seen, the present invention provides a protective system for a sheet metal forming machine tool based on machine vision, which includes an image acquisition component 1, a mounting bracket component 2, an image processing module 3, and a relay control module 4; the mounting bracket component 2 is fixedly connected to the side wall panel 6 of the machine tool, the image acquisition component 1 is mounted on the bracket component 2, the image processing module 3 is electrically connected to the image acquisition component 1, and one end of the relay control module 4 is electrically connected to the image processing module 3 and the other end is electrically connected to the electrical control box 5 of the machine tool.

[0028] Mounting bracket assembly 2 is connected to the side wall panel 6 of the machine tool, and fixes and supports image acquisition assembly 1; image acquisition assembly 1 acquires images of the machine tool's working field of view and outputs the images to image processing module 3; image processing module 3 analyzes and judges whether there are foreign objects by using an optimized deep learning model to receive image information of the machine tool's working area, and converts the output results into high and low levels; relay control module 4 receives high and low level signals from image processing module 3 to change the state of relay contacts, and makes the machine tool stop or work normally through electrical control box 5.

[0029] Example 2

[0030] from Figure 1 , Figure 2 , Figure 3 , Figure 4As can be seen, the protective system of the present invention includes: image acquisition component 1 comprising multiple image acquisition units, each image acquisition unit comprising a camera 1.1 and an image transmission cable 1.2; mounting bracket component 2 comprising left and right L-shaped side brackets 2.1, a crossbar 2.2, and left and right L-shaped top brackets 2.3; one end of the L-shaped side bracket 2.1 is fixedly connected to the outer side of the side wall panel 6 of the machine tool, and the other end is connected to the camera 1.1; the L-shaped top bracket 2.3 is fixedly connected to the top of the side wall panel 6 of the machine tool; both ends of the crossbar 2.2 are fixedly connected to the left and right L-shaped top brackets 2.3; multiple cameras 1.1 are installed at equal intervals on the crossbar 2.4; the cameras 1.1 are connected to the image processing module 3 through the image transmission cable 1.2.

[0031] The image acquisition unit is located on the top and outer side of the machine tool via the mounting bracket assembly 2: the camera 1.1 acquires images through the front camera 1.1.1, dissipates heat through the heat dissipation hole 1.1.3, and is connected to the image transmission cable 1.2 through the cable hole 1.1.2; the camera 1.1 transmits the acquired video image information to the image processing module 3 through the image transmission cable 1.2.

[0032] Example 3

[0033] from Figure 1 , Figure 2 , Figure 3 , Figure 4 As can be seen, the protective system of the present invention includes a positioning plate 1.3 and a positioning lock 1.4 for each image acquisition unit. The positioning plate 1.3 is connected to the back of the camera 1.1, and the positioning plate 1.3 is provided with a positioning lock 1.4. The positioning lock 1.4 is connected to the L-shaped side bracket 2.1 or the crossbar 2.2.

[0034] The positioning lock 1.4 connects the camera 1.1 to the mounting bracket assembly and allows for adjustment of the camera 1.1's angle, enabling field of view adjustment and precise positioning.

[0035] Example 4

[0036] from Figure 1 , Figure 2 , Figure 5 , Figure 6 As can be seen, the protection system of the present invention includes: an image processing module 3 comprising a camera serial interface 3.1, an embedded computing unit 3.2, and input / output pins 3.3; and a relay control module 4 comprising a DuPont wire 4.1, a relay pin 4.2, an internal relay circuit 4.3, a relay contact 4.4, and a control cable 4.5; the camera serial interface 3.1 is connected to the image transmission cable 1.2; the two ends of the DuPont wire 4.1 are respectively connected to the input / output pin 3.3 and the relay pin 4.2; and the two ends of the control cable 4.5 are respectively connected to the relay contact 4.4 and the machine tool control switch 5.

[0037] The camera serial interface 3.1 receives image data transmitted from the image transmission cable 1.2; the embedded computing unit 3.2 performs relevant processing on the image data through an internally optimized deep learning model, determines the presence of foreign objects, and outputs the detection results; the input / output pin 3.3 is used to convert the output results into high and low levels; the relay pin 4.2 receives high and low level signals through DuPont wires 4.1; the relay internal circuit 4.3 is used to activate the coil and change the contact state; the relay contact 4.4 is used to control the opening or closing of the machine tool circuit.

[0038] Example 5

[0039] like Figure 9 As shown, this invention discloses a protection method for a sheet metal forming machine tool based on machine vision. The steps are as follows: ① Multiple image acquisition units acquire image data of the working area in real time; ② The image processing module processes the image using an internal optimized deep learning model to determine whether there are foreign objects; if there are foreign objects, the input / output pins output a high level, and if there are no foreign objects, they output a low level; ③ The relay control module receives high and low level signals to control the machine tool to stop or operate normally: if it is a high level, the relay coil is energized, the normally closed contact opens, the main circuit of the machine tool is disconnected and stops working, the foreign object is manually removed, the machine tool is reset, the protection system is initialized, the output signal becomes an initial low level, and the system judges the next frame of the working area image; if it is a low level, the relay coil is de-energized, the normally closed contact closes, and the main circuit of the machine tool is connected and operates normally.

[0040] Example 6

[0041] The protection method of this invention: The method for determining the number and specific location of the image acquisition units in step ① is as follows:

[0042] I. Determine the inspection range based on the hazardous area during machine tool processing: For a machine tool with an upper die length of L and a lower die thickness of W, the hazardous area is divided into the following parts: the front end area of ​​the upper die cutting edge with a length of L and a width of 200mm; the rear end area of ​​the upper die cutting edge with a length of L and a width of 200mm; the two side areas of the cutting edge with a length of 50mm and a width of W+400mm on the left and right sides; and the lower die area with a length of L and a width of W. The above areas form a rectangular area with a length of L+100mm and a width of W+400mm.

[0043] II. Determine the number of cameras with adjustable angles: The length of the detection area covered by the field of view of a single camera is D, and the width is W+400mm. The number of cameras is the value of (L+100) / D rounded up (CEIL function).

[0044] III. Arrange cameras horizontally above the machine tool and on both sides of the machine tool so that the camera's field of view covers a rectangular area with a length of L+100mm and a width of W+400mm.

[0045] like Figure 7 As shown, the length of the die 7 on the machine tool is L, the thickness of the die 8 on the lower part of the machine tool is W, the area 9 to be detected by the system is represented by a solid line frame, and the working field of view 10 of the camera is represented by a dashed rectangle frame. The camera device acquires images within the field of view and transmits them to the image processing module via an image transmission cable.

[0046] The detection area length D of a single camera is generally 825mm-1000mm; the number of cameras is rounded using the CEIL function to ensure that adjacent cameras have a certain degree of overlap in their field of view, so that the camera's field of view can fully cover the entire rectangular area without blind spots, thereby enabling effective monitoring.

[0047] The hazardous area refers to a zone where the presence of any foreign object would be considered a danger, ensuring sufficient time and space to prevent dangerous situations in emergencies. The hazardous area is primarily concentrated near the cutting edge of the upper die. The definition of the hazardous area needs to be adjusted based on the machine tool manufacturer's recommendations and the actual operating environment, depending on the specific machine tool.

[0048] Example 7

[0049] The protection method of this invention is as follows: In step ①, the camera captures video images and transmits the images to the image processing module using a high-speed interface and an appropriate data transmission protocol. To ensure that each camera device acquires images at the same time, the system adopts a software clock synchronization mechanism and sets a uniform frame rate so that each camera acquires images at the same speed, avoiding data inconsistency or information loss. In image transmission, an appropriate bandwidth is set according to the number of camera devices to ensure that camera data can be transmitted smoothly. To cope with the synchronous reception of multiple data images, the embedded device adopts multi-threaded processing technology, that is, each thread receives data streams from a specific camera, thereby improving the overall data reception efficiency.

[0050] Example 8

[0051] like Figure 9 As shown, the protection method of the present invention: the image processing flow of the optimized deep learning model in step ② is as follows:

[0052] I. Preprocessing: While maintaining the aspect ratio of the image, the image is scaled to a specific size, the image pixel values ​​are normalized from 0-255 to the range of 0-1, the image is randomly transformed, and data augmentation methods such as rotation, translation, scaling, and flipping are used to increase data diversity and improve the generalization ability of the model.

[0053] II. Feature Extraction: A convolutional kernel is used to slide across the image, extracting basic features (including low-level features such as image edges and corners) through weighted summation. Then, an activation function (ReLU function) is used to process these features, changing negative values ​​to 0 and keeping positive values ​​unchanged, making important details in the image more prominent. Finally, pooling is performed to simplify the image by extracting the maximum and average values ​​of local regions, reducing the amount of data and retaining the main features.

[0054] III. Object Detection: The processed image is divided into multiple grids, each grid is responsible for detecting objects within its range. In each grid, the model generates multiple bounding boxes surrounding the object based on the size and shape of the object in the training dataset, and calculates the confidence score and probability of the class to which each bounding box belongs, generating multiple possible prediction results.

[0055] IV. Generating Prediction Results: The extracted image features are flattened into a one-dimensional vector in sequence and input into a fully connected layer. The features are further processed through weighted summation and activation functions to extract more complex feature representations, such as combined features of shape and color. The output layer receives the feature representation from the fully connected layer, uses a classification task activation function to transform the feature values ​​into a probability distribution, and selects the category with the highest probability as the final prediction result.

[0056] V. Post-processing: Adjust the bounding boxes to match the actual target, remove overlapping predicted boxes using non-maximum suppression (NMS), and retain only the boxes with the highest confidence; sum the probabilities defined as foreign object categories to obtain the total probability of having a foreign object, set a threshold, and if the probability is greater than the threshold, it is determined that there is a foreign object; if it is less than the threshold, it is determined that there is no foreign object.

[0057] Based on the judgment results of the deep learning model, the image processing module will generate corresponding level signals. If a foreign object is detected, a high level will be output; if no foreign object is detected, a low level will be output.

[0058] Example 9

[0059] like Figure 8 As shown, the protection method of the present invention includes the following steps: Step ② involves optimizing the deep learning model by importing the partitioned dataset into the model for training using a deep learning framework to obtain a preliminary model; integrating the compression and activation (SE) attention mechanism into the backbone network to obtain a hazardous area focus module; adding multi-size target capture units after the hazardous area focus module to obtain a hazardous object recognition module; classifying and recognizing the captured target areas, determining whether the target is a hazardous object, and outputting the category label and confidence level; based on the output of the hazardous object recognition module, the hazard assessment output layer further performs comprehensive analysis on the recognition results, assesses the risk level of the detected target in the current scene, and finally outputs a control signal to guide the machine tool's operating logic.

[0060] Specifically: the Dangerous Area Focus Module summarizes the extracted foreign object features, filtering out useless information while reducing the number of model parameters and improving the model's recognition accuracy; the Multi-Size Target Capture Unit adds small-sized convolutional kernels and dilated convolutions with different dilation rates to the original pyramid pooling module, which can reduce the number of channels while enabling the model to better maintain the target's position information during detection, reduce computational complexity, reduce information loss in the detection area, improve the model's ability to perceive small foreign objects, and achieve accurate recognition of objects of multiple sizes; in summary, the deep model obtained after improving the preliminary model has improved the recognition accuracy, and has also achieved significant optimization in terms of efficiency and applicability.

[0061] After improving the model, its performance needs to be comprehensively tested. A validation set should be used to evaluate the model's performance under different scenarios and conditions. Further adjustments to parameters and optimization algorithms are then made to ensure the model's stability and efficiency. Once testing is complete, the final model will be deployed to the image processing module.

[0062] Example 10

[0063] The protection method of this invention: The method for constructing the preliminary model of the deep learning model in step ② is as follows: I. When the machine tool is not working, simulate the occurrence of sheet metal and various foreign objects in different scenarios in the dangerous area, and collect these images to create a dataset; II. After completing the collection of dataset samples, use image annotation tools to classify and label the processed sheet metal and various foreign objects, and randomly divide them into training set, validation set and test set in a ratio of 7:1:2; III. Import the divided dataset into the built-in model of the image processing module through the deep learning framework for training to obtain the preliminary model of the deep learning model.

[0064] In step II: the pixel coordinates of the four vertices of the detection region are determined from the collected dataset images, the ratio of their coordinates to the total pixel coordinates of the image is calculated, a detection region mask is created, and logical operations are used to keep the pixel values ​​of the detection region unchanged while making the pixel values ​​of the images outside the region zero, thereby achieving the effect of extracting the detection region and masking other regions.

[0065] In Step I, the specific foreign objects appearing in the danger zone are as follows: 1. Human body parts: such as fingers, palms, and arms. If human body parts enter the danger zone, especially at the front, rear, and side of the cutting edge, it can cause serious personal injury. 2. Tools: such as wrenches, screwdrivers, pliers, and measuring tools. If these tools are accidentally dropped or left in the lower die area during operation, they may interfere with the normal operation of the machine tool, leading to decreased machining accuracy or even damage to the die. 3. Material fragments: such as metal fragments and sheet metal scraps. The fragments generated during machine tool processing are mainly found in the lower die area. The accumulation of these fragments will accelerate equipment wear, affecting machining quality and the lifespan of the machine tool. Based on the above analysis, by analyzing the spatial position and characteristic attributes of the object, it is determined whether it is in the danger zone, so as to distinguish between normal and abnormal situations. The following are the possible situations listed and analyzed: (1) Normal scenario: The normal scenario includes two situations: no plate is placed and plate is placed for processing. When no plate is placed, there should be no foreign object characteristics in the danger zone. When plate is placed for processing, the plate should always be in the cutting edge front area and slowly pushed from the worktable to the cutting edge area. In this case, the cutting edge front and the lower die area only have plate characteristics, and other detection areas should not have any foreign object characteristics. Based on the above two situations, if the cutting edge front and the lower die area only have plate characteristics and all detection areas have no foreign object characteristics, it is considered a normal situation. (2) Human body intrusion: When the operator pushes the plate, he accidentally puts his fingers, palm or arm into the danger zone. At this time, the cutting edge front area has both plate and human body characteristics, which is considered an abnormal situation. (3) Tool left behind: Before the machine tool is working, the tool is left in the danger zone. When the lower die area has tool characteristics, it is considered an abnormal situation. (4) Material Fragment Accumulation: During processing, material fragments accumulate in the lower die area. An appropriate threshold is designed based on fragment size and density. When the accumulated fragments in the lower die area reach the set safety threshold, it is considered an abnormal situation. Any scenarios not listed as normal are also classified as abnormal situations. Therefore, when the machine tool is not in operation, simulating the presence of sheet metal and various foreign objects in dangerous areas under different scenarios, and collecting these images to create a dataset for training a deep learning model, can ensure that the system can accurately identify and monitor foreign objects in dangerous areas.

[0066] In addition, to enhance the diversity of the dataset, environmental factors need to be considered during sample collection. Lighting conditions are a crucial factor; we will collect data under different lighting conditions, including natural daylight and nighttime working environments. By adjusting the intensity of artificial light sources, we will simulate changes in lighting at different times of day and collect images under various lighting conditions. Secondly, mechanical interference is also a significant factor. We will use a machine tool vibration simulator and impact generator to simulate the vibrations and impacts that machine tools may experience during actual operation and collect images under these interference conditions.

[0067] The advantages of the protection system of this invention are: 1. By setting the dangerous area range (including the front and rear positions of the upper and lower die-cutting molds, as well as the space between the upper and lower die-cutting molds), designing a reasonable camera arrangement scheme, and adjusting the cameras to appropriate angles, all-round monitoring of the dangerous area is achieved, improving adaptability to different machine tool equipment, avoiding blind spots, and thus improving machine tool safety; 2. By optimizing the deep learning model, adding an attention mechanism and an improved pyramid pooling module after the model backbone network, the detection speed and recognition accuracy are improved, enabling the model to intelligently analyze complex scenes and accurately detect abnormal situations where small foreign objects enter the dangerous area; 3. The relay control module can receive signals to realize the emergency stop and reset of the machine tool, achieving rapid response, improving the automation level of the machine tool, and reducing safety hazards caused by operator error. In summary, when machine tools process sheet metal, the protection system of this invention provides high-precision all-round monitoring of its working area, and can quickly respond to form an efficient protection scheme when foreign objects appear in the working area, improving the safety and production efficiency of the machine tool processing process.

Claims

1. A protective system for a sheet metal forming machine tool based on machine vision, characterized in that: It includes an image acquisition component (1), a mounting bracket component (2), an image processing module (3), and a relay control module (4); the mounting bracket component (2) is fixedly connected to the side wall panel (6) of the machine tool, the image acquisition component (1) is mounted on the bracket component (2), and the image processing module (3) is electrically connected to the image acquisition component (1) to process the image through an internal optimized deep learning model to determine whether there is a foreign object; one end of the relay control module (4) is electrically connected to the image processing module (3), and the other end is electrically connected to the electrical control box (5) of the machine tool to receive high and low level signals and control the machine tool to stop or work normally; the image acquisition component (1) includes multiple image acquisition units, each image acquisition unit includes a camera (1.1) and an image transmission cable (1.2), and the mounting bracket component (2) includes two left and right L-shaped side brackets (2.1), a crossbar (2.2), and two left and right L-shaped top brackets (2.3); one end of the L-shaped side bracket (2.1) is connected to the outer side wall panel (6) of the machine tool. The side is fixedly connected, and the other end is connected to the camera (1.1); the L-shaped top bracket (2.3) is fixedly connected to the top of the side wall panel (6) of the machine tool, and the two ends of the crossbar (2.2) are fixedly connected to the left and right L-shaped top brackets (2.3); multiple cameras (1.1) are installed at equal intervals on the crossbar (2.2); the camera (1.1) is connected to the image processing module (3) through the image transmission cable (1.2); the method for determining the number and specific location of the image acquisition unit is: I. According to the processing of the machine tool Determining the detection range of the hazardous area: For a machine tool with an upper die length of L and a lower die thickness of W, the hazardous area is divided into the following parts: the front end area of ​​the upper die cutting edge with a length of L and a width of 200mm; the rear end area of ​​the upper die cutting edge with a length of L and a width of 200mm; the two side areas of the cutting edge with a length of 50mm and a width of W+400mm on the left and right sides; and the lower die area with a length of L and a width of W. The above areas form a rectangular area with a length of L+100mm and a width of W+400mm. II. Determine the number of adjustable-angle cameras: The length of the detection area covered by the field of view of a single camera is D, and the width is W+400mm. The number of cameras is the value of (L+100) / D rounded up. III. Arrange cameras horizontally above the machine tool and on both sides of the machine tool so that the field of view of the cameras covers a rectangular area with a length of L+100mm and a width of W+400mm.

2. The protection system according to claim 1, characterized in that: Each image acquisition unit also includes a positioning plate (1.3) and a positioning lock (1.4). The positioning plate (1.3) is connected to the back of the camera (1.1), and the positioning plate (1.3) is equipped with a positioning lock (1.4). The positioning lock (1.4) is connected to the L-shaped side bracket (2.1) or the crossbar (2.2).

3. The protection system according to claim 1, characterized in that: The image processing module (3) includes a camera serial interface (3.1), an embedded computing unit (3.2), and input / output pins (3.3). The relay control module (4) includes a DuPont wire (4.1), a relay pin (4.2), an internal relay circuit (4.3), a relay contact (4.4), and a control cable (4.5). The camera serial interface (3.1) is connected to the image transmission cable (1.2). The two ends of the DuPont wire (4.1) are connected to the input / output pin (3.3) and the relay pin (4.2) respectively. The two ends of the control cable (4.5) are connected to the relay contact (4.4) and the machine tool control switch (5) respectively.

4. The protection method of the protection system for a sheet metal forming machine tool based on machine vision according to any one of claims 1-3, comprising the following steps: ① Multiple image acquisition units acquire image data of the working area in real time; ② The image processing module processes the image through an internal optimized deep learning model to determine whether there is a foreign object; if there is a foreign object, the input / output pin outputs a high level, and if there is no foreign object, it outputs a low level; ③ The relay control module receives high and low level signals to control the machine tool to stop or operate normally: if it is a high level, the relay coil is energized, the normally closed contact is opened, the main circuit of the machine tool is disconnected and stops working, the foreign object is manually removed, the machine tool is reset, the protection system is initialized, the output signal becomes an initial low level, and the system judges the next frame of the working area image; if it is a low level, the relay coil is de-energized, the normally closed contact is closed, and the main circuit of the machine tool is connected and operates normally.

5. The protection method according to claim 4, characterized in that: in step ①: the camera captures video images and transmits the images to the image processing module using a high-speed interface and an appropriate data transmission protocol; to ensure that each camera device acquires images at the same time, the system adopts a software clock synchronization mechanism and sets a uniform frame rate so that each camera acquires images at the same speed, avoiding data inconsistency or information loss; in image transmission, an appropriate bandwidth is set according to the number of camera devices so that camera data can be transmitted smoothly; to cope with the synchronous reception of multiple data images, the embedded device adopts multi-threaded processing technology, that is, each thread receives data streams from a specific camera, thereby improving the overall data reception efficiency.

6. The protection method according to claim 4, characterized in that: The image processing flow of the optimized deep learning model in step ② is as follows: Ⅰ Preprocessing: While maintaining the aspect ratio of the image, the image is scaled to a specific size, the image pixel values ​​are normalized from 0-255 to the range of 0-1, the image is randomly transformed, and data diversity is increased through data augmentation methods such as rotation, translation, scaling, and flipping to improve the generalization ability of the model. II. Feature Extraction: A convolutional kernel slides across the image, extracting basic features through weighted summation. Then, an activation function processes these features, converting negative values ​​to 0 and leaving positive values ​​unchanged, making important details in the image more prominent. Finally, pooling simplifies the image by extracting the maximum and average values ​​of local regions, reducing the amount of data while retaining key features. III. Object Detection: The processed image is divided into multiple grids, each responsible for detecting objects within its range. Within each grid, the model generates multiple bounding boxes around the object based on the size and shape of the target object in the training dataset, calculating the confidence score and probability of each bounding box's class, and generating multiple possible predictions. Results; IV. Generating Prediction Results: The extracted image features are flattened into a one-dimensional vector in sequence and input into a fully connected layer. The features are further processed by a weighted summation activation function to extract combined shape and color features. The output layer receives the feature representation from the fully connected layer and uses a classification task activation function to transform the feature values ​​into a probability distribution. The category with the highest probability is selected as the final prediction result. V. Post-processing: The bounding boxes are adjusted to match the actual target. Overlapping prediction boxes are removed by non-maximum suppression, and only the boxes with the highest confidence are retained. The probabilities defined as foreign object categories are summed to obtain the total probability of the presence of a foreign object. A threshold is set. If the probability is greater than the threshold, it is determined that there is a foreign object; if it is less than the threshold, it is determined that there is no foreign object.

7. The protection method according to claim 4, characterized in that: The method for optimizing the deep learning model in step ② is as follows: the dataset after division is imported into the model for training through the deep learning framework to obtain the preliminary model; after the compression and incentive attention mechanism are integrated into the backbone network, the danger area focus module is obtained. A multi-size target capture unit is added after the hazardous area focus module to obtain the hazardous object recognition module. The hazardous object recognition module classifies and identifies the captured target areas, determines whether the target is a hazardous object, and outputs the category label and confidence level. Based on the output of the hazardous object recognition module, the hazard assessment output layer further performs a comprehensive analysis of the recognition results, assesses the risk level of the detected target in the current scene, and finally outputs control signals to guide the machine tool's operating logic.

8. The protection method according to claim 4, characterized in that: The preliminary model construction method for the deep learning model in step ② is as follows: Ⅰ. When the machine tool is not working, simulate the occurrence of sheet metal and various foreign objects in different scenarios in the dangerous area, and collect these images to create a dataset; Ⅱ. After completing the collection of dataset samples, use image annotation tools to classify and label the processed sheet metal and various foreign objects, and randomly divide them into training set, validation set and test set in a ratio of 7:1:2; Ⅲ. Import the divided dataset into the built-in model of the image processing module through the deep learning framework for training, and obtain the preliminary model of the deep learning model.

Images