Protection system and protection method for plate forming machine tool based on machine vision

Abstract

The invention discloses a protection system of a plate forming machine tool based on machine vision. The protection system comprises an image acquisition assembly, a mounting bracket assembly, an image processing module and a relay control module, the mounting bracket assembly is fixedly connected with a side wall plate of a machine tool, the image acquisition assembly is mounted on the bracket assembly, the image processing module is electrically connected with the image acquisition assembly, one end of the relay control module is electrically connected with the image processing module, and the other end of the relay control module is electrically connected with an electric cabinet of the machine tool. According to the protection system, high-precision omnibearing monitoring on the working area of the machine tool is achieved when the machine tool machines plates, quick response can be achieved when foreign matter appears in the working area, and the safety of the machine tool in the machining process is improved. Meanwhile, the invention further provides a protection method of the plate forming machine tool based on machine vision.

Description

Technical Field

[0001] The invention belongs to the field of industrial automation and numerical control machine tool processing, and in particular relates to a protection system and a protection method for a plate forming processing machine tool based on machine vision. Background Art

[0002] With the continuous development of industrial manufacturing, safety issues in industrial production have received more and more attention. As an important equipment in the field of mechanical processing, sheet metal forming processing machine tools are widely used in industrial production. Shearing machines and bending machines are typical sheet metal forming processing equipment, and their working process is as follows: the staff places the sheet to be processed between the upper cutting die and the lower cutting die, and uses the reciprocating motion of the upper cutting die relative to the lower cutting die to process the sheet into the required shape. During the operation, the staff may extend body parts or tools between the upper and lower cutting dies, which poses a risk of personal injury and damage to the machine tool. In order to avoid human-machine failures during the production process and protect the safety of staff and equipment, a safety protection system is crucial.

[0003] Existing safety protection schemes mostly adopt the method of installing infrared devices at different positions of machine tools to achieve protection. There are two common schemes: one is Chinese patent CN216802715U (announcement date 2022.06.24), which discloses a protective device for a CNC shearing machine, which is to install an infrared grating at the front end of the machine tool workbench to form a plane light path in the front end area of ​​the upper and lower cutting dies of the machine tool. If an object appears in this area to block the light path, the infrared device triggers the machine tool to stop working; another Chinese patent is CN219684111U (announcement date 2023.09.15), which discloses a new shearing machine with a protective structure, which is to install an infrared device on the machine tool frame. When the machine tool is working, the infrared device moves with the upper cutting die to form a plane light path in the area below the upper cutting die. When an object enters the area below the upper cutting die to block the light path, the machine tool stops working. The infrared sensors in the above two patents are used as point detectors, and have the following disadvantages in practical applications: First, there is a gap between the different transmitting and receiving ends of the infrared sensor. Therefore, when the height of a small object or obstacle does not exceed the gap, it cannot be detected even if the object exists, especially when foreign objects are placed or dropped into the groove of the lower die; Second, the vibration of the shearing machine during operation will interfere with the infrared equipment, significantly increase the false alarm rate, affect the stability of the equipment, and make debugging and maintenance time-consuming and labor-intensive. Summary of the invention

[0004] The problem to be solved by the present invention is to provide a protection system for a sheet metal forming machine tool based on machine vision, which can realize high-precision and all-round monitoring of its working area when the machine tool is processing a sheet metal, and can respond quickly when foreign matter appears in the working area, thereby improving the safety of the machine tool during processing. At the same time, the present invention also provides a protection method for a sheet metal forming machine tool based on machine vision.

[0005] The present invention discloses a protection system for a sheet metal forming processing machine tool based on machine vision, which comprises an image acquisition component, a mounting bracket component, an image processing module, and a relay control module; the mounting bracket component is fixedly connected to a side wall panel of the machine tool, the image acquisition component is installed on the bracket component, the image processing module is electrically connected to the image acquisition component, one end of the relay control module is electrically connected to the image processing module, and the other end is electrically connected to an electric control box of the machine tool.

[0006] Furthermore, the image acquisition assembly includes multiple image acquisition units, each image acquisition unit includes a camera and an image transmission cable, and the mounting bracket assembly includes left and right L-shaped side brackets, a cross bar, and left and right L-shaped top brackets; one end of the L-shaped side bracket is fixedly connected to the outer side of the side wall panel of the machine tool, and the other end is connected to the camera; the L-shaped top bracket is fixedly connected to the upper side of the side wall panel of the machine tool, and the two ends of the cross bar are fixedly connected to the left and right L-shaped top brackets; multiple cameras are installed on the cross bar at equal intervals; the camera is connected to the image processing module through an image transmission cable.

[0007] Furthermore, each image acquisition unit also includes a positioning plate and a positioning lock. The positioning plate is connected to the back of the camera. The positioning plate is provided with a positioning lock, and the positioning lock is connected to the L-shaped side bracket or the cross bar.

[0008] Furthermore, the image processing module includes a camera serial interface, an embedded computing unit, input and output pins, and the relay control module includes a DuPont line, a relay pin, a relay internal circuit, a relay contact, and a control cable; the camera serial interface is connected to the image transmission cable, the two ends of the DuPont line are respectively connected to the input and output pins and the relay pins, and the two ends of the control cable are respectively connected to the relay contacts and the control switch of the machine tool.

[0009] The present invention discloses a protection method for a sheet metal forming processing machine tool based on machine vision, and the steps are as follows: 1. multiple image acquisition units collect image data of a working area in real time; 2. an image processing module processes the image through an internal optimized deep learning model to determine whether there is a foreign object; if there is a foreign object, the input and output pins output a high level, and if there is no foreign object, the input and output pins output a low level; 3. a relay control module receives high and low level signals to control the shutdown or normal operation of the machine tool: if it is a high level, the relay coil is energized, the normally closed contact is disconnected, the main circuit of the machine tool is disconnected and stops working, the foreign object is manually removed, the machine tool is reset, the protection system is initialized, the output signal becomes an initial low level, and the system determines the next frame of the working area image; if it is a low level, the relay coil is powered off, the normally closed contact is closed, and the main circuit of the machine tool is connected to work normally.

[0010] Furthermore, the method for determining the number and specific positions of the image acquisition units in step ① is as follows: Ⅰ. Determine the detection range according to the dangerous area during machine tool processing: For a machine tool with an upper die length of L and a lower die thickness of W, the dangerous area is divided into the following parts: a front end area of ​​the upper die blade with a length of L and a width of 200 mm; a rear end area of ​​the upper die blade with a length of L and a width of 200 mm; two side areas of the blade with a length of 50 mm and a width of W+400 mm on the left and right sides; a lower die blade with a length of L and a width of W. Area; The above areas form a rectangular area with a length of L+100mm and a width of W+400mm; II. Determine the number of cameras with adjustable angles: the detection area covered by the field of view of a single camera is of length D and width W+400mm, and the number of cameras is the value of (L+100) / D rounded up; III. Arrange cameras horizontally above the machine tool and on both sides of the machine tool so that the field of view of the camera covers a rectangular area with a length of L+100mm and a width of W+400mm.

[0011] Furthermore, in step ①: the camera captures the video image, and uses a high-speed interface and an appropriate data transmission protocol to transmit the image to the image processing module; to ensure that each camera device collects images at the same time point, the system adopts a software clock synchronization mechanism and sets a unified frame rate so that each camera collects images at the same speed to avoid data inconsistency or information loss; in image transmission, a suitable bandwidth is set according to the number of camera devices so that the camera data can be transmitted smoothly. In order to cope with the synchronous reception of multiple data images, the embedded device adopts multi-threaded processing technology, that is, each thread receives the data stream from a specific camera, thereby improving the overall data reception efficiency.

[0012] Furthermore, the image processing flow of the deep learning model optimized in step ② is as follows: Ⅰ Preprocessing: Under the premise of maintaining the image aspect ratio, scale the image to a specific size, normalize the image pixel value from 0-255 to the range of 0-1, randomly transform the image, and increase data diversity through data enhancement methods such as rotation, translation, scaling, flipping, etc., to improve the generalization ability of the model; Ⅱ Feature extraction: Use the convolution kernel to slide on the image, and extract the basic features of the image through weighted summation (including low-level features such as image edges and corners); then use the activation function (RELU function) to process these features, change negative values ​​to 0, and keep positive values ​​unchanged, so that important detail features in the image are more obvious; finally, perform pooling operation to simplify the image, and reduce the amount of data and retain the main features by extracting the maximum and average values ​​of the local area; Ⅲ Target detection: Divide the processed image into multiple grids, each grid is responsible for detecting the target within its range, and in each grid, the target is detected. In the grid, the model generates multiple bounding boxes surrounding the object based on the size and shape of the target object in the training data set, and calculates the confidence of each bounding box and the probability of the category to which it belongs, generating multiple possible prediction results; IV Generate prediction results: Flatten the image features extracted by features into a one-dimensional vector in order, input it into the fully connected layer, further process the features through weighted summation and activation function, and extract more complex feature representations, such as the combination of shape and color features; The output layer receives the feature representation of the fully connected layer, uses the classification task activation function to convert the feature value into a probability distribution, and selects the category with the highest probability as the final prediction result; V Post-processing: Adjust the bounding box to match the actual target, remove overlapping prediction boxes through non-maximum suppression (NMS), and only keep the box with the highest confidence; Sum the probabilities defined as foreign body categories to obtain the total probability of foreign body, set a threshold, if the probability is greater than the threshold, it is judged that there is a foreign body, if it is less than the threshold, it is judged that there is no foreign body.

[0013] Furthermore, the method for optimizing the deep learning model in step ② is as follows: the divided data set is imported into the model for training through the deep learning framework to obtain a preliminary model; after integrating the compression and excitation (SE) attention mechanism into the backbone network, a dangerous area focus module is obtained; a multi-size target capture unit is added after the dangerous area focus module to obtain a dangerous object recognition module; the dangerous object recognition module classifies and recognizes the captured target area, determines whether the target belongs to a dangerous object, and outputs a category label and confidence; based on the output of the dangerous object recognition module, the hazard assessment output layer further comprehensively analyzes the recognition results, evaluates the risk level of the detected target in the current scene, and finally outputs a control signal to guide the operation logic of the machine tool.

[0014] Furthermore, the method for constructing the preliminary model of the deep learning model in step ② is as follows: Ⅰ. When the machine tool is not working, simulate the appearance of plates and various foreign objects in different scenarios in the dangerous area, and collect these images to produce a data set; Ⅱ. After completing the collection of data set samples, use the image annotation tool to classify and annotate the processed plates and various foreign objects, and randomly divide them into training set, verification set and test set in a ratio of 7:1:2; Ⅲ. Import the divided data set into the built-in model of the image processing module through the deep learning framework for training, and obtain the preliminary model of the deep learning model.

[0015] The advantages of the protection system of the present invention are: 1. By setting the range of the dangerous area (including the front position, rear position of the upper and lower cutting dies and the space between the upper and lower cutting dies), designing a reasonable camera layout plan, and adjusting the camera to a suitable angle, all-round monitoring of the dangerous area is achieved, which improves the adaptability to different machine tools and equipment, avoids the appearance of blind spots in the field of vision, and thus improves the safety of the machine tool; 2. By optimizing the deep learning model, adding an attention mechanism and an improved pyramid pooling module after the model backbone network, the detection speed and recognition accuracy are improved, and the model can be intelligently analyzed for complex scenes, and accurately detect the abnormal situation of small foreign objects entering the dangerous area; 3. The relay control module can receive signals to realize the emergency stop and reset of the machine tool, achieve rapid response, improve the automation of the machine tool, and reduce the safety hazards caused by operator misoperation. In short, when the machine tool processes the plate, the protection system of the present invention monitors its working area with high precision and in all directions, and can quickly respond to form an efficient protection plan when foreign objects appear in the working area, thereby improving the safety and production efficiency during the machine tool processing process. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] Figure 1 It is a three-dimensional diagram of the protection system of the present invention.

[0017] Figure 2 It is a stereoscopic diagram of the image acquisition assembly arranged on the mounting bracket assembly.

[0018] Figure 3 is the stereoscopic image acquisition unit Figure 1 .

[0019] Figure 4 is the stereoscopic image acquisition unit Figure 2 .

[0020] Figure 5 It is a schematic diagram of the image processing module.

[0021] Figure 6 is a schematic diagram of the relay control module.

[0022] Figure 7 Schematic diagram of the visual system camera field of view and detection area

[0023] Figure 8 It is an optimized deep learning model processing flowchart.

[0024] Fig. 9 It is a working principle diagram of the protection method of the present invention. DETAILED DESCRIPTION

[0025] In order to enable those skilled in the art to better understand the technical solution of the present invention, the present invention is further described in detail below in conjunction with the accompanying drawings and specific implementation methods.

[0026] Example 1

[0027] from Figure 1 , Figure 2 It can be seen that the present invention is a protection system for a sheet metal forming processing machine tool based on machine vision, which includes an image acquisition component 1, a mounting bracket component 2, an image processing module 3, and a relay control module 4; the mounting bracket component 2 is fixedly connected to the side wall panel 6 of the machine tool, the image acquisition component 1 is installed on the bracket component 2, the image processing module 3 is electrically connected to the image acquisition component 1, one end of the relay control module 4 is electrically connected to the image processing module 3, and the other end is electrically connected to the electric control box 5 of the machine tool.

[0028] The mounting bracket assembly 2 is connected to the side wall panel 6 of the machine tool, and fixes and supports the image acquisition assembly 1; the image acquisition assembly 1 acquires the image of the working field of the machine tool, and outputs the image to the image processing module 3; the image processing module 3 analyzes the received image information of the working area of ​​the machine tool through an optimized deep learning model, determines whether there is foreign matter, and converts the output result into high and low levels; the relay control module 4 receives the high and low level signals of the image processing module 3 to change the state of the relay contact, and stops or operates the machine tool normally through the electric control box 5.

[0029] Example 2

[0030] from Figure 1 , Figure 2 , Figure 3 , Figure 4It can be known that the protection system of the present invention: the image acquisition component 1 includes a plurality of image acquisition units, each image acquisition unit includes a camera 1.1, an image transmission cable 1.2, the mounting bracket component 2 includes left and right L-shaped side brackets 2.1, a cross bar 2.2, and left and right L-shaped top brackets 2.3; one end of the L-shaped side bracket 2.1 is fixedly connected to the outer side of the side wall panel 6 of the machine tool, and the other end is connected to the camera 1.1; the L-shaped top bracket 2.3 is fixedly connected to the upper side of the side wall panel 6 of the machine tool, and the two ends of the cross bar 2.2 are fixedly connected to the left and right L-shaped top brackets 2.3; a plurality of cameras 1.1 are installed at equal intervals on the cross bar 2.4; the camera 1.1 is connected to the image processing module 3 via the image transmission cable 1.2.

[0031] The image acquisition unit is located on the top and outer side of the machine tool through the mounting bracket assembly 2: the camera 1.1 acquires images through the front camera 1.1.1, dissipates heat through the heat dissipation holes 1.1.3, and is connected to the image transmission cable 1.2 through the cable hole 1.1.2; the camera 1.1 transmits the acquired video image information to the image processing module 3 through the image transmission cable 1.2.

[0032] Example 3

[0033] from Figure 1 , Figure 2 , Figure 3 , Figure 4 It can be seen that the protection system of the present invention: each image acquisition unit also includes a positioning plate 1.3 and a positioning lock 1.4, the positioning plate 1.3 is connected to the back of the camera 1.1, the positioning plate 1.3 is provided with a positioning lock 1.4, and the positioning lock 1.4 is connected to the L-shaped side bracket 2.1 or the cross bar 2.2.

[0034] The positioning lock 1.4 connects the camera 1.1 to the mounting bracket assembly and can adjust the angle of the camera 1.1 to adjust the field of view and accurately position it.

[0035] Example 4

[0036] from Figure 1 , Figure 2 , Figure 5 , Figure 6 It can be seen that the protection system of the present invention: the image processing module 3 includes a camera serial interface 3.1, an embedded computing unit 3.2, and an input-output pin 3.3; the relay control module 4 includes a DuPont line 4.1, a relay pin 4.2, a relay internal circuit 4.3, a relay contact 4.4, and a control cable 4.5; the camera serial interface 3.1 is connected to the image transmission cable 1.2, the two ends of the DuPont line 4.1 are respectively connected to the input-output pin 3.3 and the relay pin 4.2, and the two ends of the control cable 4.5 are respectively connected to the relay contact 4.4 and the control switch 5 of the machine tool.

[0037] The camera serial interface 3.1 receives the image data transmitted by the image transmission cable 1.2; the embedded computing unit 3.2 performs relevant processing on the image data through an internally optimized deep learning model, determines whether there is a foreign object and outputs the detection result; the input and output pin 3.3 is used to convert the output result into a high or low level; the relay pin 4.2 receives the high and low level signal through the DuPont line 4.1; the relay internal circuit 4.3 is used to activate the coil to change the contact state; the relay contact 4.4 is used to control the opening or closing of the machine tool circuit.

[0038] Example 5

[0039] like Fig. 9 As shown, the present invention provides a protection method for a sheet metal forming processing machine tool based on machine vision, and the steps are: ① multiple image acquisition units collect image data of the working area in real time; ② the image processing module processes the image through an internal optimized deep learning model to determine whether there is a foreign object; if there is a foreign object, the input and output pins output a high level, and if there is no foreign object, a low level is output; ③ the relay control module receives high and low level signals to control the shutdown or normal operation of the machine tool: if it is a high level, the relay coil is energized, the normally closed contact is disconnected, the main circuit of the machine tool is disconnected and stops working, the foreign object is manually removed, the machine tool is reset, the protection system is initialized, the output signal becomes an initial low level, and the system judges the next frame of the working area image; if it is a low level, the relay coil is powered off, the normally closed contact is closed, and the main circuit of the machine tool is connected to work normally.

[0040] Example 6

[0041] The protection method of the present invention: The method for determining the number and specific positions of the image acquisition units in step ① is:

[0042] Ⅰ. Determine the detection range according to the dangerous area during machine tool processing: For a machine tool with an upper die length of L and a lower die thickness of W, the dangerous area is divided into the following parts: the front end area of ​​the upper die blade with a length of L and a width of 200mm; the rear end area of ​​the upper die blade with a length of L and a width of 200mm; the two side areas of the blade with a length of 50mm and a width of W+400mm on the left and right sides; the lower die area with a length of L and a width of W; the above areas form a rectangular area with a length of L+100mm and a width of W+400mm;

[0043] Ⅱ. Determine the number of cameras with adjustable angles: the length of the detection area covered by a single camera's field of view is D, and the width is W+400mm. The number of cameras is the value of (L+100) / D rounded up (CEIL function);

[0044] III. Arrange cameras horizontally above the machine tool and on both sides of the machine tool so that the camera's field of view covers a rectangular area with a length of L+100mm and a width of W+400mm.

[0045] like Figure 7 As shown, the length of the upper die 7 of the machine tool is L, the thickness of the lower die 8 of the machine tool is W, the area 9 that the system needs to detect is represented by a solid line frame, and the camera working field 10 is represented by a dotted rectangular frame. The camera device collects images in the field of view and transmits them to the image processing module through the image transmission cable.

[0046] The length D of the detection area of ​​a single camera is generally 825mm-1000mm. The number of cameras is rounded using the CEIL function to ensure that adjacent cameras have a certain degree of overlap in field of view, so that the field of view of the camera can fully cover the entire rectangular area without blind spots, thereby effectively monitoring it.

[0047] The danger zone refers to the area where any foreign object entering will be considered dangerous, thus ensuring that there is enough time and space to avoid dangerous situations in an emergency. The danger zone is mainly concentrated near the upper die cutting edge. For different machine tools, the setting of the danger zone needs to be adjusted according to the machine tool manufacturer's recommendations and the actual operating environment.

[0048] Example 7

[0049] The protection method of the present invention comprises the following steps: in step ①, a camera captures a video image, and uses a high-speed interface and an appropriate data transmission protocol to transmit the image to an image processing module; to ensure that each camera device collects images at the same time point, the system adopts a software clock synchronization mechanism and sets a uniform frame rate, so that each camera collects images at the same speed, thereby avoiding data inconsistency or information loss; in image transmission, a suitable bandwidth is set according to the number of camera devices, so that camera data can be smoothly transmitted, and in order to cope with the synchronous reception of multi-channel data images, the embedded device adopts a multi-threaded processing technology, that is, each thread receives a data stream from a specific camera, thereby improving the overall data reception efficiency.

[0050] Example 8

[0051] like Fig. 9 As shown, the protection method of the present invention: the process of processing the image by the optimized deep learning model in step ② is:

[0052] Ⅰ Preprocessing: Under the premise of maintaining the aspect ratio of the image, the image is scaled to a specific size, the image pixel value is normalized from 0-255 to the range of 0-1, the image is randomly transformed, and data enhancement methods such as rotation, translation, scaling, and flipping are used to increase data diversity and improve the generalization ability of the model;

[0053] Ⅱ Feature extraction: Use the convolution kernel to slide on the image, and extract the basic features of the image through weighted summation (including low-level features such as image edges and corners); then use the activation function (RELU function) to process these features, changing negative values ​​to 0 and keeping positive values ​​unchanged, so that the important details in the image are more obvious; finally, perform pooling to simplify the image, and reduce the amount of data by extracting the maximum and average values ​​of the local area, while retaining the main features;

[0054] III. Object detection: The processed image is divided into multiple grids, each of which is responsible for detecting the object within its range. In each grid, the model generates multiple bounding boxes surrounding the object based on the size and shape of the target object in the training dataset, and calculates the confidence and probability of each bounding box to generate multiple possible prediction results.

[0055] IV Generate prediction results: Flatten the extracted image features into a one-dimensional vector in order, input it into the fully connected layer, further process the features through weighted summation and activation function, and extract more complex feature representations, such as the combination of shape and color features; the output layer receives the feature representation of the fully connected layer, uses the classification task activation function to convert the feature value into a probability distribution, and selects the category with the highest probability as the final prediction result;

[0056] V. Post-processing: Adjust the bounding box to match the actual target, remove overlapping prediction boxes through non-maximum suppression (NMS), and only keep the box with the highest confidence; sum the probabilities defined as foreign object categories to obtain the total probability of foreign objects, set a threshold, and if the probability is greater than the threshold, it is judged that there is a foreign object, if it is less than the threshold, it is judged that there is no foreign object.

[0057] According to the judgment results of the deep learning model, the image processing module will generate a corresponding level signal. If a foreign object is detected, a high level will be output; if no foreign object is detected, a low level will be output.

[0058] Example 9

[0059] like Figure 8 As shown, the protection method of the present invention: the method for optimizing the deep learning model in step ② is: importing the divided data set into the model for training through a deep learning framework to obtain a preliminary model; integrating the compression and excitation (SE) attention mechanism into the backbone network to obtain a dangerous area focus module; adding a multi-size target capture unit after the dangerous area focus module to obtain a dangerous object recognition module; the dangerous object recognition module classifies and recognizes the captured target area, determines whether the target belongs to a dangerous object, and outputs a category label and confidence; based on the output of the dangerous object recognition module, the risk assessment output layer further comprehensively analyzes the recognition results, evaluates the risk level of the detected target in the current scene, and finally outputs a control signal to guide the operation logic of the machine tool.

[0060] Among them: the dangerous area focus module summarizes the extracted foreign body features, filters out useless information, reduces the number of model parameters, and improves the recognition accuracy of the model; the multi-size target capture unit adds small-size convolution kernels and dilated convolutions with different expansion rates to the original pyramid pooling module, which can reduce the number of channels while enabling the model to better maintain the target's position information during the detection process, reduce the complexity of calculation, reduce the information loss in the detection area, improve the model's perception of small target foreign objects, and achieve accurate recognition of multi-size objects; in short, the deep model obtained after improving the preliminary model improves the recognition accuracy, and also achieves significant optimization in efficiency and applicability.

[0061] After the model is improved, its performance needs to be fully tested, and the validation set is used to evaluate the performance of the model in different scenarios and conditions, and further adjust the parameters and optimize the algorithm to ensure the stability and efficiency of the model. After the test is completed, the final model is deployed to the image processing module.

[0062] Example 10

[0063] The protection method of the present invention: the construction method of the preliminary model of the deep learning model in step ② is: Ⅰ, when the machine tool is not working, simulate the appearance of plates and various foreign objects in different scenes in the dangerous area, and collect these images to produce a data set; Ⅱ, after completing the collection of data set samples, use the image annotation tool to classify and annotate the processed plates and various foreign objects, and randomly divide them into training set, verification set and test set in a ratio of 7:1:2; Ⅲ, import the divided data set into the built-in model of the image processing module through the deep learning framework for training, and obtain the preliminary model of the deep learning model.

[0064] In step II: determine the pixel coordinates of the four vertices of the detection area in the collected data set image, calculate the ratio of their coordinates to the total pixel coordinates of the image, make a detection area mask, and use logical operations to keep the image pixel values ​​of the detection area unchanged and the image pixel values ​​outside the area zero, so as to extract the detection area and shield other areas.

[0065] In step I: foreign objects appearing in the dangerous area are as follows: 1. Human body parts: such as fingers, palms and arms. Human body parts entering the dangerous area, especially in the front, rear and side areas of the blade, will cause serious personal injury; 2. Tools: such as wrenches, screwdrivers, pliers and measuring tools. These tools accidentally dropped or left in the lower die area during operation may interfere with the normal operation of the machine tool, resulting in reduced processing accuracy and even damage to the mold; 3. Material fragments: such as metal fragments and plate scraps. The fragments generated during the machine tool processing are mainly in the lower die area. The accumulation of these fragments will aggravate the wear of the equipment, affecting the processing quality and the life of the machine tool equipment. Combined with the above analysis, by analyzing the spatial position and characteristic attributes of the object, it is determined whether it is in the danger zone to distinguish between normal and abnormal situations. The following lists and analyzes the possible situations: (1) Normal scenario: The normal scenario includes two situations: no plate is placed and the plate is placed for processing. When the plate is not placed, there should be no foreign body features in the danger zone; when the plate is placed for processing, the plate position should always be in the front end area of ​​the blade, and the workbench is slowly pushed to the blade area. In this case, there are only plate features in the front end of the blade and the lower die area, and no foreign body features should appear in other detection areas; based on the above two situations, if there are only plate features in the front end of the blade and the lower die area, and there are no foreign body features in all detection areas, it is considered a normal situation. (2) Human body intrusion: When the operator accidentally extends his fingers, palms or arms into the danger zone when pushing the plate, the front end area of ​​the blade shows both plate and human body features, which is considered an abnormal situation. (3) Tool left behind: Before the machine tool works, the tool is left in the danger zone. When there are tool features in the lower die area, it is considered an abnormal situation. (4) Accumulation of material fragments: During the processing, material fragments accumulate in the lower die area. Appropriate thresholds are designed based on the size and density of the fragments. When the accumulation of fragments in the lower die area reaches the set safety threshold, it is considered an abnormal situation. All scenes that are not listed and are not normal are classified as abnormal situations. Therefore, when the machine tool is not working, the appearance of plates and various foreign objects in dangerous areas in different scenarios is simulated, and these images are collected to create a data set for training deep learning models, which can ensure that the system can accurately identify and monitor foreign objects in dangerous areas.

[0066] In addition, to enhance the diversity of the dataset, environmental factors need to be considered when collecting samples. Lighting conditions are an important factor. We will collect samples under different lighting conditions, including natural lighting during the day and working environments at night. We will adjust the light intensity of artificial light sources to simulate changes in lighting at different times and collect images under various lighting conditions. Secondly, mechanical interference is also a factor that cannot be ignored. We use machine tool vibration simulators and shock generators to simulate the vibration and shock that may occur in machine tools during actual work, and collect images when interference exists.

[0067] The advantages of the protection system of the present invention are: 1. By setting the range of the dangerous area (including the front position, rear position of the upper and lower cutting dies and the space between the upper and lower cutting dies), designing a reasonable camera layout plan, and adjusting the camera to a suitable angle, all-round monitoring of the dangerous area is achieved, which improves the adaptability to different machine tools and equipment, avoids the appearance of blind spots in the field of vision, and thus improves the safety of the machine tool; 2. By optimizing the deep learning model, adding an attention mechanism and an improved pyramid pooling module after the model backbone network, the detection speed and recognition accuracy are improved, and the model can be intelligently analyzed for complex scenes, and accurately detect the abnormal situation of small foreign objects entering the dangerous area; 3. The relay control module can receive signals to realize the emergency stop and reset of the machine tool, achieve rapid response, improve the automation of the machine tool, and reduce the safety hazards caused by operator misoperation. In short, when the machine tool processes the plate, the protection system of the present invention monitors its working area with high precision and in all directions, and can quickly respond to form an efficient protection plan when foreign objects appear in the working area, thereby improving the safety and production efficiency during the machine tool processing process.

Claims

1. A protection system for a sheet metal forming machine tool based on machine vision, characterized by: It comprises an image acquisition component (1), a mounting bracket component (2), an image processing module (3), and a relay control module (4); the mounting bracket component (2) is fixedly connected to a side wall panel (6) of a machine tool, the image acquisition component (1) is mounted on the bracket component (2), the image processing module (3) is electrically connected to the image acquisition component (1), and one end of the relay control module (4) is electrically connected to the image processing module (3) and the other end is electrically connected to an electric control box (5) of the machine tool.

2. The protection system according to claim 1, characterized in that: The image acquisition component (1) comprises a plurality of image acquisition units, each of which comprises a camera (1.1) and an image transmission cable (1.2); the mounting bracket component (2) comprises left and right L-shaped side brackets (2.1), a crossbar (2.2), and left and right L-shaped top brackets (2.3); one end of the L-shaped side bracket (2.1) is fixedly connected to the outer side surface of a side wall panel (6) of a machine tool, and the other end is connected to the camera (1.1); the L-shaped top bracket (2.3) is fixedly connected to the upper surface of the side wall panel (6) of the machine tool, and the two ends of the crossbar (2.2) are fixedly connected to the two left and right L-shaped top brackets (2.3); a plurality of cameras (1.1) are mounted on the crossbar (2.4) at equal intervals; and the camera (1.1) is connected to an image processing module (3) via an image transmission cable (1.2).

3. The protection system according to claim 2, characterized in that: Each image acquisition unit also includes a positioning plate (1.3) and a positioning lock (1.4); the positioning plate (1.3) is connected to the back of the camera (1.1); the positioning plate (1.3) is provided with a positioning lock (1.4); and the positioning lock (1.4) is connected to an L-shaped side bracket (2.1) or a crossbar (2.2).

4. The protection system according to claim 1, characterized in that: The image processing module (3) comprises a camera serial interface (3.1), an embedded computing unit (3.2), and input / output pins (3.3); the relay control module (4) comprises a DuPont line (4.1), a relay pin (4.2), a relay internal circuit (4.3), a relay contact (4.4), and a control cable (4.5); the camera serial interface (3.1) is connected to the image transmission cable (1.2); two ends of the DuPont line (4.1) are respectively connected to the input / output pins (3.3) and the relay pins (4.2); and two ends of the control cable (4.5) are respectively connected to the relay contact (4.4) and the control switch (5) of the machine tool.

5. A protection method for a sheet metal forming processing machine tool based on machine vision, the steps of which are: ① multiple image acquisition units collect image data of the working area in real time; ② the image processing module processes the image through an internal optimized deep learning model to determine whether there is a foreign object; if there is a foreign object, the input and output pins output a high level, and if there is no foreign object, a low level is output; ③ the relay control module receives high and low level signals to control the shutdown or normal operation of the machine tool: if it is a high level, the relay coil is energized, the normally closed contact is disconnected, the main circuit of the machine tool is disconnected and stops working, the foreign object is manually removed, the machine tool is reset, the protection system is initialized, the output signal becomes an initial low level, and the system judges the next frame of the working area image; if it is a low level, the relay coil is de-energized, the normally closed contact is closed, and the main circuit of the machine tool is connected to work normally.

6. The protection method according to claim 5, characterized in that: The method for determining the number and specific positions of the image acquisition units in step ① is as follows: Ⅰ. Determine the detection range according to the dangerous area during machine tool processing: For a machine tool with an upper die length of L and a lower die thickness of W, the dangerous area is divided into the following parts: the front end area of ​​the upper die blade with a length of L and a width of 200mm; the rear end area of ​​the upper die blade with a length of L and a width of 200mm; the two left and right sides of the blade side areas with a length of 50mm and a width of W+400mm; the lower die area with a length of L and a width of W; the above areas form a rectangular area with a length of L+100mm and a width of W+400mm; Ⅱ. Determine the number of cameras with adjustable angles: the length of the detection area covered by the field of view of a single camera is D, and the width is W+400mm. The number of cameras is the value of (L+100) / D rounded up; III. Arrange cameras horizontally above the machine tool and on both sides of the machine tool so that the camera's field of view covers a rectangular area with a length of L+100mm and a width of W+400mm.

7. The protection method according to claim 5 is characterized in that: in step ①: the camera captures the video image, and uses a high-speed interface and an appropriate data transmission protocol to transmit the image to the image processing module; to ensure that each camera device collects images at the same time point, the system adopts a software clock synchronization mechanism and sets a unified frame rate so that each camera collects images at the same speed to avoid data inconsistency or information loss; in image transmission, a suitable bandwidth is set according to the number of camera devices so that the camera data can be transmitted smoothly. In order to cope with the synchronous reception of multiple data images, the embedded device adopts multi-threaded processing technology, that is, each thread receives the data stream from a specific camera, thereby improving the overall data reception efficiency.

8. The protection method according to claim 5, characterized in that: The image processing flow of the optimized deep learning model in step ② is as follows: Ⅰ Preprocessing: Under the premise of maintaining the image aspect ratio, scale the image to a specific size, normalize the image pixel value from 0-255 to the range of 0-1, randomly transform the image, and increase data diversity through data enhancement methods such as rotation, translation, scaling, and flipping to improve the generalization ability of the model; Ⅱ Feature extraction: Use the convolution kernel to slide on the image, and extract the basic features of the image through weighted summation (including low-level features such as image edges and corners); then use the activation function (RELU function) to process these features, changing negative values ​​to 0 and keeping positive values ​​unchanged, making the important details in the image more obvious; Finally, a pooling operation is performed to simplify the image. By extracting the maximum and average values ​​of the local area, the amount of data is reduced and the main features are retained; III. Target detection: The processed image is divided into multiple grids, each of which is responsible for detecting the target within its range. In each grid, the model generates multiple bounding boxes surrounding the object based on the size and shape of the target object in the training data set, and calculates the confidence of each bounding box and the probability of its category, generating multiple possible prediction results; IV. Generate prediction results: The image features extracted by the feature are flattened into a one-dimensional vector in sequence, and input into the fully connected layer, and weighted summation and activation are performed. The activation function further processes the features and extracts more complex feature representations, such as the combination of shape and color features; the output layer receives the feature representation of the fully connected layer, uses the classification task activation function to convert the feature value into a probability distribution, and selects the category with the highest probability as the final prediction result; V post-processing: adjust the bounding box to match the actual target, remove overlapping prediction boxes through non-maximum suppression (NMS), and only retain the box with the highest confidence; sum the probabilities defined as foreign body categories to obtain the total probability of foreign body, set a threshold, if the probability is greater than the threshold, it is judged that there is a foreign body, if it is less than the threshold, it is judged that there is no foreign body.

9. The protection method according to claim 5, characterized in that: The method for optimizing the deep learning model in step ② is as follows: the divided data set is imported into the model for training through the deep learning framework to obtain a preliminary model; after integrating the compression and excitation (SE) attention mechanism into the backbone network, a dangerous area focus module is obtained; A multi-size target capture unit is added after the dangerous area focus module to obtain a dangerous object recognition module; the dangerous object recognition module classifies and identifies the captured target area, determines whether the target is a dangerous object, and outputs the category label and confidence level; based on the output of the dangerous object recognition module, the hazard assessment output layer further comprehensively analyzes the recognition results, evaluates the risk level of the detected target in the current scene, and finally outputs a control signal to guide the operation logic of the machine tool.

10. The protection method according to claim 5, characterized in that: The method for constructing the preliminary model of the deep learning model in step ② is as follows: Ⅰ. When the machine tool is not working, simulate the appearance of plates and various foreign objects in dangerous areas under different scenarios, and collect these images to produce a data set; Ⅱ. After completing the collection of data set samples, use image annotation tools to classify and annotate the processed plates and various foreign objects, and randomly divide them into training set, validation set and test set in a ratio of 7:1:2; Ⅲ. Import the divided data set into the built-in model of the image processing module through the deep learning framework for training, and obtain the preliminary model of the deep learning model.

Images