A laser cutting machine visual self-adaptive anti-collision method and storage medium

Abstract

The application provides a laser cutting machine visual self-adaptive anti-collision method and a storage medium, belongs to the technical field of visual detection of the front platform of a laser cutting machine, and the method is characterized in that: a visual device is installed in the interior of the protective shell of the laser cutting machine, and the installation position and the shooting angle are calibrated; video stream data of a target monitoring area collected by the visual device in real time are acquired and processed, and the existence, position and motion state information of a human target in the target monitoring area are output; the current motion state information of the cross beam of the laser cutting machine is acquired in real time; the motion state information of the human target is compared with the motion state information of the cross beam of the laser cutting machine, and a risk assessment of whether a collision occurs is performed in combination with a preset decision rule; a control instruction is generated according to the risk assessment result, and is sent to the laser cutting machine control system. The application dynamically assesses the collision risk by monitoring the motion state of the human target and the cross beam, and improves the safety of human-machine cooperation.

Description

Technical Field

[0001] This application belongs to the field of vision inspection technology for the front platform of laser cutting machines, specifically relating to a vision-adaptive anti-collision method and storage medium for laser cutting machines. Background Technology

[0002] Laser cutting, as a high-precision and high-efficiency processing method, is increasingly widely used in manufacturing. With the development of laser cutting, the cutting speed and idle speed of laser cutting machines are constantly increasing, with some machines reaching idle speeds of over 200 m / min. While improving production efficiency, this also brings safety hazards. The working area of ​​a laser cutting machine is not completely enclosed; operators sometimes need to approach or enter this area for loading, unloading, and inspection, posing a risk of collision with the high-speed moving laser head beam.

[0003] Currently, common safety protection methods include physical safety fences, light curtains, and emergency stop buttons. However, physical fences affect operational convenience; light curtains provide fixed protection areas, which may have blind spots on large workbenches or pose a risk of protection failure due to equipment structure; emergency stop buttons rely on personnel's active reaction, and in emergency situations, they may not be able to prevent accidents due to insufficient reaction time. Another protection method is sensor-based collision avoidance systems, but these systems are easily affected by the complex environment of the workshop and have insufficient reliability.

[0004] In recent years, computer vision has provided a new approach to solving such problems. By monitoring the work area in real time through cameras and using target detection and tracking algorithms to identify personnel positions, collision avoidance can be achieved to some extent. For example, some solutions are based on the YOLO algorithm for personnel detection and the Deepsort algorithm for tracking. However, these solutions usually directly apply general computer vision models without fully considering the special characteristics of high-speed laser cutting machine scenarios. First, the false alarm rate of general algorithms may be high in complex industrial environments, and frequent accidental shutdowns will seriously affect production efficiency. Second, they fail to make decisions deeply coupled with the machine's own motion state, making it impossible to accurately judge the actual collision risk. Third, there is a lack of dedicated visual coverage solutions for the structure of the laser cutting machine's work area (front door, side door), which may lead to blind spots in monitoring.

[0005] Therefore, there is an urgent need for a visual adaptive collision avoidance method that can adapt to the high-speed working environment of laser cutting machines, has a low false alarm rate, and can intelligently link with the movement status of the equipment. Summary of the Invention

[0006] In a first aspect, embodiments of this application provide a vision-adaptive collision avoidance method for laser cutting machines, comprising the following steps: S1. Install a vision device inside the protective housing of the laser cutting machine, and calibrate the installation position and shooting angle of the vision device based on the target monitoring area; S2. Acquire video stream data of the target monitoring area collected in real time by the vision device, process the video stream data, and output information on the existence, location and motion status of human targets within the target monitoring area; S3. Real-time acquisition of the current motion status information of the laser cutting machine beam; S4. Compare the motion state information of the human target with the motion state information of the laser cutting machine beam, and conduct a risk assessment of whether a collision will occur based on the preset decision rules. S5. Generate control instructions based on the risk assessment results and send them to the laser cutting machine control system; the control instructions include at least an operation permission instruction and an emergency stop instruction.

[0007] Furthermore, the specific steps of step S1 are as follows: S11. Based on the movement direction of the laser cutting machine table, install the vision device on the left sheet metal inside the protective shell of the laser cutting machine. The vision device includes a camera, an infrared light source, and a diffuser plate. S12. Using the front sheet metal inside the guide rail and protective housing of the laser cutting machine as a reference, adjust the installation height of the vision device using a calibration tool; and using the field of view covering the front and right doors of the laser cutting machine as a reference, adjust the shooting angle of the vision device using a calibration tool. S13. Adjust the illumination angle of the infrared light source so that the optical axis of the infrared light source deflects by a preset angle away from the baseline perpendicular to the table surface, and the light is evenly diffused by the diffuser plate.

[0008] Furthermore, the specific steps of step S12 are as follows: S121. Obtain the dimensions of the laser cutting machine's side door, camera parameters, and installation area; S122. Calculate the minimum installation height, maximum installation height, and recommended installation angle of the camera; S123. Load the camera's optical parameters using a calibration tool, and input the laser cutting machine's side door dimensions, camera parameters, and installation area range to calculate the theoretically recommended installation parameters required to fully cover the target monitoring area; the theoretically recommended installation parameters include the camera's minimum installation height, maximum installation height, and recommended installation angle; S124. Obtain the virtual calibration box that is manually added to represent the expected monitoring range, and compare the virtual positioning box with the ideal field of view corresponding to the theoretically recommended installation parameters, and provide a prompt on the compliance of the installation. S125. Obtain the actual installation angle of the camera through image algorithms or sensors, and compare the actual installation angle with the theoretically recommended installation angle. If the deviation between the two exceeds the preset tolerance, prompt the camera angle to be adjusted.

[0009] Furthermore, the specific steps of step S2 are as follows: S21. Preprocessing the image frames in the video stream data, the preprocessing including at least converting the RGB image to a grayscale image and applying Gaussian filtering to the grayscale image to eliminate noise; S22. Input the preprocessed image frames into the human detection model based on the YOLO algorithm, identify and locate human targets in each image frame, and output the bounding box information and category confidence of each human target; the bounding box information is the position coordinates of the bounding box region; S23. Extract key point information of human skeleton from the bounding box region based on the OpenPose algorithm; S24. Based on the detection results of the current image frame and historical image frames, the DeepSORT multi-target tracking algorithm is used to establish or update the motion trajectory for each detected human target, and predict the motion state information within a set time period in the future; the motion state information includes at least the predicted position, predicted speed and predicted motion direction.

[0010] Furthermore, the specific steps of step S24 are as follows: S241. Maintain a Kalman filter for each tracked human target to predict the corresponding position in the next image frame based on the motion state information of the tracked human target; S242. Calculate the correlation between the bounding box of the detected human target in the current image frame and all predicted locations, wherein the correlation is calculated by fusing Mahalanobis distance and minimum cosine distance; S243. Using the Hungarian algorithm, with the correlation degree as the cost matrix, perform optimal matching between the detection results of new human targets and existing tracking trajectories, update the state of the successfully matched existing tracking trajectories, initialize new trajectories for the detection results of unmatched new human targets, and mark the unmatched existing tracking trajectories as temporarily lost.

[0011] Furthermore, in step S22, a combined recognition strategy is used to identify and locate human targets in each image frame: First, identify the human leg region in the image frame. If the human leg region is continuously detected within a set spatial range for N consecutive image frames, then the set spatial range is marked as the first spatial region. Then, enhanced recognition of the torso and head regions is performed on the corresponding area above the first spatial region to achieve comprehensive judgment of human targets and reduce false recognition.

[0012] Furthermore, in step S3, the current motion status information of the laser cutting machine beam is read in real time by the machine tool CNC system or obtained in real time by the encoder sensor; The current motion status information includes the real-time position, real-time speed, and real-time direction of motion of the laser cutting machine beam.

[0013] Furthermore, the specific steps of step S4 are as follows: S41. Receive the predicted speed and predicted direction of motion of the human target, as well as the real-time speed and real-time direction of motion of the laser cutting machine beam; S42. Conduct a risk assessment based on the following pre-defined decision-making rules: If the speed of the beam movement is less than the set ratio of the human body movement speed, and the beam and the human body move in the same direction, it is assessed as a low-risk level. If the speed of the beam movement is greater than the speed of the human body movement, and the beam and the human body move in the same direction, it is assessed as a high-risk level. If the speed of the beam movement is greater than the speed of the human body movement, and the beam and the human body move in opposite directions, the risk level is assessed as low. If the speed of the beam movement is less than the speed of the human body movement, and the beam and the human body move in opposite directions, the risk level is assessed as low. S43. Output decision signals corresponding to the risk level.

[0014] Furthermore, the specific steps of step S5 are as follows: S51. Identify the risk level corresponding to decision signals; If the risk level is low, a command to allow operation is generated, and the audible and visual alarm is triggered to issue a level one alarm. If the risk level is high, an emergency shutdown command will be generated. S52. Convert the allow run command or emergency stop command into a control signal protocol that can be recognized by the CNC system of the laser cutting machine, and send it through the industrial communication interface to control the laser cutting machine to perform the corresponding actions.

[0015] Secondly, embodiments of this application also provide a storage medium storing a computer program thereon, which, when executed by a processor, implements the steps of the laser cutting machine vision adaptive anti-collision method as described in the first aspect.

[0016] As can be seen from the above technical solutions, this application has the following advantages: The laser cutting machine vision-adaptive anti-collision method and medium provided in this application, by precisely installing a vision device inside the protective shell, combined with real-time video stream processing and human target detection and tracking, can effectively monitor the operator's position and movement status, and deeply couple with the movement status of the laser cutting machine's crossbeam. Based on preset decision rules, it intelligently assesses collision risks and promptly generates and sends corresponding control commands to the control system. This improves the safety of the laser cutting machine's working area, reduces the risk of collisions between operators and the high-speed moving laser head crossbeam, and reduces frequent shutdowns due to false alarms, ensuring production efficiency. Through deep learning algorithms and image processing, it has good adaptability to complex industrial environments, achieving comprehensive monitoring of key areas such as the front and side doors of the laser cutting machine, effectively avoiding monitoring blind spots, and providing strong technical support for the efficient and safe operation of the laser cutting machine. Attached Figure Description

[0017] To more clearly illustrate the technical solution of this application, the accompanying drawings used in the description will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0018] Figure 1 This is a schematic flowchart of the laser cutting machine vision adaptive collision avoidance method of the present invention.

[0019] Figure 2 This is a flowchart illustrating the installation and calibration steps of the vision device of the present invention.

[0020] Figure 3 This is a flowchart illustrating the video stream data acquisition and human state perception steps of the present invention.

[0021] Figure 4 This is a flowchart illustrating the risk decision-making steps for collisions between a human body and the crossbeam of a laser cutting machine according to the present invention.

[0022] Figure 5 This is a flowchart illustrating the control and execution steps of the laser cutting machine according to the present invention. Detailed Implementation

[0023] The various embodiments of this disclosure will be described more fully in the following detailed description of the specific steps of the vision-adaptive collision avoidance method for laser cutting machines. This disclosure may have various embodiments, and adjustments and changes may be made therein. However, it should be understood that there is no intention to limit the various embodiments of this disclosure to the specific embodiments disclosed herein, but rather this disclosure should be understood to cover all adjustments, equivalents, and / or alternatives falling within the spirit and scope of the various embodiments of this disclosure.

[0024] For example, common protective methods for laser cutting machines operating at high speeds have the following drawbacks: First, physical barriers limit operational flexibility and do not meet the needs of human-machine collaboration; second, grating protection has dead zones and structural limitations, making it difficult to adapt to complex work areas; third, emergency stop buttons rely entirely on human reaction and cannot proactively predict risks; fourth, sensor-based anti-collision systems are greatly affected by workshop environment interference, have insufficient reliability, and are difficult to adapt to high-speed laser cutting machines; fifth, anti-collision methods based on general vision have a high false alarm rate, do not take into account the equipment's motion status, cannot accurately judge collision risks, and have monitoring blind spots.

[0025] To address the aforementioned issues, this embodiment provides a vision-adaptive anti-collision method for laser cutting machines. By using a vision device to monitor human targets in real time and combining this with the movement status of the crossbeam, the method dynamically assesses collision risks, achieving safety protection while reducing false alarms and ensuring production efficiency. This method is suitable for complex industrial environments.

[0026] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0027] Please see Figure 1 The diagram shows a flowchart of a vision-adaptive anti-collision method for a laser cutting machine in a specific embodiment. The method includes the following steps: S1. Install a vision device inside the protective housing of the laser cutting machine, and calibrate the installation position and shooting angle of the vision device based on the target monitoring area; It should be noted that this step ensures that the vision device accurately covers the key working areas of the laser cutting machine, such as the front door and side door, providing comprehensive and clear video stream data for subsequent human target detection and tracking. This is the foundation for building the entire vision adaptive collision avoidance system, effectively avoiding blind spots in monitoring and improving the accuracy and reliability of monitoring. S2. Acquire video stream data of the target monitoring area collected in real time by the vision device, process the video stream data, and output information on the existence, location and motion status of human targets within the target monitoring area; It should be noted that this step is the core link in realizing the anti-collision function. Through real-time processing and analysis of video stream data, the position and movement trend of the operator can be accurately identified, providing a key basis for subsequent comparison with the movement state of the laser cutting machine beam and risk assessment. This enables the anti-collision system to detect potential dangers in a timely manner and ensure the safety of the operator. S3. Real-time acquisition of the current motion status information of the laser cutting machine beam; It should be noted that this step achieves deep coupling between the human target's motion state and the equipment's own motion state, enabling the collision avoidance system to accurately judge the collision risk based on the actual situation, avoiding misjudgments caused by monitoring only the human target or the crossbeam's motion state, and effectively improving the rationality and accuracy of risk assessment. S4. Compare the motion state information of the human target with the motion state information of the laser cutting machine beam, and conduct a risk assessment of whether a collision will occur based on the preset decision rules. It should be noted that this step is a key decision-making step in the entire collision avoidance system. By comparing and analyzing key parameters such as speed and direction of motion, the risk level can be reasonably and accurately classified, providing a clear basis for generating corresponding control commands. This enables intelligent control of the laser cutting machine's operating status, effectively avoiding collision accidents, reducing unnecessary downtime, and ensuring production efficiency. S5. Generate control instructions based on the risk assessment results and send them to the laser cutting machine control system; the control instructions include at least an operation permission instruction and an emergency stop instruction. It should be noted that this step is the final execution step for the anti-collision system to realize its protective function. It can control the operating status of the laser cutting machine in a timely and accurate manner to ensure that measures are taken quickly to protect the safety of operators in dangerous situations, while maintaining the normal operation of the equipment in low-risk situations. This reduces production interruptions caused by false alarms and effectively improves the production efficiency and safety of the laser cutting machine.

[0028] This embodiment, through the overall process of visual device installation, video stream data processing, human target information output, and then to beam motion status monitoring, risk assessment, and control command transmission, enables the laser cutting machine to maintain efficient production operations while ensuring operator safety. It is highly adaptable to complex industrial environments and solves the limitations of existing protection methods.

[0029] Furthermore, as a refinement and extension of the specific implementation of the above embodiments, in order to fully illustrate the specific implementation process in this embodiment, another vision-adaptive anti-collision method for laser cutting machines is provided, which includes the following steps: S1. Install a vision device inside the protective casing of the laser cutting machine, and calibrate the installation position and shooting angle of the vision device based on the target monitoring area; such as Figure 2 As shown, the specific steps of step S1 are as follows: S11. Based on the movement direction of the laser cutting machine table, install the vision device on the left sheet metal inside the protective shell of the laser cutting machine. The vision device includes a camera, an infrared light source, and a diffuser plate. S12. Using the front sheet metal inside the guide rail and protective housing of the laser cutting machine as a reference, adjust the installation height of the vision device using a calibration tool; and using the field of view covering the front and right doors of the laser cutting machine as a reference, adjust the shooting angle of the vision device using a calibration tool. For example, the initial shooting angle of the vision device is set to tilt 15 degrees to the right in the positive direction of the machine tool movement; The specific steps of step S12 are as follows: S121. Obtain the dimensions of the laser cutting machine's side door, camera parameters, and installation area; S122. Calculate the minimum installation height, maximum installation height, and recommended installation angle of the camera; S123. Load the camera's optical parameters using a calibration tool, and input the laser cutting machine's side door dimensions, camera parameters, and installation area range to calculate the theoretically recommended installation parameters required to fully cover the target monitoring area; the theoretically recommended installation parameters include the camera's minimum installation height, maximum installation height, and recommended installation angle; S124. Obtain the virtual calibration box that is manually added to represent the expected monitoring range, and compare the virtual positioning box with the ideal field of view corresponding to the theoretically recommended installation parameters, and provide a prompt on the compliance of the installation. S125. Obtain the actual installation angle of the camera through image algorithms or sensors, and compare the actual installation angle with the theoretically recommended installation angle. If the deviation between the two exceeds the preset tolerance, prompt the camera angle to be adjusted. S13. Adjust the illumination angle of the infrared light source so that the optical axis of the infrared light source deflects by a preset angle away from the baseline perpendicular to the table surface, and the light is evenly diffused by the diffuser plate to eliminate imaging dead angles and reduce interference caused by uneven illumination. S2. Acquire video stream data of the target monitoring area in real time collected by the vision device, process the video stream data, and output information on the presence, location, and motion status of human targets within the target monitoring area; such as... Figure 3 As shown, the specific steps of step S2 are as follows: S21. Preprocessing the image frames in the video stream data, the preprocessing including at least converting the RGB image to a grayscale image and applying Gaussian filtering to the grayscale image to eliminate noise; The RGB format image frames captured by the vision device are converted to grayscale images using the following weighting method. :

[0030] in, Represents the grayscale value of an image frame, where R, G, and B are the red, green, and blue channel values ​​of pixels in an RGB format image frame, respectively. Use Gaussian filtering to eliminate noise in the image frames:

[0031] in, This represents the pixel value at coordinates (x+i, y+j) in a grayscale image frame. This represents the grayscale value before filtering; represents the filtered grayscale value; a and b represent the radius parameters of the convolution kernel, controlling the filtering range; G(i,j) is the Gaussian weight function;

[0032] in, i ∈[− a , a ], j ∈[− b , b ]; S22. Input the preprocessed image frames into the human detection model based on the YOLO algorithm, identify and locate human targets in each image frame, and output the bounding box information and category confidence of each human target; the bounding box information is the position coordinates of the bounding box region; S23. Extract key point information of human skeleton from the bounding box region based on the OpenPose algorithm; S24. Based on the detection results of the current image frame and historical image frames, the DeepSORT multi-target tracking algorithm is used to establish or update the motion trajectory for each detected human target, and predict the motion state information within a set time period in the future; the motion state information includes at least the predicted position, predicted speed, and predicted motion direction; the specific steps of step S24 are as follows: S241. Maintain a Kalman filter for each tracked human target to predict the corresponding position in the next image frame based on the motion state information of the tracked human target; S242. Calculate the correlation between the bounding box of the detected human target in the current image frame and all predicted locations, wherein the correlation is calculated by fusing Mahalanobis distance and minimum cosine distance; S243. Using the Hungarian algorithm, with the correlation degree as the cost matrix, perform optimal matching between the detection results of new human targets and existing tracking trajectories, update the state of the successfully matched existing tracking trajectories, initialize new trajectories for the detection results of unmatched new human targets, and mark the unmatched existing tracking trajectories as temporarily lost. S3. Real-time acquisition of the current motion status information of the laser cutting machine crossbeam; In step S3, the current motion status information of the laser cutting machine crossbeam is read in real time by the machine tool CNC system or acquired in real time by the encoder sensor; The current motion status information includes the real-time position, real-time speed, and real-time direction of motion of the laser cutting machine beam; S4. Compare the motion state information of the human target with the motion state information of the laser cutting machine beam, and conduct a risk assessment of whether a collision will occur based on preset decision rules; such as Figure 4 As shown, the specific steps of step S4 are as follows: S41. Receive the predicted speed and predicted direction of motion of the human target, as well as the real-time speed and real-time direction of motion of the laser cutting machine beam; S42. Conduct a risk assessment based on the following pre-defined decision-making rules: If the speed of the beam movement is less than a set percentage of the human body's movement speed (e.g., 50%), and the beam and the human body move in the same direction, it is assessed as a low-risk level. If the speed of the beam movement is greater than the speed of the human body movement, and the beam and the human body move in the same direction, it is assessed as a high-risk level. If the speed of the beam movement is greater than the speed of the human body movement, and the beam and the human body move in opposite directions, the risk level is assessed as low. If the speed of the beam movement is less than the speed of the human body movement, and the beam and the human body move in opposite directions, the risk level is assessed as low. S43. Output decision signals corresponding to the risk level; S5. Generate control commands based on the risk assessment results and send them to the laser cutting machine control system; the control commands include at least a permission to run command and an emergency stop command; such as Figure 5 As shown, the specific steps of step S5 are as follows: S51. Identify the risk level corresponding to decision signals; If the risk level is low, a command to allow operation is generated, and the audible and visual alarm is triggered to issue a level one alarm. If the risk level is high, an emergency shutdown command will be generated. S52. Convert the allow run command or emergency stop command into a control signal protocol that can be recognized by the CNC system of the laser cutting machine, and send it through the industrial communication interface to control the laser cutting machine to perform the corresponding actions.

[0033] In one embodiment of the present invention, based on step S22, the following will provide a possible embodiment and describe its specific implementation in a non-limiting manner.

[0034] In step S22, a combined recognition strategy is used to identify and locate human targets in each image frame: First, identify the human leg region in the image frame. If the human leg region is continuously detected within a set spatial range for N consecutive image frames, then the set spatial range is marked as the first spatial region. Then, enhanced recognition of the torso and head regions is performed on the corresponding area above the first spatial region to achieve comprehensive judgment of human targets and reduce false recognition; It should be noted that analysis of a large number of collected samples revealed that, apart from the limbs, the posture of other parts of the human body is relatively fixed. Based on this, the detection algorithm can be optimized through combined recognition. Firstly, the leg region can be identified, and then the corresponding area above the legs can be used to identify the shoulder to head region, thereby reducing the false recognition rate. Since leg posture changes due to walking, it is difficult to identify using readily observable features. Therefore, SVM can be used to identify the legs, thus reducing the false recognition rate.

[0035] In one embodiment of the present invention, based on step S24, a possible embodiment will be given below, and its specific implementation will be described in a non-limiting manner.

[0036] The DeepSORT multi-target tracking algorithm described in step S24 is implemented by including the following steps: S2401. Detection stage: Use the YOLO-based object detection algorithm to analyze each input image frame, identify objects belonging to the human body category, and output the classification confidence and bounding box coordinates of the human body category objects; S2402. Feature Extraction and Motion Prediction Stage: A convolutional neural network is used to extract the apparent features of the target within the bounding box. At the same time, a Kalman filter is maintained for each tracked target to predict its position and velocity state in the next frame based on its motion history. Taking the detection of a human body in a laser cutting machine as an example: Kalman filter prediction equation:

[0037] in, It is the human body vector state. It is the location of the human body. It is the speed of the human body;

[0038] in, It is the acceleration of human movement. It is the combined force acting on the human body. It refers to body mass; Next position:

[0039] in, It is the time difference between adjacent moments; Convert the position at the next moment into a matrix representation:

[0040] in, ,

[0041] This transforms the matrix expression for the human body's position at the next moment into: ; S2403. Similarity Calculation Stage: Calculate the similarity between the current frame detection result and the existing trajectory prediction state. The similarity is calculated by fusing motion feature similarity and appearance feature similarity to obtain a fused similarity. The calculation and fusion of motion feature similarity and appearance feature similarity in step S2403 are specifically implemented in the following way: S24031. Motion feature similarity calculation: Using the tracking trajectory state predicted by the Kalman filter, calculate the Mahalanobis distance between the existing tracking trajectory and the new human target in the detection box of the current image frame, and use the Mahalanobis distance as the similarity measure of motion features; and preset a Mahalanobis distance threshold. If the Mahalanobis distance exceeds the Mahalanobis distance threshold, the possibility of association between the currently detected human target and the existing tracking trajectory is directly excluded. The distance similarity of the target position dimension is calculated by using the Mahalanobis distance between the detection box in the current frame and the tracking box in the previous frame, thereby realizing data association matching for motion features. Since Mahalanobis distance can measure the distance between sample data and is not affected by the correlation between size or variables, this application uses the similarity between the detection box and the tracking box in the beam sample by Mahalanobis distance.

[0042] Where i represents the sequence number of the tracking box and j represents the sequence number of the detection box; This represents the Mahalanobis distance between the two. Let be the covariance matrix of the i-th Kalman filter, representing the fault tolerance of the target position; Let be the observation vector of the j-th detection box; This is the state-corresponding vector for the i-th tracking box; The smaller the value, the higher the similarity of the motion features between the tracking box and the detection box, and the greater the matching correlation between the two; if If the value is less than the preset threshold, it is considered that the tracking box and the detection box may correspond to the same identity, and appearance feature matching is performed. If the value is greater than the preset threshold, the next round of matching calculation is performed. S24032. Appearance feature similarity calculation: Using the trained deep feature model, extract the feature vectors of the current frame detection box and the trajectory history appearance feature library respectively, and calculate the minimum cosine distance between them. Use the minimum cosine distance as the similarity measure of appearance features. The trained model is used to extract features from the j-th detection box and the ith tracking box to obtain the apparent feature vector. , The k-th apparent feature vector of the i-th tracking box Stored in the appearance feature library Ri, Let be the cosine similarity of the appearance features between the j-th detection box and the ith tracking box, and let be the minimum cosine distance between the appearance features. The sum is 1; where, The smaller the value, the higher the similarity in appearance features between the tracking box and the detection box, and the greater the correlation and matching degree between the two.

[0043] S24033. Similarity Fusion: The Mahalanobis distance and the minimum cosine distance are linearly weighted and fused using a hyperparameter λ to obtain a comprehensive metric, as shown in the formula:

[0044] in, For the overall cost, The Mahalanobis distance, Let λ be the cosine distance, and λ be a hyperparameter. S24034. Use the fused comprehensive metric C as the cost matrix for Hungarian algorithm matching to complete the data association; S2404. Data Association Stage: Using the fusion similarity calculated in step S2403 as the cost matrix, the Hungarian algorithm is used to optimally associate the human target detection results with the existing tracking trajectories, and the same trajectory ID is assigned to the successfully associated human targets.

[0045] It should be understood that the sequence number of each step in the above embodiments does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present invention.

[0046] The storage medium provided in this application stores a program product capable of implementing a vision-adaptive anti-collision method for laser cutting machines.

[0047] The vision-adaptive anti-collision method for laser cutting machines includes: installing a vision device inside the protective shell of the laser cutting machine, and calibrating the installation position and shooting angle of the vision device based on the target monitoring area; acquiring video stream data of the target monitoring area collected in real time by the vision device, processing the video stream data, and outputting information on the presence, position, and motion status of human targets within the target monitoring area; acquiring the current motion status information of the laser cutting machine beam in real time; comparing the motion status information of the human targets with the motion status information of the laser cutting machine beam, and conducting a risk assessment of whether a collision will occur based on preset decision rules; generating control commands based on the risk assessment results and sending them to the laser cutting machine control system; the control commands include at least a permission to run command and an emergency stop command.

[0048] In some possible implementations, the laser cutting machine vision adaptive collision avoidance method of this disclosure can be implemented as a program product including program code that, when the program product is run on a terminal device, causes the terminal device to perform the steps described in the "Exemplary Methods" section above according to various exemplary embodiments of this disclosure.

[0049] The storage medium disclosed herein may be any combination of one or more readable media. A readable medium may be a readable signal medium or a readable storage medium. A readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples (a non-exhaustive list) of readable storage media include: an electrical connection having one or more wires, a portable disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination thereof.

[0050] The above description of the disclosed embodiments enables those skilled in the art to make or use the invention. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the invention. Therefore, the invention is not to be limited to the embodiments shown herein, but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A vision-adaptive collision avoidance method for laser cutting machines, characterized in that, Includes the following steps: S1. Install a vision device inside the protective housing of the laser cutting machine, and calibrate the installation position and shooting angle of the vision device based on the target monitoring area; S2. Acquire video stream data of the target monitoring area collected in real time by the vision device, process the video stream data, and output information on the existence, location and motion status of human targets within the target monitoring area; The specific steps of step S2 are as follows: S21. Preprocessing the image frames in the video stream data, the preprocessing including at least converting the RGB image to a grayscale image and applying Gaussian filtering to the grayscale image to eliminate noise; S22. Input the preprocessed image frames into the human detection model based on the YOLO algorithm, identify and locate human targets in each image frame, and output the bounding box information and category confidence of each human target; the bounding box information is the position coordinates of the bounding box region; S23. Extract key point information of human skeleton from the bounding box region based on the OpenPose algorithm; S24. Based on the detection results of the current image frame and historical image frames, the DeepSORT multi-target tracking algorithm is used to establish or update the motion trajectory for each detected human target, and predict the motion state information within a set time period in the future; the motion state information includes at least the predicted position, predicted speed and predicted motion direction; The specific steps of step S24 are as follows: S241. Maintain a Kalman filter for each tracked human target to predict the corresponding position in the next image frame based on the motion state information of the tracked human target; S242. Calculate the correlation between the bounding box of the detected human target in the current image frame and all predicted locations, wherein the correlation is calculated by fusing Mahalanobis distance and minimum cosine distance; S243. Using the Hungarian algorithm, with the correlation degree as the cost matrix, perform optimal matching between the detection results of new human targets and existing tracking trajectories, update the state of the successfully matched existing tracking trajectories, initialize new trajectories for the detection results of unmatched new human targets, and mark the unmatched existing tracking trajectories as temporarily lost. S3. Real-time acquisition of the current motion status information of the laser cutting machine beam; S4. Compare the motion state information of the human target with the motion state information of the laser cutting machine beam, and conduct a risk assessment of whether a collision will occur based on the preset decision rules. S5. Generate control instructions based on the risk assessment results and send them to the laser cutting machine control system; the control instructions include at least an operation permission instruction and an emergency stop instruction.

2. The vision-adaptive anti-collision method for laser cutting machines according to claim 1, characterized in that, The specific steps of step S1 are as follows: S11. Based on the movement direction of the laser cutting machine table, install the vision device on the left sheet metal inside the protective shell of the laser cutting machine. The vision device includes a camera, an infrared light source, and a diffuser plate. S12. Using the front sheet metal inside the guide rail and protective housing of the laser cutting machine as a reference, adjust the installation height of the vision device using a calibration tool; and using the field of view covering the front and right doors of the laser cutting machine as a reference, adjust the shooting angle of the vision device using a calibration tool. S13. Adjust the illumination angle of the infrared light source so that the optical axis of the infrared light source deflects by a preset angle away from the baseline perpendicular to the table surface, and the light is evenly diffused by the diffuser plate.

3. The vision-adaptive anti-collision method for laser cutting machines according to claim 1, characterized in that, The specific steps of step S12 are as follows: S121. Obtain the dimensions of the laser cutting machine's side door, camera parameters, and installation area; S122. Calculate the minimum installation height, maximum installation height, and recommended installation angle of the camera; S123. Load the camera's optical parameters using a calibration tool, and input the laser cutting machine's side door dimensions, camera parameters, and installation area range to calculate the theoretically recommended installation parameters required to fully cover the target monitoring area; the theoretically recommended installation parameters include the camera's minimum installation height, maximum installation height, and recommended installation angle; S124. Obtain the virtual calibration box that is manually added to represent the expected monitoring range, and compare the virtual calibration box with the ideal field of view corresponding to the theoretically recommended installation parameters to provide a prompt on the compliance of the installation. S125. Obtain the actual installation angle of the camera through image algorithms or sensors, and compare the actual installation angle with the theoretically recommended installation angle. If the deviation between the two exceeds the preset tolerance, prompt the camera angle to be adjusted.

4. The vision-adaptive anti-collision method for laser cutting machines according to claim 1, characterized in that, In step S22, a combined recognition strategy is used to identify and locate human targets in each image frame: First, identify the human leg region in the image frame. If the human leg region is continuously detected within a set spatial range for N consecutive image frames, then the set spatial range is marked as the first spatial region. Then, enhanced recognition of the torso and head regions is performed on the corresponding area above the first spatial region to achieve comprehensive judgment of human targets and reduce false recognition.

5. The vision-adaptive anti-collision method for laser cutting machines according to claim 1, characterized in that, In step S3, the current motion status information of the laser cutting machine beam is read in real time by the machine tool CNC system or obtained in real time by the encoder sensor. The current motion status information includes the real-time position, real-time speed, and real-time direction of motion of the laser cutting machine beam.

6. The vision-adaptive anti-collision method for laser cutting machines according to claim 5, characterized in that, The specific steps of step S4 are as follows: S41. Receive the predicted speed and predicted direction of motion of the human target, as well as the real-time speed and real-time direction of motion of the laser cutting machine beam; S42. Conduct a risk assessment based on the following pre-defined decision-making rules: If the speed of the beam movement is less than the set ratio of the human body movement speed, and the beam and the human body move in the same direction, it is assessed as a low-risk level. If the speed of the beam movement is greater than the speed of the human body movement, and the beam and the human body move in the same direction, it is assessed as a high-risk level. If the speed of the beam movement is greater than the speed of the human body movement, and the beam and the human body move in opposite directions, the risk level is assessed as low. If the speed of the beam movement is less than the speed of the human body movement, and the beam and the human body move in opposite directions, the risk level is assessed as low. S43. Output decision signals corresponding to the risk level.

7. The vision-adaptive anti-collision method for laser cutting machines according to claim 6, characterized in that, The specific steps of step S5 are as follows: S51. Identify the risk level corresponding to decision signals; If the risk level is low, a command to allow operation is generated, and the audible and visual alarm is triggered to issue a level one alarm. If the risk level is high, an emergency shutdown command will be generated. S52. Convert the allow run command or emergency stop command into a control signal protocol that can be recognized by the CNC system of the laser cutting machine, and send it through the industrial communication interface to control the laser cutting machine to perform the corresponding actions.

8. A storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the steps of the vision-adaptive anti-collision method for laser cutting machines as described in any one of claims 1 to 7.

Images