A multi-modal fusion tool magazine collision early warning method and system

Abstract

The application provides a multi-modal fusion tool magazine collision early warning method and system, belonging to the field of intelligent data processing. The method comprises: collecting tool attribute characteristics, mechanical arm wear index, preset tool changing speed and tool changing interference path of a tool changing system in a preset future time zone, analyzing the tool changing path space overlap ratio according to the preset tool changing speed and tool changing interference path; performing tool changing collision risk prediction and outputting a predicted tool changing collision risk coefficient; correcting an initial collision early warning index set according to the predicted tool changing collision risk coefficient to obtain an adaptive collision early warning index set; obtaining real-time running data set of the tool changing system in the preset future time zone through multi-modal sensor monitoring, and mapping and judging the real-time running data set according to the adaptive collision early warning index set and tool collision early warning. The collision early warning index is self-adaptively adjusted according to the actual tool changing scene, effectively reducing misjudgment and omission, and improving the collision early warning precision and reliability.

Description

Technical Field

[0001] This invention relates to the field of intelligent data processing, and in particular to a multimodal fusion method and system for tool magazine collision early warning. Background Technology

[0002] As core equipment in modern manufacturing, the safety and reliability of CNC machine tools' automated tool changing systems are of paramount importance. During CNC machining, the tool magazine changing system frequently performs tool loading and unloading operations. Due to the complex motion trajectory of the robotic arm, the variety of tool specifications, and wear and tear from long-term use, there is a high risk of collisions during tool changing.

[0003] Currently, existing tool magazine collision warning systems primarily employ fixed threshold monitoring methods. This involves pre-setting uniform collision warning indicators, such as motion trajectory error range, positioning accuracy error range, and tool change torque error range. Sensors then monitor these parameters in real-time to determine if a collision risk exists. However, different tool change scenarios have different characteristics. For example, tool weight and size vary, robotic arm wear changes over time, tool change speed is adjusted according to machining requirements, and spatial interference may exist between different tool pick-up and drop paths. These factors significantly affect the magnitude of the collision risk. Existing technologies cannot dynamically adjust collision warning indicators based on these changes in actual tool change scenarios. This leads to overly sensitive warnings causing false positives in some scenarios and insufficient warnings causing missed positives in others, resulting in low collision warning accuracy and frequent false positives and missed positives. Summary of the Invention

[0004] This invention addresses the technical problem in existing technologies where collision warning indicators cannot be dynamically adjusted according to actual tool changing scenarios, resulting in low collision warning accuracy and frequent misjudgments and omissions. It provides a multimodal fusion tool magazine collision warning method and system to solve this problem.

[0005] The technical solution of the present invention to solve the above-mentioned technical problems is as follows:

[0006] In a first aspect, the present invention provides a multimodal fusion-based tool magazine collision early warning method, comprising: collecting tool attribute characteristics, robotic arm wear index, preset tool changing speed, and tool changing interference path of a tool changing system in a preset future time zone; analyzing the spatial overlap ratio of the tool changing path based on the preset tool changing speed and tool changing interference path; predicting tool changing collision risk based on the tool attribute characteristics, robotic arm wear index, preset tool changing speed, and tool changing path spatial overlap ratio, and outputting a predicted tool changing collision risk coefficient; correcting an initial collision early warning index set based on the predicted tool changing collision risk coefficient to obtain an adapted collision early warning index set; and acquiring a real-time operating dataset of the tool changing system in the preset future time zone through multimodal sensor monitoring, and performing mapping judgment and tool collision early warning on the real-time operating dataset based on the adapted collision early warning index set.

[0007] Secondly, this invention provides a multimodal fusion tool magazine collision early warning system, comprising: a data acquisition module, used to acquire tool attribute characteristics, robotic arm wear index, preset tool changing speed, and tool changing interference path of the tool changing system in a preset future time zone, and to analyze the spatial overlap ratio of the tool changing path based on the preset tool changing speed and tool changing interference path; a risk prediction module, used to predict the tool changing collision risk based on the tool attribute characteristics, robotic arm wear index, preset tool changing speed, and tool changing path spatial overlap ratio, and to output a predicted tool changing collision risk coefficient; an index correction module, used to correct the initial collision early warning index set according to the predicted tool changing collision risk coefficient to obtain an adapted collision early warning index set; and a real-time early warning module, used to monitor and acquire the real-time operating dataset of the tool changing system in the preset future time zone through multimodal sensors, and to perform mapping judgment and tool collision early warning on the real-time operating dataset according to the adapted collision early warning index set.

[0008] The beneficial effects of this invention are:

[0009] The system collects tool attribute characteristics, robotic arm wear index, preset tool change speed, and tool change interference path data in a preset future time zone. Based on the preset tool change speed and interference path analysis, the spatial overlap ratio of the tool change path is obtained, thus comprehensively acquiring scenario parameters affecting tool change collision risk and providing a data foundation for subsequent risk prediction. Tool change collision risk is predicted based on tool attribute characteristics, robotic arm wear index, preset tool change speed, and spatial overlap ratio of the tool change path, outputting a predicted tool change collision risk coefficient. This comprehensive analysis of multiple scenario factors quantifies and assesses the collision risk level in the current tool change scenario, providing a basis for adaptive adjustment of warning indicators. The initial collision warning indicator set is corrected based on the predicted collision risk coefficient to obtain an adapted collision warning indicator set, enabling dynamic adjustment of the collision warning indicators. This allows the warning threshold to adapt to the characteristics of different tool change scenarios, avoiding misjudgments and missed judgments caused by using fixed thresholds. By using multimodal sensors to monitor and acquire real-time operating data of the tool changing system within a preset future time zone, and mapping and judging the real-time operating data and issuing tool collision warnings based on the adapted collision warning index set, comprehensive real-time monitoring data can be obtained by fusion of multiple sensors, and accurate collision warning judgments can be made based on the obtained adapted warning indexes, thereby improving the accuracy and reliability of the warnings.

[0010] The above technical solution achieves the goal of adaptively adjusting the collision warning indicators according to the actual tool changing scenario, effectively reducing false alarms and false misses, and improving the accuracy and reliability of tool magazine collision warning. Attached Figure Description

[0011] Figure 1 A flowchart illustrating a multimodal fusion-based tool magazine collision early warning method provided by the present invention;

[0012] Figure 2 This is a schematic diagram of the structure of a multimodal fusion tool magazine collision early warning system provided by the present invention.

[0013] In the attached diagram, the components represented by each label are as follows:

[0014] Data acquisition module 11, risk prediction module 12, indicator correction module 13, and real-time early warning module 14. Detailed Implementation

[0015] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0016] In the description of this invention, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of indicated technical features. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of the stated features. In the description of this invention, "a plurality of" means two or more, unless otherwise explicitly specified.

[0017] In the description of this invention, the term "for example" is used to mean "used as an example, illustration, or description." Any embodiment described as "for example" in this invention is not necessarily to be construed as being more preferred or advantageous than other embodiments. The following description is provided to enable any person skilled in the art to make and use the invention. Details are set forth in the following description for purposes of explanation. It should be understood that those skilled in the art will recognize that the invention can be made without using these specific details. In other instances, well-known structures and processes will not be described in detail to avoid obscuring the description of the invention with unnecessary detail. Therefore, the invention is not intended to be limited to the embodiments shown, but is consistent with the broadest scope of the principles and features disclosed herein.

[0018] Example 1, as Figure 1 As shown, this embodiment of the invention provides a multimodal fusion-based tool magazine collision early warning method, including:

[0019] S1. Collect the tool attribute characteristics, robotic arm wear index, preset tool changing speed and tool changing interference path of the tool changing system in a preset future time zone, and analyze the spatial overlap ratio of the tool changing path based on the preset tool changing speed and tool changing interference path.

[0020] Specifically, firstly, the tool attribute characteristics, robotic arm wear index, preset tool changing speed, and tool changing interference path of the tool changing system within a preset future time zone are considered. The tool attribute characteristics reflect the physical properties of the tool changing object, including the weight, length, diameter, and other geometric dimensions of the tool to be picked up and the tool to be returned. These parameters directly affect the collision probability. The heavier and larger the tool, the greater the impact force generated when a collision occurs, and the higher the collision risk. The robotic arm wear index characterizes the current accuracy status of the actuator. It is calculated by statistically analyzing historical operating data such as the cumulative usage time and number of tool changes of the tool changing robotic arm. The longer the usage time, the greater the wear of the transmission mechanism and the lower the positioning accuracy, thus increasing the collision probability. The preset tool changing speed reflects the dynamic requirements of the tool changing task. It is preset according to the current machining task and tool characteristics. The faster the tool changing speed, the longer the braking distance of the robotic arm and the shorter the reaction time, and the higher the collision probability. The tool changing interference path describes the spatial conflict area that may exist during the tool changing process, including the preset tool picking trajectory and the preset tool placing trajectory. When the tool picking path and the tool placing path are adjacent or intersecting in space, the collision probability increases significantly.

[0021] Secondly, the spatial overlap ratio of the tool change path is obtained based on the preset tool change speed and tool change interference path analysis. Specifically, firstly, a trajectory extension safety interval is configured, and the ratio of the preset tool change speed to the standard tool change speed is set as the extension adjustment coefficient. The trajectory extension safety interval is then corrected to obtain an adaptive trajectory extension safety interval. The larger the preset tool change speed, the larger the trajectory extension safety interval, in order to cope with higher collision risks. Subsequently, the preset tool retrieval trajectory and the preset tool placement trajectory are extended according to the adaptive trajectory extension safety interval to obtain the tool retrieval trajectory coverage area and the tool placement trajectory coverage area. Afterward, spatial overlap calculation is performed on the tool retrieval trajectory coverage area and the tool placement trajectory coverage area, and the ratio of the overlapping area to the sum of the areas of the tool retrieval and tool placement trajectory coverage areas is output as the tool change path spatial overlap ratio.

[0022] Through the above parameter collection and spatial overlap analysis, a dataset describing the risk characteristics of the current tool changing scenario was obtained, providing basic data support for subsequent collision risk prediction.

[0023] S2. Based on the tool attribute characteristics, robotic arm wear index, preset tool change speed and tool change path spatial overlap ratio, predict the tool change collision risk and output the predicted tool change collision risk coefficient.

[0024] Specifically, firstly, based on the tool changing operation monitoring logs of similar tool changing systems, a historical sample dataset is collected. This historical sample dataset includes a sample tool attribute feature set, a sample robotic arm wear index set, a sample tool changing speed set, and a sample tool changing path spatial overlap ratio set. At the same time, the percentage of historical tool changing collisions under different combinations of sample parameters is obtained, and this percentage of historical tool changing collisions is set as the sample tool changing collision risk coefficient, thereby constructing a complete sample tool changing collision risk coefficient set.

[0025] Then, a deep neural network model is trained using sample tool attribute feature sets, sample robotic arm wear index sets, sample tool change speed sets, and sample tool change path spatial overlap ratio sets as input features, and sample tool change collision risk coefficient sets as supervision labels. The network parameters are continuously optimized through backpropagation until the model converges, resulting in a tool change collision risk predictor that can accurately map the relationship between input features and collision risk.

[0026] Next, using the trained tool change collision risk predictor, the tool attribute characteristics of the current tool change scenario, the robotic arm wear index, the preset tool change speed, and the spatial overlap ratio of the tool change path are taken as input to calculate the tool change collision risk and output the predicted tool change collision risk coefficient. This predicted tool change collision risk coefficient quantitatively reflects the probability of a collision occurring in the current tool change scenario.

[0027] The above-mentioned collision risk prediction for tool changing can accurately assess the collision risk level of the current tool changing scenario, providing a basis for the dynamic adjustment of subsequent early warning indicators.

[0028] S3. Based on the predicted tool change collision risk coefficient, the initial collision warning index set is corrected to obtain the adapted collision warning index set.

[0029] Specifically, firstly, the motion trajectory error threshold, positioning accuracy error threshold, tool changing torque error threshold, and robotic arm amplitude threshold of the tool changing system under preset standard tool changing conditions are used as initial collision warning indicators to form an initial collision warning indicator set. These indicators are set based on standard operating conditions to provide a benchmark reference for collision warning.

[0030] Then, the average historical collision risk coefficient of the tool changing system under the preset standard tool changing state is obtained, and the ratio of this average historical collision risk coefficient to the current predicted collision risk coefficient is set as the index correction coefficient. This index correction coefficient reflects the degree of risk deviation of the current tool changing scenario relative to the standard tool changing scenario.

[0031] Subsequently, the initial collision warning index set is modified according to the index correction coefficient to obtain an adapted collision warning index set. The modification principle is as follows: when the predicted tool change collision risk coefficient is small, the index correction coefficient increases the initial collision warning index accordingly, making the monitoring more lenient; when the predicted tool change collision risk coefficient is large, the index correction coefficient decreases the initial collision warning index accordingly, making the monitoring more sensitive. Through this dynamic adjustment mechanism, it is ensured that the sensitivity of the collision warning can be adjusted according to the actual risk level.

[0032] By dynamically correcting the aforementioned warning indicators, a shift from fixed thresholds to adaptive thresholds has been achieved, effectively reducing false alarms and missed alarms caused by scene differences, and improving the accuracy and reliability of collision warnings.

[0033] S4. The real-time running dataset of the tool changing system in a preset future time zone is obtained by monitoring with multimodal sensors, and the real-time running dataset is mapped and judged and tool collision warning is given according to the adaptive collision warning index set.

[0034] Specifically, firstly, a multimodal sensor system is configured, comprising four core types: image sensors, positioning sensors, torque sensors, and vibration sensors. These sensors monitor visual, positional, mechanical, and vibration information during the tool changing process, forming a comprehensive sensing network. Secondly, the multimodal sensors monitor and acquire real-time operational data of the tool changing system within a preset future time zone. The image sensors use industrial cameras to monitor and capture the robotic arm's trajectory, and image processing algorithms analyze this data to obtain the real-time tool retrieval and placement trajectories. The positioning sensors collect the robotic arm's spatial position information in real time and output real-time positioning data. The torque sensors monitor torque changes during the tool changing process and acquire real-time torque data. The vibration sensors detect the vibration amplitude of the robotic arm during operation to obtain the real-time amplitude of the robotic arm.

[0035] Subsequently, based on the real-time tool retrieval trajectory, real-time tool placement trajectory, real-time positioning data, and real-time torque calculations, multiple real-time indicator deviations are obtained. Combined with the real-time amplitude of the robotic arm, a real-time status dataset is generated, reflecting the actual operating status of the current tool changing system. Then, the real-time status dataset is mapped and judged according to the adaptive collision warning indicator set. The real-time status data is compared one by one with the corresponding adaptive collision warning indicators. If the number of real-time operating data exceeding the corresponding adaptive collision warning indicator is not zero, a collision risk is determined, and the tool collision warning mechanism is immediately triggered, sending a warning signal to the operator or control center.

[0036] Through multimodal real-time monitoring and intelligent judgment mechanisms, comprehensive perception and accurate early warning of the tool changing process are achieved, effectively preventing tool collision accidents.

[0037] Furthermore, the tool attribute characteristics and tool change interference paths of the tool changing system in a preset future time zone are collected, including:

[0038] S11. Collect the tool attribute information of the tool to be picked up and the tool attribute information of the tool return in the preset future time zone of the tool changing system, and use them as tool attribute features;

[0039] S12. Collect the preset tool picking trajectory and preset tool placing trajectory of the tool changing system in a preset future time zone as the tool changing interference path.

[0040] In one feasible implementation, firstly, the tool changer system collects the attribute information of the tool to be retrieved and the tool to be returned in a preset future time zone, respectively, as tool attribute features. In modern CNC machining systems, especially in applications such as ultra-large machining centers, complex multi-spindle equipment, and robot-assisted tool changer systems, a dual-arm concurrent tool change mode is often adopted to improve tool change efficiency. This involves independent tool retrieval and tool placement robotic arms, or a tool exchange platform combined with a dual-arm structure design. In this tool change mode, two tool objects need to be processed simultaneously: one is the tool to be retrieved from the tool magazine and installed onto the spindle, and the other is the tool to be returned to the tool magazine from the spindle. Since the tool retrieval and placement actions may occur simultaneously, the attribute features of both tools will affect the collision risk during the tool change process. Therefore, it is necessary to comprehensively collect the attribute information of both the tool to be retrieved and the tool to be returned. Specifically, the tool changer system first queries the upcoming tool change task information to determine the tool number of the tool to be retrieved. Based on the tool number of the tool to be retrieved, detailed attribute information of the tool is extracted from the tool attribute database, including tool weight, tool length, tool diameter, tool holder type, and tool material, to obtain the tool attribute information. Simultaneously, the tool number of the currently returned tool is obtained through the tool changing system, and detailed attribute information of the returned tool is extracted from the tool attribute database based on this number, resulting in the returned tool attribute information. The tool attribute information of the tool to be retrieved and the returned tool attribute information are then summarized to obtain the tool attribute characteristics.

[0041] Secondly, the preset tool retrieval trajectory and preset tool placement trajectory of the tool changing system in a preset future time zone are collected as the tool changing interference path. Specifically, firstly, the preset tool retrieval trajectory is obtained from the tool changing system's path database. This preset tool retrieval trajectory is a standard running path pre-set according to the storage position of the tool to be retrieved in the tool magazine, including the complete three-dimensional coordinate sequence of the robotic arm moving from the initial position to the position of the tool to be retrieved, covering the motion trajectory of the approach phase, the grasping phase, and the withdrawal phase. Simultaneously, the preset tool placement trajectory is obtained from the path database. This preset tool placement trajectory is a standard running path pre-set according to the target storage position of the returned tool, describing the complete path of the robotic arm transporting the returned tool from the spindle position to the designated target storage position in the tool magazine. In dual-arm tool changing, the preset tool retrieval trajectory and the preset tool placement trajectory correspond to different robotic arms, requiring the acquisition of two independent preset running trajectories. The preset tool retrieval trajectory and the preset tool placement trajectory are used together as the tool changing interference path.

[0042] Through the above data acquisition process, tool attribute characteristics and tool change interference paths were obtained, providing a foundation of tool characteristic data and trajectory data for subsequent collision risk assessment and supporting the prediction of tool change collision risks.

[0043] Furthermore, based on the preset tool change speed and tool change interference path analysis, the spatial overlap ratio of the tool change path is obtained, including:

[0044] S13. Configure the trajectory extension safety range, set the ratio of the preset tool change speed to the standard tool change speed as the extension adjustment coefficient, correct the trajectory extension safety range, and obtain the adaptive trajectory extension safety range.

[0045] S14. Expand the safety zone according to the adaptation trajectory, and expand the preset tool retrieval trajectory and the preset tool release trajectory to obtain the tool retrieval trajectory coverage area and the tool release trajectory coverage area.

[0046] S15. Perform spatial overlap calculation on the area covered by the tool pick-up trajectory and the area covered by the tool release trajectory, and output the spatial overlap ratio of the tool change path.

[0047] In a preferred embodiment, firstly, a trajectory extension safety zone is configured. This zone defines the extension range of the trajectory in three-dimensional space, for example, set to ±2 cm, to account for potential positional deviations and dynamic errors that may occur during the actual operation of the robotic arm. Secondly, the ratio of the preset tool change speed to the standard tool change speed is set as the extension adjustment coefficient. The standard tool change speed refers to the recommended tool change execution speed under standard operating conditions, typically preset based on the robotic arm's performance parameters and safety requirements. When the preset tool change speed is higher than the standard tool change speed, the extension adjustment coefficient is greater than 1, indicating that the trajectory extension safety zone needs to be increased to cope with higher collision risks; when the preset tool change speed is lower than the standard tool change speed, the extension adjustment coefficient is less than 1, allowing for an appropriate reduction in the trajectory extension safety zone. Then, the initial trajectory extension safety zone is corrected based on the extension adjustment coefficient, i.e., the trajectory extension safety zone is multiplied by the extension adjustment coefficient to obtain the adaptive trajectory extension safety zone. Through this dynamic adjustment mechanism, a suitable safety zone range can be adaptively determined based on the preset tool change speed; the higher the speed, the larger the trajectory coverage area, effectively addressing higher collision risks.

[0048] Then, according to the safety interval of the adapted trajectory, the preset tool retrieval trajectory and the preset tool placement trajectory are spatially expanded to obtain the corresponding trajectory coverage areas. Specifically, for each trajectory point on the preset tool retrieval trajectory, a spherical expansion region is constructed in three-dimensional space with that trajectory point as the center and the adapted trajectory expansion safety interval as the radius. The union operation of the spherical expansion regions corresponding to all trajectory points is performed to obtain the tool retrieval trajectory coverage area. This tool retrieval trajectory coverage area represents the complete spatial range that the tool retrieval robot arm may occupy when performing the tool changing task. Similarly, the same expansion process is performed on the preset tool placement trajectory to obtain the tool placement trajectory coverage area.

[0049] Subsequently, spatial overlap calculations are performed on the areas covered by the tool pick-up trajectory and the tool drop trajectory, outputting a quantified spatial overlap ratio for the tool change path. Specifically, the intersection of the areas covered by the tool pick-up trajectory and the tool drop trajectory is first calculated to obtain the spatial overlap area. This spatial overlap area represents the region where the tool pick-up and tool drop trajectories may interfere in space. Then, the volumes of the areas covered by the tool pick-up trajectory, the tool drop trajectory, and the spatial overlap area are calculated separately. Afterward, the spatial overlap ratio for the tool change path is calculated using volume proportions. Specifically, the volume of the spatial overlap area is used as the numerator, and the sum of the volumes of the areas covered by the tool pick-up trajectory and the tool drop trajectory minus the volume of the spatial overlap area is used as the denominator. Dividing the numerator by the denominator yields the spatial overlap ratio for the tool change path. In actual calculations, a voxelization method can be used to calculate the volume, that is, dividing the three-dimensional space into regular cubic mesh units, counting the number of voxels covered by each region, and obtaining the total volume of the corresponding region by multiplying the number of voxels by the volume of a single voxel. The obtained tool change path spatial overlap ratio is between 0 and 1. When the tool change path spatial overlap ratio is 0, it means that there is basically no spatial interference between the two trajectories.

[0050] The above steps can accurately quantify the spatial overlap of the tool change path, providing spatial characteristic parameters for subsequent collision risk prediction.

[0051] Furthermore, based on the aforementioned tool attribute characteristics, robotic arm wear index, preset tool change speed, and tool change path spatial overlap ratio, a tool change collision risk prediction is performed, outputting a predicted tool change collision risk coefficient, including:

[0052] S21. Based on the tool changing operation monitoring logs of similar tool changing systems, collect sample tool attribute feature sets, sample robotic arm wear index sets, sample tool changing speed sets, and sample tool changing path spatial overlap ratio sets. Obtain the proportion of historical tool changing collisions under different sample tool attribute features, sample robotic arm wear indexes, sample tool changing speeds, and sample tool changing path spatial overlap ratios, set as the sample tool changing collision risk coefficient, and obtain the sample tool changing collision risk coefficient set.

[0053] S22. Using the sample tool attribute feature set, sample robotic arm wear index set, sample tool change speed set and sample tool change path spatial overlap ratio set as inputs, and using the sample tool change collision risk coefficient set as supervision, train a deep neural network until convergence to obtain a tool change collision risk predictor.

[0054] S23. Using the tool change collision risk predictor, the tool change collision risk is predicted based on the tool attribute characteristics, the robotic arm wear index, the preset tool change speed, and the spatial overlap ratio of the tool change path, and the predicted tool change collision risk coefficient is obtained.

[0055] In a preferred embodiment, in order to accurately predict the collision risk level in the current tool changing scenario, a tool changing collision risk predictor is constructed based on machine learning methods to achieve intelligent mapping from multi-dimensional input features to tool changing collision risk coefficients.

[0056] First, based on the tool change operation monitoring logs of similar tool changer systems, a sample dataset for training is constructed, including a sample tool attribute feature set, a sample robotic arm wear index set, a sample tool change speed set, and a sample tool change path spatial overlap ratio set. Specifically, complete records of each tool change operation are extracted from the tool change operation monitoring logs of similar tool changer systems. Each record contains the sample tool attribute features, sample robotic arm wear index, sample tool change speed, and sample tool change path spatial overlap ratio for that tool change. By collecting a large number of historical tool change operation monitoring logs, the sample tool attribute feature set, sample robotic arm wear index set, sample tool change speed set, and sample tool change path spatial overlap ratio set are obtained respectively.

[0057] Then, the historical tool change collision frequency ratios corresponding to different sample tool attribute features, sample robotic arm wear index, sample tool change speed, and sample tool change path spatial overlap ratio are obtained. Specifically, for each specific combination of sample tool attribute features, sample robotic arm wear index, sample tool change speed, and sample tool change path spatial overlap ratio, the ratio of the actual number of tool changes that collided under that combination to the total number of tool changes is calculated to obtain the historical tool change collision frequency ratio, which is then set as the sample tool change collision risk coefficient. By traversing all historical data combinations, a complete set of sample tool change collision risk coefficients is obtained, providing reliable supervision labels for subsequent tool change collision risk predictor training.

[0058] Subsequently, a deep neural network was used to construct a tool change collision risk predictor. Specifically, the sample tool attribute feature set, sample robotic arm wear index set, sample tool change speed set, and sample tool change path spatial overlap ratio set were used as input features of the deep neural network, and the sample tool change collision risk coefficient set was used as supervision labels for network training. During training, the input features were first normalized to ensure that feature parameters of different dimensions were within the same numerical range. Then, the backpropagation algorithm was used to continuously adjust the network weights and bias parameters, so that the error between the network output and the real sample tool change collision risk coefficients gradually decreased. When the prediction accuracy of the network on the validation set reached a preset threshold and the loss function converged, the training process ended, and the tool change collision risk predictor was obtained. This tool change collision risk predictor can learn the complex nonlinear mapping relationship between input features and collision risk.

[0059] Next, the trained tool-changing collision risk predictor is used to predict the collision risk of the current tool-changing scenario. Specifically, the tool attribute characteristics of the current tool-changing task, the robotic arm wear index, the preset tool-changing speed, and the spatial overlap ratio of the tool-changing path are input into the tool-changing collision risk predictor. After forward propagation calculation by the neural network, the predicted tool-changing collision risk coefficient is output. This predicted tool-changing collision risk coefficient is a value between 0 and 1. The larger the predicted tool-changing collision risk coefficient, the higher the probability of a collision in the current tool-changing scenario; the smaller the predicted tool-changing collision risk coefficient, the lower the collision risk.

[0060] Through the above steps, a collision risk assessment system based on learning from historical data and real-time prediction has been established. This system can accurately quantify the risk level of the current tool-changing scenario and provide a basis for the dynamic adjustment of subsequent early warning indicators.

[0061] Furthermore, the initial collision warning index set is corrected based on the predicted tool change collision risk coefficient to obtain an adapted collision warning index set, including:

[0062] S31. The motion trajectory error threshold, positioning accuracy error threshold, tool changing torque error threshold, and robotic arm amplitude threshold of the tool changing system under the preset standard tool changing state are used as initial collision warning indicators to obtain the initial collision warning indicator set.

[0063] S32. Obtain the average historical tool change collision risk coefficient of the tool changing system under the preset standard tool change state, and set the ratio of the average historical tool change collision risk coefficient to the predicted tool change collision risk coefficient as the index correction coefficient.

[0064] S33. The initial collision warning index set is corrected according to the index correction coefficient to obtain the adapted collision warning index set.

[0065] In a preferred embodiment, firstly, an initial collision warning index set based on standard operating conditions is constructed. Specifically, the motion trajectory error threshold, positioning accuracy error threshold, tool changing torque error threshold, and robotic arm amplitude threshold of the tool changing system under preset standard tool changing conditions are used as initial collision warning indicators to obtain the initial collision warning index set. Among them, the motion trajectory error threshold is used to monitor the degree of deviation between the actual running trajectory of the robotic arm and the preset trajectory; the positioning accuracy error threshold is used to monitor the accuracy level of the robotic arm reaching the target position; the tool changing torque error threshold is used to monitor the degree of abnormality in torque changes during tool changing; and the robotic arm amplitude threshold is used to monitor the vibration level of the robotic arm during operation. These initial collision warning indicators are set based on standard tool changing conditions, providing benchmark reference values ​​for collision warning.

[0066] Then, the indicator correction coefficient for dynamically adjusting the early warning indicators is calculated. Specifically, firstly, the average historical tool-changing collision risk coefficient of the tool-changing system under preset standard tool-changing conditions is obtained. This average historical tool-changing collision risk coefficient reflects the average collision risk level under standard operating conditions. Then, the ratio of the average historical tool-changing collision risk coefficient to the currently predicted tool-changing collision risk coefficient is set as the indicator correction coefficient. When the predicted tool-changing collision risk coefficient is lower than the average historical tool-changing collision risk coefficient, the indicator correction coefficient is greater than 1, indicating that the current scenario risk is low, and the early warning threshold can be appropriately relaxed; when the predicted tool-changing collision risk coefficient is higher than the average historical tool-changing collision risk coefficient, the indicator correction coefficient is less than 1, indicating that the current scenario risk is high, and the early warning threshold needs to be tightened to improve monitoring sensitivity.

[0067] Subsequently, the initial collision warning index set is corrected based on the index correction coefficient to obtain an adapted collision warning index set suitable for the current tool changing scenario. Specifically, each index value in the initial collision warning index set is multiplied by the index correction coefficient to obtain the corresponding adapted collision warning index. That is, the adapted motion trajectory error threshold equals the motion trajectory error threshold multiplied by the index correction coefficient, the adapted positioning accuracy error threshold equals the positioning accuracy error threshold multiplied by the index correction coefficient, the adapted tool changing torque error threshold equals the tool changing torque error threshold multiplied by the index correction coefficient, and the adapted robotic arm amplitude threshold equals the robotic arm amplitude threshold multiplied by the index correction coefficient. The adapted motion trajectory error threshold, adapted positioning accuracy error threshold, adapted tool changing torque error threshold, and adapted robotic arm amplitude threshold constitute the adapted collision warning index set. The correction principle is as follows: when the predicted tool changing collision risk coefficient is small, the index correction coefficient increases the warning threshold accordingly, making the monitoring more tolerant and reducing false alarms caused by normal fluctuations; when the predicted tool changing collision risk coefficient is large, the index correction coefficient decreases the warning threshold accordingly, making the monitoring more sensitive and ensuring timely detection of potential collision risks.

[0068] By implementing the above three steps, a dynamic adjustment mechanism from a fixed threshold to an adaptive threshold is established. This mechanism can intelligently adjust the warning indicators according to the actual risk level of the current tool-changing scenario, effectively reducing misjudgments and missed judgments caused by scenario differences, and improving the accuracy and reliability of the collision warning system.

[0069] Furthermore, the initial collision warning index set is corrected according to the index correction coefficient to obtain an adapted collision warning index set, including:

[0070] S331. Based on the tool changing operation monitoring logs of similar tool changing systems, analyze the impact of each initial collision warning indicator and tool changing collision risk, and obtain the sensitivity coefficient set of the initial collision warning indicator set, wherein the sensitivity coefficient and the impact of the indicator are positively correlated.

[0071] S332. Calculate the set of index correction coefficients based on the index correction coefficients and the set of index sensitivity coefficients;

[0072] S333. The initial collision warning index set is corrected according to the index correction coefficient set to obtain the adapted collision warning index set.

[0073] In a preferred embodiment, firstly, based on the tool change operation monitoring logs of similar tool change systems, the influence of each initial collision warning indicator on the tool change collision risk is analyzed. Specifically, motion trajectory error, positioning accuracy error, tool change torque error, and robotic arm amplitude, along with the corresponding tool change collision result records, are extracted from the tool change operation monitoring logs. Then, the tool change collision result is used as the dependent variable, and the motion trajectory error, positioning accuracy error, tool change torque error, and robotic arm amplitude are used as independent variables, respectively, to calculate the Pearson correlation coefficient between the independent and dependent variables. The larger the absolute value of the correlation coefficient, the stronger the influence of the indicator on the collision risk. Simultaneously, a stepwise regression analysis method is used to establish a multiple linear regression model between collision risk and each warning indicator. By analyzing the standardized coefficients of each indicator in the regression model, the importance ranking of each indicator is further determined. In addition, information theory methods such as information gain or Gini coefficient can be used to calculate the amount of information provided by each indicator in predicting collision risk; the greater the information gain, the stronger the ability to discriminate collision risk. By synthesizing the results of multiple statistical analysis methods, each initial collision warning indicator is assigned a corresponding degree of influence. For example, if the correlation coefficient of the motion trajectory error is 0.85, the regression coefficient is 0.72, and the information gain is 0.65, then the overall assessment of its indicator impact is high-level; if the correlation coefficient of the robotic arm amplitude is 0.42, the regression coefficient is 0.31, and the information gain is 0.28, then its indicator impact is assessed as medium-level. Subsequently, the indicator sensitivity coefficient set of the initial collision warning indicator set is calculated based on the indicator impact, where the indicator sensitivity coefficient and indicator impact are positively correlated. Specifically, the impact of each indicator is normalized to obtain the corresponding indicator sensitivity coefficient. The larger the indicator impact, the larger its indicator sensitivity coefficient, set above 1.0; the smaller the indicator impact, the smaller its indicator sensitivity coefficient, set below 1.0, thereby achieving differentiated weight allocation based on importance. For example, if the motion trajectory error is more sensitive to collision risk, its sensitivity coefficient is set to 1.2, indicating that the indicator needs a larger adjustment range; if the amplitude of the robotic arm has a relatively small impact on collision risk, its sensitivity coefficient is set to 0.8, indicating that the adjustment range of the indicator can be relatively mild.

[0074] Subsequently, based on a unified set of indicator correction coefficients and indicator sensitivity coefficients, a set of indicator correction coefficients is calculated for each initial collision warning indicator. Specifically, since each collision warning indicator has a different degree of impact on collision risk, a dedicated correction coefficient needs to be set for each collision warning indicator to achieve differentiated adjustments. Each initial collision warning indicator is multiplied by the unified indicator correction coefficient and the indicator sensitivity coefficient corresponding to that initial collision warning indicator to obtain the adapted collision warning indicator for that initial collision warning indicator. That is, the adapted collision warning indicator for the i-th indicator is equal to the initial collision warning indicator of that indicator multiplied by the unified indicator correction coefficient and then multiplied by the indicator sensitivity coefficient of that indicator. Among them, the unified indicator correction coefficient is the ratio of the average historical tool-changing collision risk coefficient to the predicted tool-changing collision risk coefficient, reflecting the overall risk deviation of the current tool-changing scenario relative to the standard tool-changing scenario; the indicator sensitivity coefficient reflects the degree of impact of the indicator on collision risk. Specifically, for the motion trajectory error index, the motion trajectory error threshold is multiplied by the unified index correction coefficient and the corresponding index sensitivity coefficient to obtain the adapted motion trajectory error threshold; for the positioning accuracy error index, the positioning accuracy error threshold is multiplied by the unified index correction coefficient and the corresponding index sensitivity coefficient to obtain the adapted positioning accuracy error threshold; for the tool changing torque error index, the tool changing torque error threshold is multiplied by the unified index correction coefficient and the corresponding index sensitivity coefficient to obtain the adapted tool changing torque error threshold; for the robotic arm amplitude index, the robotic arm amplitude threshold is multiplied by the unified index correction coefficient and the corresponding index sensitivity coefficient to obtain the adapted robotic arm amplitude threshold. The obtained adapted motion trajectory error thresholds, adapted positioning accuracy error thresholds, adapted tool changing torque error thresholds, and adapted robotic arm amplitude thresholds are summarized to form an adapted collision warning index set.

[0075] Through the above steps, a differentiated adjustment mechanism based on the importance of indicators has been established, ensuring that indicators that have a greater impact on collision risk can be adjusted more accurately, thereby further improving the accuracy and reliability of collision warnings.

[0076] Furthermore, the real-time operating dataset of the tool changing system within a preset future time zone is acquired through multimodal sensor monitoring. Based on the adaptive collision warning index set, the real-time operating dataset is mapped and judged, and tool collision warnings are issued, including:

[0077] S41. Configure a multimodal sensor, wherein the multimodal sensor includes an image sensor, a positioning sensor, a torque sensor, and a vibration sensor;

[0078] S42. The real-time operation data set of the tool changing system in the preset future time zone is obtained by monitoring the multimodal sensor, and the real-time tool picking trajectory, real-time tool releasing trajectory, real-time positioning data, real-time torque and real-time amplitude of the robotic arm are analyzed.

[0079] S43. Based on the real-time tool picking trajectory, real-time tool releasing trajectory, real-time positioning data, and real-time torque, multiple real-time index deviations are calculated, and combined with the real-time amplitude of the robotic arm, a real-time status dataset is obtained.

[0080] S44. Map the real-time status dataset according to the adaptive collision warning index set. If the number of real-time status data exceeding the corresponding adaptive collision warning index is not 0, then perform tool collision warning.

[0081] In a preferred embodiment, firstly, multimodal sensors are configured to achieve comprehensive monitoring of the tool changing process. Specifically, the multimodal sensors include image sensors, positioning sensors, torque sensors, and vibration sensors. The image sensors, employing industrial cameras, are installed at key locations in the tool changing area to capture the real-time motion of the robotic arm and the tool. The positioning sensors, including encoders and laser displacement sensors, are used to accurately measure the positional information of each joint of the robotic arm. The torque sensors are installed at key transmission parts of the robotic arm to monitor torque changes during the tool changing process. The vibration sensors are arranged on the robotic arm body to detect vibration signals during its operation. Through the collaborative work of the multimodal sensors, a comprehensive sensing network covering visual, positional, mechanical, and vibration information is formed.

[0082] Then, the real-time operation dataset of the tool changing system within a preset future time zone is acquired through multimodal sensor monitoring. Specifically, image sensors capture the tool changing process in real time using an industrial camera, and image processing algorithms are used to extract the motion trajectory information of the robotic arm, analyzing it to obtain the real-time tool picking trajectory and real-time tool placing trajectory; positioning sensors collect the angle and position information of each joint of the robotic arm in real time, and obtain real-time positioning data through forward kinematics calculation; torque sensors continuously monitor the torque changes of each joint of the robotic arm during the tool changing process, and obtain real-time torque data; vibration sensors detect the vibration signal of the robotic arm during the tool changing process, and the amplitude features are extracted through signal processing algorithms to obtain the real-time amplitude of the robotic arm.

[0083] Subsequently, a real-time status dataset for early warning judgment is calculated based on real-time monitoring data. Specifically, firstly, the real-time motion trajectory deviation is calculated by comparing the real-time tool retrieval trajectory with the preset tool retrieval trajectory; secondly, the real-time positioning accuracy deviation is calculated by comparing the real-time positioning data with the target positioning data; and thirdly, the real-time tool changing torque deviation is calculated by comparing the real-time torque data with the standard torque data. These calculated real-time motion trajectory deviations, real-time positioning accuracy deviations, and real-time tool changing torque deviations are used as multiple real-time indicator deviations. Combined with the real-time amplitude of the robotic arm directly obtained from the vibration sensor, they form a real-time status dataset that comprehensively reflects the degree of deviation of the current tool changing system from the standard state.

[0084] Then, the real-time status dataset is mapped and judged according to the adapted collision warning indicator set to achieve intelligent collision warning. Specifically, the real-time motion trajectory deviation in the real-time status dataset is compared with the adapted motion trajectory error threshold, the real-time positioning accuracy deviation is compared with the adapted positioning accuracy error threshold, the real-time tool changing torque deviation is compared with the adapted tool changing torque error threshold, and the real-time amplitude of the robotic arm is compared with the adapted robotic arm amplitude threshold. If the number of real-time status data exceeding the corresponding adapted collision warning indicator is not zero, that is, if any one or more real-time monitoring data exceeds its corresponding adapted warning threshold, a collision risk is immediately determined, the tool collision warning mechanism is triggered, and a warning signal is issued to the operator.

[0085] Through the above steps, a complete closed loop from multimodal perception to intelligent early warning is established, which can monitor the tool changing process in real time and provide accurate early warnings based on the risk characteristics of the current scenario, effectively preventing tool collision accidents.

[0086] Example 2, as Figure 2 As shown, based on the same inventive concept as the multimodal fusion tool magazine collision warning method provided in Embodiment 1, this embodiment of the invention also provides a multimodal fusion tool magazine collision warning system, including:

[0087] Data acquisition module 11 is used to collect tool attribute characteristics, robotic arm wear index, preset tool changing speed and tool changing interference path of the tool changing system in a preset future time zone, and to analyze the spatial overlap ratio of the tool changing path based on the preset tool changing speed and tool changing interference path.

[0088] Risk prediction module 12 is used to predict tool change collision risk based on the tool attribute characteristics, robotic arm wear index, preset tool change speed and tool change path spatial overlap ratio, and output the predicted tool change collision risk coefficient.

[0089] The index correction module 13 is used to correct the initial collision warning index set according to the predicted tool change collision risk coefficient to obtain an adapted collision warning index set.

[0090] The real-time early warning module 14 is used to monitor and acquire the real-time operation dataset of the tool changing system in a preset future time zone through multi-modal sensors, and to perform mapping judgment and tool collision early warning on the real-time operation dataset according to the adaptive collision early warning index set.

[0091] Furthermore, the execution steps of the data acquisition module 11 include:

[0092] The tool changer system collects the tool attribute information of the tool to be picked up and the tool attribute information of the returned tool in a preset future time zone, which are used as tool attribute features.

[0093] The preset tool retrieval trajectory and preset tool placement trajectory of the tool changing system in a preset future time zone are collected as the tool changing interference path.

[0094] Furthermore, the execution steps of the data acquisition module 11 also include:

[0095] Configure the trajectory extension safety range, set the ratio of the preset tool change speed to the standard tool change speed as the extension adjustment coefficient, correct the trajectory extension safety range, and obtain the adaptive trajectory extension safety range;

[0096] The safe zone is extended according to the adaptation trajectory, and the preset tool retrieval trajectory and preset tool release trajectory are extended to obtain the tool retrieval trajectory coverage area and the tool release trajectory coverage area.

[0097] The spatial overlap of the tool pick-up trajectory area and the tool drop trajectory area is calculated, and the spatial overlap ratio of the tool change path is output.

[0098] Furthermore, the execution steps of the risk prediction module 12 include:

[0099] Based on the tool changing operation monitoring logs of similar tool changing systems, sample tool attribute feature sets, sample robotic arm wear index sets, sample tool changing speed sets, and sample tool changing path spatial overlap ratio sets are collected. The proportion of historical tool changing collisions under different sample tool attribute features, sample robotic arm wear indexes, sample tool changing speeds, and sample tool changing path spatial overlap ratios is obtained and set as the sample tool changing collision risk coefficient. The sample tool changing collision risk coefficient set is obtained.

[0100] Using the sample tool attribute feature set, sample robotic arm wear index set, sample tool change speed set, and sample tool change path spatial overlap ratio set as inputs, and using the sample tool change collision risk coefficient set as supervision, a deep neural network is trained until convergence to obtain a tool change collision risk predictor.

[0101] Using the tool change collision risk predictor, the tool change collision risk is predicted based on the tool attribute characteristics, the robotic arm wear index, the preset tool change speed, and the spatial overlap ratio of the tool change path, and the predicted tool change collision risk coefficient is obtained.

[0102] Furthermore, the execution steps of the indicator correction module 13 include:

[0103] The initial collision warning index set is obtained by using the motion trajectory error threshold, positioning accuracy error threshold, tool changing torque error threshold, and robotic arm amplitude threshold of the tool changing system under the preset standard tool changing state as initial collision warning indicators.

[0104] Obtain the average historical tool change collision risk coefficient of the tool change system under a preset standard tool change state, and set the ratio of the average historical tool change collision risk coefficient to the predicted tool change collision risk coefficient as the index correction coefficient.

[0105] The initial collision warning index set is corrected according to the index correction coefficient to obtain the adapted collision warning index set.

[0106] Furthermore, the execution steps of the indicator correction module 13 also include:

[0107] Based on the tool changing operation monitoring logs of similar tool changing systems, the impact of each initial collision warning indicator and tool changing collision risk is analyzed to obtain the sensitivity coefficient set of the initial collision warning indicator set. The sensitivity coefficient and the impact of the indicator are positively correlated.

[0108] The set of index correction coefficients is calculated based on the set of index correction coefficients and the set of index sensitivity coefficients.

[0109] The initial collision warning index set is corrected based on the index correction coefficient set to obtain the adapted collision warning index set.

[0110] Furthermore, the execution steps of the real-time early warning module 14 include:

[0111] A multimodal sensor is configured, wherein the multimodal sensor includes an image sensor, a positioning sensor, a torque sensor, and a vibration sensor;

[0112] The multimodal sensor monitors and acquires the real-time operation data set of the tool changing system within a preset future time zone, and analyzes it to obtain the real-time tool picking trajectory, real-time tool placing trajectory, real-time positioning data, real-time torque, and real-time amplitude of the robotic arm.

[0113] Based on the real-time tool retrieval trajectory, real-time tool release trajectory, real-time positioning data, and real-time torque, multiple real-time index deviations are calculated, and combined with the real-time amplitude of the robotic arm, a real-time status dataset is obtained.

[0114] The real-time status dataset is mapped and judged according to the adaptive collision warning index set. If the number of real-time status data exceeding the corresponding adaptive collision warning index is not 0, a tool collision warning is issued.

[0115] It should be noted that the descriptions of each embodiment in the above embodiments have different focuses. For parts that are not described in detail in a certain embodiment, please refer to the relevant descriptions in other embodiments.

[0116] Those skilled in the art will understand that embodiments of the present invention can be provided as methods, systems, or computer program products. Therefore, the present invention can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention can take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0117] This invention is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded computer, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart illustrations. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0118] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0119] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0120] Although preferred embodiments of the invention have been described, those skilled in the art, once they have learned the basic inventive concept, can make other changes and modifications to these embodiments.

[0121] Obviously, those skilled in the art can make various modifications and variations to this invention without departing from its spirit and scope. Therefore, if these modifications and variations fall within the scope of this invention and its equivalents, this invention also intends to include these modifications and variations.

Claims

1. A multimodal fusion-based tool magazine collision early warning method, characterized in that, The methods include: The tool change system collects tool attribute characteristics, robotic arm wear index, preset tool change speed and tool change interference path in a preset future time zone, and analyzes the spatial overlap ratio of the tool change path based on the preset tool change speed and tool change interference path. Based on the tool attribute characteristics, robotic arm wear index, preset tool change speed and tool change path spatial overlap ratio, the tool change collision risk is predicted, and the predicted tool change collision risk coefficient is output. The initial collision warning index set is corrected based on the predicted tool change collision risk coefficient to obtain an adapted collision warning index set. The real-time operation dataset of the tool changing system in a preset future time zone is obtained by monitoring with multimodal sensors, and the real-time operation dataset is mapped and judged and tool collision warning is given according to the adaptive collision warning index set. The acquisition of tool attribute characteristics and tool change interference paths of the tool changing system in a preset future time zone includes: The tool changer system collects the tool attribute information of the tool to be picked up and the tool attribute information of the returned tool in a preset future time zone, which are used as tool attribute features. The preset tool pick-up trajectory and preset tool placement trajectory of the tool changing system in a preset future time zone are collected as the tool changing interference path; The method for determining the spatial overlap ratio of the tool change path based on the preset tool change speed and tool change interference path analysis includes: Configure the trajectory extension safety range, set the ratio of the preset tool change speed to the standard tool change speed as the extension adjustment coefficient, correct the trajectory extension safety range, and obtain the adaptive trajectory extension safety range; The safe zone is extended according to the adaptation trajectory, and the preset tool retrieval trajectory and preset tool release trajectory are extended to obtain the tool retrieval trajectory coverage area and the tool release trajectory coverage area. The spatial overlap of the tool pick-up trajectory and the tool drop trajectory is calculated, and the spatial overlap ratio of the tool change path is output. The process includes acquiring a real-time operating dataset of the tool changing system within a preset future time zone through multimodal sensor monitoring, and mapping and judging the real-time operating dataset and issuing tool collision warnings based on the adaptive collision warning index set, including: A multimodal sensor is configured, wherein the multimodal sensor includes an image sensor, a positioning sensor, a torque sensor, and a vibration sensor; The multimodal sensor monitors and acquires the real-time operation data set of the tool changing system within a preset future time zone, and analyzes it to obtain the real-time tool picking trajectory, real-time tool placing trajectory, real-time positioning data, real-time torque, and real-time amplitude of the robotic arm. Based on the real-time tool retrieval trajectory, real-time tool release trajectory, real-time positioning data, and real-time torque, multiple real-time index deviations are calculated, and combined with the real-time amplitude of the robotic arm, a real-time status dataset is obtained. The real-time status dataset is mapped and judged according to the adaptive collision warning index set. If the number of real-time status data exceeding the corresponding adaptive collision warning index is not 0, a tool collision warning is issued.

2. The multimodal fusion-based tool magazine collision early warning method according to claim 1, characterized in that, Based on the aforementioned tool attribute characteristics, robotic arm wear index, preset tool change speed, and tool change path spatial overlap ratio, a tool change collision risk prediction is performed, outputting a predicted tool change collision risk coefficient, including: Based on the tool changing operation monitoring logs of similar tool changing systems, sample tool attribute feature sets, sample robotic arm wear index sets, sample tool changing speed sets, and sample tool changing path spatial overlap ratio sets are collected. The proportion of historical tool changing collisions under different sample tool attribute features, sample robotic arm wear indexes, sample tool changing speeds, and sample tool changing path spatial overlap ratios is obtained and set as the sample tool changing collision risk coefficient. The sample tool changing collision risk coefficient set is obtained. Using the sample tool attribute feature set, sample robotic arm wear index set, sample tool change speed set, and sample tool change path spatial overlap ratio set as inputs, and using the sample tool change collision risk coefficient set as supervision, a deep neural network is trained until convergence to obtain a tool change collision risk predictor. Using the tool change collision risk predictor, the tool change collision risk is predicted based on the tool attribute characteristics, the robotic arm wear index, the preset tool change speed, and the spatial overlap ratio of the tool change path, and the predicted tool change collision risk coefficient is obtained.

3. The multimodal fusion-based tool magazine collision early warning method according to claim 1, characterized in that, The initial collision warning index set is corrected based on the predicted tool change collision risk coefficient to obtain an adapted collision warning index set, including: The initial collision warning index set is obtained by using the motion trajectory error threshold, positioning accuracy error threshold, tool changing torque error threshold, and robotic arm amplitude threshold of the tool changing system under the preset standard tool changing state as initial collision warning indicators. Obtain the average historical tool change collision risk coefficient of the tool change system under a preset standard tool change state, and set the ratio of the average historical tool change collision risk coefficient to the predicted tool change collision risk coefficient as the index correction coefficient. The initial collision warning index set is corrected according to the index correction coefficient to obtain the adapted collision warning index set.

4. The multimodal fusion-based tool magazine collision early warning method according to claim 3, characterized in that, The initial collision warning index set is corrected according to the index correction coefficient to obtain an adapted collision warning index set, including: Based on the tool changing operation monitoring logs of similar tool changing systems, the impact of each initial collision warning indicator and tool changing collision risk is analyzed to obtain the sensitivity coefficient set of the initial collision warning indicator set. The sensitivity coefficient and the impact of the indicator are positively correlated. The set of index correction coefficients is calculated based on the set of index correction coefficients and the set of index sensitivity coefficients. The initial collision warning index set is corrected based on the index correction coefficient set to obtain the adapted collision warning index set.

5. A multimodal fusion tool magazine collision early warning system, characterized in that, The system is used to implement a multimodal fusion-based tool magazine collision warning method as described in any one of claims 1 to 4, the system comprising: The data acquisition module is used to collect the tool attribute characteristics, robotic arm wear index, preset tool change speed and tool change interference path of the tool changing system in a preset future time zone, and to analyze the spatial overlap ratio of the tool change path based on the preset tool change speed and tool change interference path. The risk prediction module is used to predict the tool change collision risk based on the tool attribute characteristics, robotic arm wear index, preset tool change speed and tool change path spatial overlap ratio, and output the predicted tool change collision risk coefficient. The index correction module is used to correct the initial collision warning index set according to the predicted tool change collision risk coefficient to obtain an adapted collision warning index set. The real-time early warning module is used to monitor and acquire the real-time operation dataset of the tool changing system in a preset future time zone through multi-modal sensors, and to perform mapping judgment and tool collision early warning on the real-time operation dataset according to the adaptive collision early warning index set. The acquisition of tool attribute characteristics and tool change interference paths of the tool changing system in a preset future time zone includes: The tool changer system collects the tool attribute information of the tool to be picked up and the tool attribute information of the returned tool in a preset future time zone, which are used as tool attribute features. The preset tool pick-up trajectory and preset tool placement trajectory of the tool changing system in a preset future time zone are collected as the tool changing interference path; The method for determining the spatial overlap ratio of the tool change path based on the preset tool change speed and tool change interference path analysis includes: Configure the trajectory extension safety range, set the ratio of the preset tool change speed to the standard tool change speed as the extension adjustment coefficient, correct the trajectory extension safety range, and obtain the adaptive trajectory extension safety range; The safe zone is extended according to the adaptation trajectory, and the preset tool retrieval trajectory and preset tool release trajectory are extended to obtain the tool retrieval trajectory coverage area and the tool release trajectory coverage area. The spatial overlap of the tool pick-up trajectory and the tool release trajectory is calculated, and the spatial overlap ratio of the tool change path is output. The process includes acquiring a real-time operating dataset of the tool changing system within a preset future time zone through multimodal sensor monitoring, and mapping and judging the real-time operating dataset and issuing tool collision warnings based on the adaptive collision warning index set, including: A multimodal sensor is configured, wherein the multimodal sensor includes an image sensor, a positioning sensor, a torque sensor, and a vibration sensor; The multimodal sensor monitors and acquires the real-time operation data set of the tool changing system within a preset future time zone, and analyzes it to obtain the real-time tool picking trajectory, real-time tool placing trajectory, real-time positioning data, real-time torque, and real-time amplitude of the robotic arm. Based on the real-time tool retrieval trajectory, real-time tool release trajectory, real-time positioning data, and real-time torque, multiple real-time index deviations are calculated, and combined with the real-time amplitude of the robotic arm, a real-time status dataset is obtained. The real-time status dataset is mapped and judged according to the adaptive collision warning index set. If the number of real-time status data exceeding the corresponding adaptive collision warning index is not 0, a tool collision warning is issued.

Images