Artificial intelligence based mechanical automated assembly system

Abstract

The present application relates to the technical field of automatic assembly, and particularly relates to a mechanical automatic assembly system based on artificial intelligence. The present application acquires pose data and environment state of an assembly executor in real time through a perception positioning module, constructs an expected twin model according to the pose data and the environment state, divides a running period into a plurality of sub-periods to evaluate joint trajectory dispersion, determines a set of monitoring time points for a low-dispersion sub-period and calculates instantaneous deviation based on the dispersion classification result, selectively monitors abnormalities, calculates tracking errors for a high-dispersion sub-period in real time, generates an error tolerance threshold based on illumination intensity and environment image interference, and finally determines abnormalities according to the tracking error representation value and the error tolerance threshold. When facing a large number of monitoring targets, the present application adopts different error analysis strategies for different sub-periods, introduces a dynamic threshold adjustment mechanism, thereby saving monitoring computing power and improving monitoring accuracy and reliability.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of automatic assembly, in particular to a mechanical automatic assembly system based on artificial intelligence. BACKGROUND

[0002] In the field of mechanical automatic assembly, especially in the scene of precise part assembly and complex structure docking, the pose accuracy and trajectory control of assembly executors such as mechanical arms are the key to ensure product quality and production efficiency. Therefore, dynamic monitoring of mechanical arms is crucial. At present, there are various means for dynamic monitoring of mechanical arms.

[0003] For example, Chinese Patent Publication No. CN120269610A discloses a multi-joint robot running trajectory intelligent monitoring system based on artificial intelligence, which includes a robot trajectory monitoring platform, a robot running unit, a trajectory generation unit, a joint trajectory tracking unit, an interference matching unit, a trajectory action analysis unit, and an offset management unit. Under the premise of determining the running trajectory, the joint position coordinates of the multi-joint robot are preliminarily analyzed for accuracy. Under the premise of accurate joint position coordinates, the trajectory during the joint running step process is analyzed in depth, and the vibration interference factors of the multi-joint robot are analyzed to reasonably analyze the running trajectory coincidence degree and analyze the running trajectory coincidence degree to understand the running trajectory standard, so as to reasonably and specifically adjust the running trajectory according to the information feedback to improve the accuracy and stability of the multi-joint robot running trajectory.

[0004] However, the prior art has the following problems,

[0005] 1. In the prior art, the influence of dynamic environmental factors on error detection is not considered, such as changes in light, environmental image interference, and dynamic interference of obstacles, which leads to errors in monitoring data of assembly executors, resulting in a decrease in monitoring accuracy.

[0006] 2. In the prior art, the differences in running conditions of assembly executors during the running period are not considered. When facing a large number of monitoring targets, there is a lack of real-time error monitoring and dynamic threshold adjustment mechanism for high complexity sub-periods, resulting in waste of computing power, lag in abnormal detection, and high false alarm rate.

[0007] 3. In the prior art, artificial intelligence is not combined with twin modeling and pre-analysis of the assembly process, and only mechanical structures or simple sensor feedback are relied on, which cannot achieve adaptive optimization of error tolerance threshold of assembly actions. SUMMARY

[0008] To this end, the present application provides an artificial intelligence-based mechanical automated assembly system to overcome the lack of classification monitoring and dynamic threshold adjustment mechanism for the action execution of the assembly execution body in the prior art when facing a large number of monitoring targets, resulting in waste of computing power, and problems of abnormal detection lag and high false alarm rate.

[0009] To achieve the above-mentioned purpose, the present application provides an artificial intelligence-based mechanical automated assembly system, comprising:

[0010] A perception positioning module is used to obtain the pose data of the assembly execution body and the environment state in real time, wherein the pose data includes the three-dimensional coordinates and the attitude angle of each observation point, and the environment state includes the light intensity and the environment image;

[0011] A pre-analysis module is connected with the perception positioning module and is used to construct an expected twin model based on the expected pose, analyze the expected execution trajectory of the assembly execution body within the running period based on the expected twin model, including setting a plurality of observation points, determining the expected execution trajectory of the observation points within each sub-period, and evaluating the joint trajectory dispersion based on the difference between the expected execution trajectories;

[0012] A pre-control module is connected with the pre-analysis module, classifies each sub-period based on the joint trajectory dispersion within each sub-period, and monitors the assembly execution body based on the classification results, including,

[0013] For the low-dispersion sub-period, a set of monitoring time points is determined, only the pose data of the corresponding monitoring time points is monitored, and it is determined whether the assembly execution body is abnormal;

[0014] For the high-dispersion sub-period, the pose data of the assembly execution body is obtained in real time, the execution trajectory is constructed based on the pose data, the tracking error between the execution trajectory and the expected execution trajectory is determined, and the error tolerance threshold is dynamically generated based on the environment state, and it is determined whether the assembly execution body is abnormal based on the tracking error and the error tolerance threshold.

[0015] Further, the pre-analysis module is used to construct an expected twin model based on the expected pose, including,

[0016] The expected three-dimensional coordinates and the expected attitude angle of the observation points of the assembly execution body at each time are defined according to the current assembly task;

[0017] The model of the assembly execution body is established in the virtual environment, and the expected pose data is mapped to the model to generate an expected twin model reflecting the ideal assembly process;

[0018] Wherein, the observation points are selected based on the movable joints of the assembly execution body.

[0019] Further, the pre-analysis module is configured to analyze the execution actions in the running period of the assembly execution body based on the expected twin model, including

[0020] dividing the running period of the assembly execution body into a plurality of continuous sub-periods, and extracting the expected execution trajectory of each observation point in each sub-period based on the expected twin model.

[0021] Further, the pre-analysis module is configured to evaluate the joint trajectory dispersion based on the difference between the execution trajectories, including,

[0022] configured to calculate the trajectory similarity between the expected execution trajectories of each observation point in each sub-period;

[0023] configured to calculate the mean of the trajectory similarity, and determine the inverse of the mean of the trajectory similarity as the joint trajectory dispersion.

[0024] Further, the pre-control module is configured to classify each of the sub-periods based on the joint trajectory dispersion in each sub-period, including,

[0025] configured to compare the joint trajectory dispersion in each sub-period with a preset dispersion threshold;

[0026] if the joint trajectory dispersion is greater than or equal to the preset dispersion threshold, mark the corresponding sub-period as high dispersion;

[0027] if the joint trajectory dispersion is less than the preset dispersion threshold, mark the corresponding sub-period as low dispersion. Further, the pre-control module is configured to determine a set of monitoring time points, including,

[0028] configured to determine a monitoring time point at a predetermined interval for a low dispersion sub-period, and store the monitoring time point as the set of monitoring time points.

[0029] Further, the pre-control module is configured to only monitor the pose data at the corresponding monitoring time points, and determine whether the assembly execution body has an anomaly, including,

[0030] configured to calculate the instantaneous deviation amount of the pose data and the expected pose data at each monitoring time point, the instantaneous deviation amount being calculated based on the coordinate deviation and the attitude angle deviation;

[0031] if the instantaneous deviation amount is greater than or equal to a preset tolerance threshold, determine that the assembly execution body has an anomaly.

[0032] Further, the pre-control module is configured to construct an execution trajectory based on the pose data, and determine the tracking error between the execution trajectory and the expected execution trajectory, including,

[0033] The pose data in the time domain segments of the sub-period segments is used to construct an execution trajectory, and the corresponding expected execution trajectory in the time domain segment is synchronously acquired;

[0034] The trajectory similarity of the execution trajectory and the expected execution trajectory is determined as a tracking error.

[0035] Further, the pre-control module dynamically generates an error tolerance threshold based on the environment state, including,

[0036] The illumination-sensitive component for the error tolerance threshold is adjusted based on the illumination intensity;

[0037] The environment-sensitive component for the error tolerance threshold is adjusted based on the environment image interference;

[0038] The illumination-sensitive component and the environment-sensitive component are weighted and summed to obtain a tolerance adjustment coefficient, and the product of the tolerance adjustment coefficient and a reference error threshold is determined as the error tolerance threshold;

[0039] The environment image interference is calculated based on the chromatic difference between the environment image and the assembly execution body.

[0040] Further, the pre-control module determines whether the assembly execution body is abnormal based on the tracking error and the error tolerance threshold, including,

[0041] If the tracking error is greater than the current error tolerance threshold, it is determined that the assembly execution body is abnormal.

[0042] Compared with the prior art, the pose data and the environment state of the assembly execution body are acquired in real time by the perception positioning module, the expected twin model is constructed by the front-end analysis module, the running period is divided into several sub-period segments to evaluate the joint trajectory dispersion, the pre-control module determines the monitoring time point set for the low-dispersion sub-period segment and calculates the instantaneous deviation amount according to the dispersion classification result, selectively monitors the abnormality, calculates the tracking error for the high-dispersion sub-period segment in real time, generates the error tolerance threshold based on the illumination intensity and the environment image interference, and finally determines the abnormality according to the tracking error representation value and the error tolerance threshold. When facing a large number of monitoring targets, the present application adopts different error analysis strategies for different period sub-segments, introduces a dynamic threshold adjustment mechanism, thereby saving monitoring computing power and improving monitoring accuracy and reliability.

[0043] Especially, the application divides the running cycle into continuous sub-periods and extracts the expected execution trajectory of each observation point based on the expected twin model. In actual situations, the motion of the assembled execution body is different at different time periods. There are time periods with simple motion and low error tendency, such as parallel moving motion, and time periods with complex motion of multiple components. Based on this, the application considers the difference between the expected execution trajectories, distinguishes high-dispersion and low-dispersion sub-periods, adopts sparse monitoring strategy for low-dispersion sub-periods, and adopts real-time monitoring for high-dispersion sub-periods. Different detection methods are used to reduce the system operation load while ensuring monitoring accuracy. In some application scenarios, the installation and use cost of part of the monitoring equipment can also be saved.

[0044] Especially, the application considers the influence of light intensity variation and environmental image interference on the monitoring accuracy of tracking error. In actual production environment, there may be light variation or background interference, which affects the monitoring accuracy of binocular monitoring means and may cause local distortion of the monitored data. Based on this, the light-sensitive component and the environment-sensitive component are weighted and summed to calculate the tolerance adjustment coefficient, and the error tolerance threshold is further determined. The error tolerance threshold can be adjusted in real time with the change of the environment, thereby maintaining the effectiveness of abnormal judgment and reducing the risk of missed detection.

[0045] Especially, the application considers the environmental adaptive mechanism, which can dynamically optimize the monitoring strategy according to the real-time perceived environmental parameters. This mechanism realizes intelligent adjustment of the error tolerance threshold by continuously analyzing multi-dimensional environmental information such as light, environment and obstacles, thereby maintaining the effectiveness of the abnormal judgment standard in complex industrial scenes, improving the monitoring accuracy and reliability, and reducing the risk of missed detection.

[0046] Especially, the application preforms the ideal assembly process by constructing the expected twin model in the virtual environment, thereby evaluating the joint trajectory dispersion before the actual operation of the physical system, enabling the system to identify high-dispersion sub-periods in advance, and optimizing the monitoring strategy and error tolerance threshold of different sub-periods. This changes from passive response to active prediction, speeds up the system response, and improves the monitoring accuracy and reliability when facing a large number of monitoring targets. BRIEF DESCRIPTION OF DRAWINGS

[0047] Figure 1 The structure schematic diagram of the mechanical automation assembly system based on artificial intelligence of the application embodiment;

[0048] Figure 2 The logic block diagram of classifying each sub-period of the application embodiment;

[0049] Figure 3A logic block diagram for judging whether an assembly execution body exists abnormity for a low-dispersion sub-period section of the embodiment of the application;

[0050] Figure 4 A logic block diagram for judging whether an assembly execution body exists abnormity for a high-dispersion sub-period section of the embodiment of the application. DETAILED DESCRIPTION

[0051] In order to make the objects and advantages of the present application clearer, the following further describes the present application with reference to embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and should not be used to limit the present application.

[0052] The preferred embodiments of the present application are described below with reference to the accompanying drawings. Those skilled in the art should understand that these embodiments are only used to explain the technical principles of the present application and are not used to limit the protection scope of the present application.

[0053] In addition, it should be further explained that, in the description of the present application, unless otherwise explicitly specified and limited, the term "connection" should be understood broadly, for example, it can be fixed connection, or detachable connection, or integrally connected; it can be mechanical connection, or electrical connection; it can be directly connected, or indirectly connected through an intermediate medium, or the internal communication of two elements. Those skilled in the art can understand the specific meaning of the above-mentioned term in the present application according to the specific circumstances.

[0054] Please refer to Figure 1 As shown in the figure, the mechanical automation assembly system based on artificial intelligence of the embodiment of the application includes:

[0055] A perception positioning module is used to acquire the pose data and the environment state of the assembly execution body in real time, the pose data includes the three-dimensional coordinates and the attitude angle of each observation point, and the environment state includes the light intensity and the environment image;

[0056] A pre-analysis module is connected with the perception positioning module and is used to construct an expected twin model based on the expected pose, analyze the expected execution trajectory of the assembly execution body in the running period based on the expected twin model, including setting a plurality of observation points, determining the expected execution trajectory of the observation points in each sub-period section, and evaluating the joint trajectory dispersion based on the difference between the expected execution trajectories;

[0057] A pre-control module is connected with the pre-analysis module, classifies each sub-period section based on the joint trajectory dispersion in each sub-period section, and monitors the assembly execution body based on the classification result, including,

[0058] For the low-dispersion sub-period section, a set of monitoring time points is determined, only the pose data of the corresponding monitoring time points is monitored, and it is determined whether the assembly executor exists abnormity;

[0059] For the high-dispersion sub-period section, the pose data of the assembly executor is acquired in real time, the execution trajectory is constructed based on the pose data, the tracking error between the execution trajectory and the expected execution trajectory is determined, the error tolerance threshold is dynamically generated based on the environment state, and it is determined whether the assembly executor exists abnormity based on the tracking error and the error tolerance threshold.

[0060] Specifically, the assembly executor is a mechanical arm participating in the assembly process.

[0061] Specifically, the perception positioning module comprises,

[0062] The binocular structure depth camera is used to acquire the three-dimensional coordinates of each observation point of the assembly executor and the environment image;

[0063] The inertial measurement unit is located inside the assembly executor and is used to collect the attitude angle data of the observation points of the assembly executor in real time;

[0064] The illuminance sensor is used to acquire and quantify the environmental light intensity.

[0065] Specifically, the pre-analysis module is used to construct an expected twin model based on the expected pose, comprising,

[0066] According to the current assembly task, the expected three-dimensional coordinates and the expected attitude angle of the observation points of the assembly executor at each time are defined;

[0067] A model of the assembly executor is established in the virtual environment, and the expected pose data is mapped to the model to generate an expected twin model reflecting the ideal assembly process;

[0068] Wherein, the observation points are selected based on the movable joints of the assembly executor, for example, the movable joints can be set as the observation points.

[0069] Specifically, the way of constructing the model is not limited, and it can be understood that the expected three-dimensional coordinates and the expected attitude angle of each observation point of the model at several time points are acquired, the motion of the model at each time is mapped, and the corresponding model is constructed in the virtual space, which is not repeated here.

[0070] The application can evaluate joint trajectory dispersion before the physical system actually runs by constructing a desired twin model in a virtual environment to pre-act the ideal assembly process, so that the system can identify high dispersion sub-period segments in advance, optimize the monitoring strategy and error tolerance threshold of different sub-period segments, change from passive response to active prediction, speed up the system response, and improve the monitoring accuracy and reliability when facing a large number of monitoring targets.

[0071] Specifically, the pre-analysis module is used to analyze the execution actions in the running period of the assembly executor based on the desired twin model, including

[0072] The running period of the assembly executor is divided into a plurality of continuous sub-period segments, and the expected execution trajectory of each observation point in each sub-period segment is extracted based on the desired twin model.

[0073] Specifically, the division mode of the continuous sub-period segment is not limited. The total running period can be divided into a plurality of equal-length sub-period segments according to a fixed time length, for example, a sub-period segment can be determined every 1s. The sub-period segment boundary can also be divided based on the end time point of the preset key action in the desired twin model, for example, a sub-period segment is determined based on the start time point and the end time point of the grabbing action. Those skilled in the art can divide the sub-period segment according to the actual situation of the production line assembly process, which will not be repeated here.

[0074] The application divides the running period into continuous sub-period segments and extracts the expected execution trajectory of each observation point based on the desired twin model. In actual situations, the motion of the assembly executor is different at different time periods. There are time periods with simple motion and low error tendency, such as parallel moving motion, and time periods with complex motion of multiple components. Based on this, the application considers the difference between the expected execution trajectories to distinguish high-dispersion and low-dispersion sub-period segments. The sparse monitoring strategy is adopted for the low-dispersion segment, and the real-time monitoring is adopted for the high-dispersion segment, and different detection methods are adopted to reduce the system operation load while ensuring the monitoring accuracy. In some application scenarios, the installation and use cost of part of the monitoring equipment can also be saved.

[0075] Specifically, the pre-analysis module is used to evaluate the joint trajectory dispersion based on the difference between the execution trajectories, including

[0076] used to calculate the trajectory similarity between the expected execution trajectories of each observation point in each sub-period segment;

[0077] used to calculate the mean of the trajectory similarity, and the inverse of the mean of the trajectory similarity is determined as the joint trajectory dispersion.

[0078] Specifically, the manner of calculating the trajectory similarity is not limited, and an image similarity algorithm can be used to calculate the trajectory similarity, for example, the cosine similarity between the expected execution trajectories is calculated as the trajectory similarity, of course, other forms can also be used, which will not be repeated here.

[0079] It can be understood that for a multi-joint robot, when performing a complex action, the joints can exhibit different movement modes, thereby making the trajectory similarity between the joint nodes corresponding to the expected execution trajectories low, and the joint node trajectory dispersion increases.

[0080] Please refer to Figure 2 shown, Figure 2 The logic block diagram for classifying each of the sub-period segments of the embodiment of the application, the pre-control module is used to classify each of the sub-period segments based on the joint trajectory dispersion in each sub-period segment, including,

[0081] comparing the joint trajectory dispersion in the sub-period segment with the preset dispersion threshold;

[0082] If the joint trajectory dispersion is greater than or equal to the preset dispersion threshold, the corresponding sub-period segment is marked as high dispersion;

[0083] If the joint trajectory dispersion is less than the preset dispersion threshold, the corresponding sub-period segment is marked as low dispersion.

[0084] Specifically, the preset dispersion threshold can be determined based on historical assembly task data or simulation test results, and the average value of the trajectory dispersion sequence of each sub-period segment under typical working conditions is extracted. In actual system operation, the product of the average value of the trajectory dispersion sequence and the error tolerance coefficient is used as the preset dispersion threshold. The value range of the error tolerance coefficient is [1.05, 1.12].

[0085] Specifically, the pre-control module is used to determine a set of monitoring time points, including,

[0086] For low-dispersion sub-period segments, determine the monitoring time points at a predetermined interval, and store the monitoring time points as the set of monitoring time points.

[0087] Specifically, the specific size of the predetermined interval is not limited, and the predetermined interval can be determined based on the length of the sub-period segment, and is set to between 0.25 and 0.5 times the length of the sub-period segment.

[0088] Please refer to Figure 3 shown, Figure 3 The logic block diagram for judging whether the assembly execution body has an abnormality for a low-dispersion sub-period segment, the pre-control module only monitors the pose data at the corresponding monitoring time points, and determines whether the assembly execution body has an abnormality, including,

[0089] calculating, at each monitoring time point, an instantaneous deviation amount of the pose data from the expected pose data, the instantaneous deviation amount being calculated based on a coordinate deviation and an attitude angle deviation;

[0090] If the instantaneous deviation amount is greater than or equal to a preset tolerance threshold, it is determined that the assembly executor has an abnormality.

[0091] In implementation, the coordinate deviation is a difference ratio of a three-dimensional coordinate in the pose data and an expected three-dimensional coordinate in the expected pose data;

[0092] The attitude angle deviation is a difference ratio of an attitude angle in the pose data and an expected attitude angle in the expected pose data;

[0093] The difference ratio is a ratio of an absolute value of a difference between two numerical values and an average of the two numerical values.

[0094] The instantaneous deviation amount is a weighted sum of the coordinate deviation and the attitude angle deviation, and the weight for the coordinate deviation is 0.5 and the weight for the attitude angle deviation is 0.5 for comprehensive consideration of the coordinate deviation and the attitude angle deviation.

[0095] Specifically, before the assembly task is started, pose data of the assembly executor accurately completing the assembly under a non-abnormal working condition for no less than thirty times is collected, an ideal instantaneous deviation amount of actual pose and expected pose at all monitoring time points in each execution process is calculated, and a product of an average of the ideal instantaneous deviation amounts and an error amplification coefficient is taken as the preset tolerance threshold. The error amplification coefficient of the embodiment of the present application has a value range of [1.05, 1.15].

[0096] Specifically, the pre-control module constructs an execution trajectory based on the pose data, and determines a tracking error between the execution trajectory and an expected execution trajectory, including,

[0097] to construct an execution trajectory by real-time acquisition of pose data in a plurality of time domain segments in a sub-period segment, and synchronously acquire a corresponding expected execution trajectory in the time domain segment;

[0098] to determine a trajectory similarity of the execution trajectory and the expected execution trajectory as the tracking error.

[0099] Specifically, the way of determining the trajectory similarity of the execution trajectory and the expected execution trajectory is not limited, and an image similarity algorithm can be used to calculate the trajectory similarity, for example, a cosine similarity between the execution trajectory and the expected execution trajectory is calculated as the trajectory similarity, of course, other forms can also be used, which will not be described here.

[0100] Specifically, the pre-control module dynamically generates an error tolerance threshold based on the environment state, including,

[0101] to adjust a light-sensitive component of the error tolerance threshold based on the light intensity;

[0102] an environmental image interference quantity is used to adjust the environmental sensitive component for the error tolerance threshold;

[0103] the illumination sensitive component and the environmental sensitive component are weighted and summed to obtain a tolerance adjustment coefficient, and a product of the tolerance adjustment coefficient and a reference error threshold is determined as the error tolerance threshold;

[0104] The environmental image interference quantity is calculated based on a chroma difference ratio of the environmental image and the assembly executor.

[0105] In the implementation, the reference error threshold is predetermined, before the assembly task is started, the pose data of the assembly executor in the no-abnormal working condition is collected for no less than thirty times in the accurate completion of the assembly process for a plurality of sub-periods, the execution trajectory is constructed, the tracking error between the execution trajectory and the expected execution trajectory is calculated, the mean value of the tracking error is solved, and the product of the mean value of the tracking error and the error amplification coefficient is taken as the reference error threshold. The error amplification coefficient of the embodiment of the present application is in the range of [1.05, 1.15].

[0106] In the implementation, the image of the assembly executor is obtained in advance, and the chroma value of the assembly executor is extracted.

[0107] In the actual operation process, the chroma mean value of the environmental image is determined, and the chroma difference ratio of the chroma value and the chroma mean value is calculated.

[0108] In the implementation, the environmental image interference quantity is the reciprocal of the chroma difference ratio.

[0109] In the embodiment of the present application, the illumination critical value is set to 200 lux,

[0110] When the illumination intensity is less than the illumination critical value, the illumination sensitive component is 1.5;

[0111] When the illumination intensity is greater than or equal to the illumination critical value, the illumination sensitive component is 1.

[0112] The illumination sensitive component is used to consider the influence of illumination on the visual positioning accuracy.

[0113] Specifically, the illumination critical value is predetermined by a person skilled in the art, and the reference interval of the illumination critical value in the embodiment of the present application is [150 lux, 300 lux].

[0114] Specifically, the environmental sensitive component is determined based on the environmental image interference quantity,

[0115] When the environmental image interference quantity is less than the environmental interference threshold, the environmental sensitive component is set to 1.0;

[0116] When the environmental image interference quantity is greater than or equal to the environmental interference threshold, the environmental sensitive component is set to 1.5.

[0117] Specifically, the purpose of setting the environment interference threshold is to consider the case that the environment image is similar to the assembly executor and is easy to produce visual effects, and generally when the chroma difference is less than 0.2, it indicates that the chroma of the image is relatively close, based on this, the environment image interference amount is calculated as 5, and in the implementation, the environment interference threshold is set to 5.

[0118] Specifically, the weighted weight of the light sensitive component is 0.35, and the weighted weight of the environment sensitive component is 0.65. In actual situation, since the industrial field usually adopts constant environment lighting system, the overall illumination intensity is relatively stable, and only the local transient shadow changes due to the movement of workpieces, equipment or personnel. Such shadow interference has the characteristics of spatial locality and temporal shortness, and the influence on the visual positioning accuracy is significantly lower than the environment image interference amount change. Therefore, by reducing the weight proportion of the light sensitive component, while increasing the weight proportion of the environment sensitive component, the dynamic adjustment of the error tolerance threshold is more suitable for the priority distribution of the actual interference source.

[0119] The present application considers the influence of illumination intensity change and dynamic update of environment image interference amount on the monitoring accuracy of tracking error. In actual production environment, there may be illumination change or background interference, which affects the monitoring accuracy of binocular monitoring means and may cause local distortion of the monitored data. Based on this, the light sensitive component and the environment sensitive component are weighted and summed to calculate the tolerance adjustment coefficient, and the error tolerance threshold is further determined. The error tolerance threshold can be adjusted in real time with the change of the environment, so as to maintain the effectiveness of the abnormality judgment and reduce the risk of missed detection.

[0120] Specifically, please refer to Figure 4 As shown in the figure, Figure 4 The logic block diagram for judging the existence of the assembly executor in the high dispersion sub-period section, the pre-control module judges whether the assembly executor exists abnormally based on the tracking error and the error tolerance threshold, including,

[0121] If the tracking error is greater than the error tolerance threshold, it is judged that the assembly executor exists abnormally.

[0122] The present application considers the environment adaptive mechanism, which can dynamically optimize the monitoring strategy according to the real-time perceived environment parameters. The mechanism realizes the intelligent adjustment of the error tolerance threshold by continuously analyzing the multi-dimensional environment information such as light, environment and obstacles, so as to maintain the effectiveness of the abnormality judgment standard in the complex and changeable industrial scene, improve the monitoring accuracy and reliability, and reduce the risk of missed detection.

[0123] The technical scheme of the present application has been described in combination with the preferred embodiments shown in the drawings, but it is easy for those skilled in the art to understand that the protection scope of the present application is obviously not limited to these specific embodiments. Those skilled in the art can make equivalent changes or replacements to the related technical features without departing from the principles of the present application, and the technical schemes after the changes or replacements will all fall within the protection scope of the present application.

Claims

1. An artificial intelligence based mechanical automated assembly system characterized in that, Comprise: a perception positioning module, configured to acquire pose data of an assembly executor and environment state in real time, the pose data comprising three-dimensional coordinates and attitude angles of each observation point, and the environment state comprising illumination intensity and environment image; a pre-analysis module, connected with the perception positioning module, configured to construct an expected twin model based on expected pose, analyze expected execution trajectory of the assembly executor in a running period based on the expected twin model, comprising setting a plurality of observation points, determining expected execution trajectory of the observation points in each sub-period, and evaluating joint trajectory dispersion based on differences between expected execution trajectories; a pre-control module, connected with the pre-analysis module, configured to classify each sub-period based on joint trajectory dispersion in each sub-period, and monitor the assembly executor based on classification results, comprising, for a low-dispersion sub-period, determining a set of monitoring time points, monitoring only pose data of corresponding monitoring time points, and determining whether the assembly executor is abnormal; for a high-dispersion sub-period, acquiring pose data of the assembly executor in real time, constructing execution trajectory based on the pose data, determining tracking error between the execution trajectory and the expected execution trajectory, and dynamically generating an error tolerance threshold based on the environment state, comprising, adjusting an illumination-sensitive component of the error tolerance threshold based on illumination intensity; adjusting an environment-sensitive component of the error tolerance threshold based on environment image interference; weighting and summing the illumination-sensitive component and the environment-sensitive component to obtain a tolerance adjustment coefficient, and determining a product of the tolerance adjustment coefficient and a reference error threshold as the error tolerance threshold; determining whether the assembly executor is abnormal based on the tracking error and the error tolerance threshold; wherein the environment image interference is calculated based on a chroma difference between the environment image and the assembly executor.

2. The artificial intelligence based mechanical automated assembly system as claimed in claim 1 wherein, The pre-analysis module is configured to construct an expected twin model based on expected pose, comprising, defining expected three-dimensional coordinates and expected attitude angles of a plurality of observation points of the assembly executor at each time according to a current assembly task; establishing a model of the assembly executor in a virtual environment, and mapping the expected pose data to the model to generate an expected twin model reflecting an ideal assembly process; wherein the observation points are selected based on movable joints of the assembly executor.

3. The artificial intelligence based mechanical automated assembly system as claimed in claim 1 wherein, The pre-analysis module is configured to analyze execution actions of the assembly executor in a running period based on the expected twin model, comprising, dividing the running period of the assembly executor into a plurality of continuous sub-periods, and extracting expected execution trajectories of each observation point in each sub-period based on the expected twin model.

4. The artificial intelligence based mechanical automated assembly system as claimed in claim 3, wherein, The pre-analysis module is configured to evaluate joint trajectory dispersion based on differences between execution trajectories, comprising, calculating trajectory similarity between expected execution trajectories of each observation point in each sub-period; calculating a mean value of the trajectory similarity, and determining an inverse of the mean value as joint trajectory dispersion.

5. The artificial intelligence based mechanical automated assembly system as claimed in claim 4, wherein, The pre-control module is configured to classify each sub-period based on joint trajectory dispersion in each sub-period, comprising, comparing joint trajectory dispersion in each sub-period with a preset dispersion threshold; If the joint trajectory dispersion is greater than or equal to a preset dispersion threshold, a corresponding sub-period section is marked as high dispersion; If the joint trajectory dispersion is less than the preset dispersion threshold, a corresponding sub-period section is marked as low dispersion.

6. The artificial intelligence based mechanical automated assembly system as claimed in claim 1, wherein, The pre-control module is configured to determine a set of monitoring time points, including, for a low dispersion sub-period section, determining monitoring time points at a predetermined interval, and storing the monitoring time points as the set of monitoring time points.

7. The artificial intelligence based mechanical automated assembly system as claimed in claim 1 wherein, The pre-control module only monitors the pose data at the corresponding monitoring time points, and determines whether the assembly executor has an abnormality, including, calculating an instantaneous deviation amount of the pose data and the expected pose data at each monitoring time point, the instantaneous deviation amount being calculated based on a coordinate deviation and an attitude angle deviation; if the instantaneous deviation amount is greater than or equal to a preset tolerance threshold, determining that the assembly executor has an abnormality.

8. The artificial intelligence based mechanical automated assembly system as claimed in claim 1 wherein, The pre-control module constructs an execution trajectory based on the pose data, and determines a tracking error between the execution trajectory and an expected execution trajectory, including, constructing the execution trajectory by real-time acquiring the pose data within a plurality of time domain sections of the sub-period section, and synchronously acquiring the corresponding expected execution trajectory within the time domain section; determining a trajectory similarity of the execution trajectory and the expected execution trajectory as the tracking error.

9. The artificial intelligence based mechanical automated assembly system as claimed in claim 8, wherein, The pre-control module determines whether the assembly executor has an abnormality based on the tracking error and an error tolerance threshold, including, if the tracking error is greater than the current error tolerance threshold, determining that the assembly executor has an abnormality.

Images