Multi-axis servo vision fused AI unmanned tool cabinet management system and method

Abstract

The invention discloses a multi-axis servo visual fusion AI unmanned tool cabinet management system and method, and belongs to the technical field of tool cabinet control management, and the system comprises a motor operation monitoring module, a camera exposure triggering module and a visual positioning compensation module. The method can predict the image jitter amplitude caused by motor operation, triggers the dynamic peak shifting exposure of the camera in the stable phase of the commutation gap, reduces the vibration interference, improves the image collection quality, combines image jitter evaluation and phase difference analysis, dynamically compensates the visual positioning result, eliminates the displacement deviation, and improves the tool recognition and positioning precision. And high efficiency and reliability of tool access in unmanned management are guaranteed.

Description

Technical Field

[0001] The present invention relates to the technical field of tool cabinet control and management, and in particular to an AI unmanned tool cabinet management system and method based on multi-axis servo vision fusion. Background Art

[0002] Small tools such as sockets, screwdrivers, and adapters play a crucial role in modern industrial production and various on-site operations. These tools, characterized by their diverse variety, large quantity, and compact size, are widely used in many aspects of machinery assembly, repair, and daily maintenance, providing key support for the smooth progress of work processes.

[0003] For example, the invention patent with announcement number CN110687843B discloses a ZYNQ-based multi-axis multi-motor servo device and its control method, which includes a visual acquisition module, an interface module, a visual parallel segmentation module, a visual positioning processing module, a multi-processor collaboration module, a control algorithm and control instruction generation module, an external application algorithm module, a drive signal generation module and a drive circuit board; the ZYNQ chip includes PS0, PS1, DDR, FPGA and DSP; the visual information is collected by the camera, and the relative position of the target is determined by the visual parallel segmentation module and the visual positioning processing module, and a four-loop control instruction is generated according to the relative position signal of the target and the feedback current, speed and displacement signals, and the external control signal is received to generate the final control instruction, and then the drive signal is generated, and the multiple motors are controlled and fed back through the drive circuit board.

[0004] For example, the invention patent announcement with announcement number: CN106292525B discloses a dual-arm multi-head visual recognition placement machine control system and method. The system is provided with left and right zone communication channels on an industrial mainboard. The CPU of the industrial mainboard is connected to the left feed and discharge controller, the left servo controller, the left placement head control module, and the left suction nozzle automatic replacement controller through the left zone communication channel, and is connected to the right feed and discharge controller, the right servo controller, the right placement head control module, and the right suction nozzle automatic replacement controller through the right zone communication channel. The CPU is also connected to the alarm and detection system.

[0005] However, in the process of implementing the technical solutions of the invention in the embodiments of the present application, the present application found that the above technology has at least the following technical problems:

[0006] While existing multi-axis collaborative control methods based on visual servoing can achieve a certain level of target positioning and grasping accuracy, in scenarios involving high-precision small tool handling, vibration-induced pixel-level offsets can cause edge detection algorithms to produce contour distortion and feature matching algorithms to mismatch. This in turn leads to deviations in the robotic arm's grasping coordinate calculations, resulting in misaligned grasping of small tools and collisions with tool cabinet walls. Furthermore, high-frequency vibrations can cause the dynamic response of the visual system to lag behind the motor's motion, leading to a cumulative effect of positioning errors. This can lead to disorganized placement or even tool drops, especially during rapid and continuous tool placement, reducing the reliability and efficiency of unmanned tool cabinet operations. Summary of the Invention

[0007] The first aspect of the present invention provides an AI unmanned tool cabinet management system with multi-axis servo vision fusion, comprising:

[0008] The motor operation monitoring module is used to collect the operating parameters of the multi-axis servo motor in the tool cabinet, thereby predicting the image jitter amplitude through AI and simultaneously determining the commutation gap based on the three-phase current of the multi-axis servo motor.

[0009] The camera exposure trigger module is used to analyze the operating status information of the multi-axis servo motor after the commutation gap starts, determine the commutation gap stable stage, and execute the camera exposure of the visual camera in the tool cabinet based on the image jitter amplitude prediction result.

[0010] The visual positioning compensation module is used to determine the image pre-capture duration based on the image jitter amplitude prediction results, conduct image test acquisition, analyze the image jitter evaluation value, determine the compensation offset based on the image jitter amplitude prediction results, and dynamically compensate the visual positioning results.

[0011] A second aspect of the present invention provides an AI unmanned tool cabinet management method based on multi-axis servo vision fusion, comprising the following steps:

[0012] S1 collects the operating parameters of the multi-axis servo motor in the tool cabinet, and uses AI to predict the image jitter amplitude. Simultaneously, the commutation gap is determined based on the three-phase current of the multi-axis servo motor.

[0013] S2, after the commutation gap starts, analyze the operating status information of the multi-axis servo motor and determine the commutation gap stable stage. Based on this, the camera exposure of the visual camera in the tool cabinet is executed in combination with the image jitter amplitude prediction result.

[0014] S3, based on the image jitter amplitude prediction result, determine the image pre-capture duration, conduct image test capture, analyze the image jitter evaluation value, and determine the compensation offset in combination with the image jitter amplitude prediction result to dynamically compensate the visual positioning result.

[0015] One or more technical solutions provided in the present invention have at least the following technical effects or advantages:

[0016] 1. The multi-axis servo-vision fusion AI unmanned tool cabinet management system provided by the present invention can predict the image jitter amplitude caused by motor operation, trigger the camera's dynamic staggered exposure during the commutation gap stabilization phase, reduce vibration interference, and improve image acquisition quality. Combined with image jitter evaluation and phase difference analysis, it dynamically compensates for visual positioning results, eliminates displacement deviations, improves tool recognition and positioning accuracy, and ensures the efficiency and reliability of tool access in unmanned management.

[0017] 2. The present invention executes camera exposure of the visual camera in the tool cabinet by combining the image jitter amplitude prediction results, triggers camera exposure during the commutation gap of the servo motor, and realizes dynamic staggered exposure. It can effectively avoid vibration interference during the operation of the motor, reduce image jitter, improve image acquisition quality, and thus enhance the accuracy and reliability of visual positioning, ensuring that the tool cabinet management system can accurately identify and locate tools during unmanned operation, and ensuring the stability and efficiency of the system operation.

[0018] 3. The present invention dynamically compensates the visual positioning results, determines the compensation offset based on the image jitter amplitude prediction result and the image jitter evaluation value, and dynamically adjusts the compensation direction based on the phase difference between the vibration acceleration and the displacement of the image feature points, thereby reducing the image displacement deviation caused by the operation vibration of the multi-axis servo motor, making the visual positioning results more accurately reflect the actual position of the tool, avoiding the recognition error caused by mechanical vibration, and improving the reliability and accuracy of the tool cabinet management system in accessing and positioning tools during unmanned operation. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] Figure 1 Schematic diagram of the structure of the AI ​​unmanned tool cabinet management system with multi-axis servo vision fusion provided in the embodiment of the present application.

[0020] Figure 2 Flowchart of the AI ​​unmanned tool cabinet management method with multi-axis servo vision fusion provided in an embodiment of the present application.

[0021] Figure 3 This is a flow chart for determining the commutation gap of a multi-axis servo vision motor involved in an embodiment of the present application.

[0022] Figure 4 This is a flowchart for determining the stability of the commutation gap of a multi-axis servo vision motor involved in an embodiment of the present application. DETAILED DESCRIPTION

[0023] The technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, rather than all the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present invention.

[0024] Reference Figure 1 As shown, the first aspect of the present invention provides an AI unmanned tool cabinet management system with multi-axis servo vision fusion, comprising:

[0025] The motor operation monitoring module is used to collect the operating parameters of the multi-axis servo motor in the tool cabinet, thereby predicting the image jitter amplitude through AI and simultaneously determining the commutation gap based on the three-phase current of the multi-axis servo motor.

[0026] In this embodiment, the operating parameters of the multi-axis servo motor in the tool cabinet are collected, and the image jitter amplitude is predicted through AI. The specific analysis method is as follows:

[0027] During the preset monitoring period, the operating parameters of the multi-axis servo motor in the tool cabinet are collected, including the current fluctuation amplitude, vibration acceleration amplitude, motor speed change amplitude and robot arm strain amplitude.

[0028] It should be noted that the amplitude refers to the difference between the maximum value and the minimum value within the preset monitoring period.

[0029] The difference between the maximum and minimum currents in the three phases of the multi-axis servo motor is recorded as the current fluctuation amplitude of each phase, and the maximum current fluctuation amplitude of each phase is recorded as the current fluctuation amplitude.

[0030] The robotic arm is a robotic arm that connects a multi-axis servo motor and a vision module.

[0031] It should also be noted that the operating parameters of the multi-axis servo motor can be collected through sensing devices, including current sensors, acceleration sensors, motor encoders and strain gauges.

[0032] It's important to note that there's a correlation between the current fluctuation amplitude, vibration acceleration amplitude, motor speed variation amplitude, and manipulator strain amplitude. Changes in the motor current fluctuation amplitude typically cause changes in the vibration acceleration amplitude, as current fluctuations affect the motor's electromagnetic field and torque output, generating mechanical vibration. Increases in the vibration acceleration amplitude can lead to increases in the manipulator strain amplitude, as vibrations are transmitted to the manipulator, causing deformation. Furthermore, the motor speed variation amplitude is also related to current fluctuations and vibration acceleration, as unstable speed can lead to increased current fluctuations and vibration.

[0033] The reference current fluctuation amplitude, reference vibration acceleration amplitude, reference motor speed change amplitude and reference robotic arm strain amplitude stored in the database are extracted.

[0034] The current fluctuation amplitude characteristic distribution coefficient, vibration acceleration amplitude characteristic distribution coefficient, motor speed change amplitude characteristic distribution coefficient and robotic arm strain amplitude characteristic distribution coefficient preset in the database are extracted.

[0035] It should be noted that the current fluctuation amplitude characteristic distribution coefficient, the vibration acceleration amplitude characteristic distribution coefficient, the motor speed change amplitude characteristic distribution coefficient, and the robotic arm strain amplitude characteristic distribution coefficient all have a value range of 0-1, and the sum of the current fluctuation amplitude characteristic distribution coefficient, the vibration acceleration amplitude characteristic distribution coefficient, the motor speed change amplitude characteristic distribution coefficient, and the robotic arm strain amplitude characteristic distribution coefficient is 1. When used, the pre-set value can be directly extracted from the database. The specific extraction method is, for example, to construct a one-to-one mapping set of the current fluctuation amplitude, the vibration acceleration amplitude, the motor speed change amplitude, and the robotic arm strain amplitude with the corresponding current fluctuation amplitude characteristic distribution coefficient, the vibration acceleration amplitude characteristic distribution coefficient, the motor speed change amplitude characteristic distribution coefficient, and the robotic arm strain amplitude characteristic distribution coefficient. When used, the obtained current fluctuation amplitude, the vibration acceleration amplitude, the motor speed change amplitude, and the robotic arm strain amplitude are respectively input into the corresponding mapping set, thereby extracting the current fluctuation amplitude characteristic distribution coefficient, the vibration acceleration amplitude characteristic distribution coefficient, the motor speed change amplitude characteristic distribution coefficient, and the robotic arm strain amplitude characteristic distribution coefficient.

[0036] The operating state characteristic values ​​of the multi-axis servo motor are analyzed based on the operating parameters of the multi-axis servo motor.

[0037] The operating state characteristic value of the multi-axis servo motor is a quantitative indicator of the degree to which the current fluctuation amplitude, vibration acceleration amplitude, motor speed change amplitude and robot arm strain amplitude jointly affect the operating stability of the multi-axis servo motor. The specific analysis process is as follows: the current fluctuation amplitude, vibration acceleration amplitude, motor speed change amplitude and robot arm strain amplitude are compared with the corresponding reference values ​​respectively, and the comparison results are coupled with the corresponding characteristic distribution coefficient to obtain the operating state characteristic value of the multi-axis servo motor.

[0038] In a specific embodiment, the operating state characteristic value of the multi-axis servo motor is specifically expressed as follows:

[0039]

[0040] Among them, A is the operating state characteristic value of the multi-axis servo motor, ΔI is the current fluctuation amplitude, Δa is the vibration acceleration amplitude, Δv is the motor speed change amplitude, Δf is the mechanical arm strain amplitude, ΔIvef is the reference current fluctuation amplitude, Δa vef is the reference vibration acceleration amplitude, Δv vef is the reference motor speed variation amplitude, Δf vef is the reference manipulator strain amplitude, α1 is the current fluctuation amplitude characteristic distribution coefficient, α2 is the vibration acceleration amplitude characteristic distribution coefficient, α3 is the motor speed change amplitude characteristic distribution coefficient, and α4 is the manipulator strain amplitude characteristic distribution coefficient.

[0041] An image jitter amplitude is predicted using an AI algorithm according to the operating state characteristic value of the multi-axis servo motor to obtain a first image jitter amplitude.

[0042] In this embodiment, the AI ​​algorithm for predicting image jitter amplitude is a convolutional neural network.

[0043] See Figure 3 As shown, this is a flow chart for determining the commutation gap of a multi-axis servo visual motor involved in an embodiment of the present application. When determining the commutation gap based on the three-phase current of the multi-axis servo motor, the three-phase current of the servo motor is continuously monitored. When the amplitude of the current change in any phase exceeds the threshold, current data sampling and recording is triggered to determine the starting time of commutation, predict the duration of the current commutation, and obtain the end time of commutation and record it as the starting time of the commutation gap.

[0044] In this embodiment, the commutation gap is determined based on the three-phase current of the multi-axis servo motor. The specific analysis process is as follows:

[0045] Extract the three-phase current change thresholds preset in the database.

[0046] Monitor the three-phase current of the servo motor. When the change amplitude of any phase current exceeds the three-phase current change threshold, trigger current data sampling and recording. When the change amplitude of the three-phase current is greater than the three-phase current change threshold, and the current of two phases decreases, and the current of the remaining phase increases, the time point is recorded as the starting time of commutation.

[0047] Obtain the historical commutation duration of the multi-axis servo motor, use the AI ​​algorithm to obtain the current commutation prediction duration, and thus derive the commutation end time.

[0048] It should be noted that the historical commutation duration of the multi-axis servo motor refers to the corresponding motor commutation duration extracted from the system database based on a preset historical period.

[0049] In a specific embodiment, the AI ​​algorithm for obtaining the current commutation prediction duration is a convolutional neural network.

[0050] The commutation end time is recorded as the commutation gap start time.

[0051] The camera exposure trigger module is used to analyze the operating status information of the multi-axis servo motor after the commutation gap starts, determine the commutation gap stable stage, and execute the camera exposure of the visual camera in the tool cabinet based on the image jitter amplitude prediction result.

[0052] See Figure 4 As shown, this is a flow chart for judging the stability of the commutation gap of a multi-axis servo vision motor involved in an embodiment of the present application. When the commutation gap starts, the motor operating state is continuously monitored to obtain the operating state characteristic value, which is compared with the motor operating characteristic threshold. If the operating state characteristic value is less than the threshold, the operating state characteristic deviation value is calculated, and the number of motor stability verification monitoring times is extracted based on the deviation value. After the verification result is classified as stable or fluctuating state based on the threshold, it is judged whether the number of verified stable states is not less than the number of stable judgment verifications. If so, it is judged to enter the stable stage, otherwise the operating state is marked as fluctuating operation.

[0053] In this embodiment, the operating status information of the multi-axis servo motor is analyzed, and the specific process is as follows:

[0054] After the commutation gap begins, the operating state of the multi-axis servo motor is continuously monitored to obtain the operating state characteristic values ​​of the multi-axis servo motor corresponding to each monitoring period, which are recorded as the operating state characteristic values ​​of the multi-axis servo motor.

[0055] Extract the motor operation characteristic thresholds preset in the database.

[0056] If a certain operating state characteristic value is greater than or equal to the motor operating characteristic threshold, the current operating state information of the multi-axis servo motor is recorded as fluctuating operation.

[0057] If a certain operating state characteristic value is greater than or equal to the motor operating characteristic threshold, it means that although the current multi-axis servo motor is in the commutation gap, it has not yet completely stabilized. At this time, the operating state of the motor still fluctuates to a certain extent.

[0058] If a certain operating state characteristic value is less than the motor operating characteristic threshold, the current operating state information of the multi-axis servo motor is recorded as stable operation.

[0059] If a certain operating state characteristic value is less than the motor operating characteristic threshold, it means that the current multi-axis servo motor has successfully transitioned from the commutation stage to the stable operation stage, and all operating parameters fluctuate within the normal range.

[0060] In this embodiment, the commutation gap stable stage is determined, and the specific analysis steps are as follows:

[0061] When the current operating state information of the multi-axis servo motor is stable operation, the operating state characteristic value is subtracted from the motor operating characteristic threshold to obtain the operating state characteristic deviation value.

[0062] The number of motor stability verification monitoring times is extracted based on the characteristic deviation value of the operating status.

[0063] It should be noted that the database stores a mapping set of motor stability verification monitoring times and operating status characteristic deviation values. When used, the operating status characteristic deviation value obtained in real time is input into the mapping set to extract the motor stability verification monitoring times.

[0064] It should also be noted that the larger the deviation value of the operating state characteristic, the more unstable the motor's operating state and the more significant the fluctuation. To more accurately verify whether the motor has truly reached a stable operating state, sampling and monitoring are required over a larger range, so more motor stability verification monitoring times are required. This allows for multiple monitoring cycles to fully capture changes in the motor's operating state, ensuring sufficient data to determine stable operation.

[0065] The running state verification monitoring of the multi-axis servo motor is performed based on the number of motor stability verification monitoring times, and the characteristic values ​​of each running state are obtained and recorded as each verification running state characteristic value.

[0066] Based on the motor operation characteristic threshold and each verification operation state characteristic value, the verification monitoring results are classified into verification fluctuation state and verification stable state. Specifically: the verification monitoring result whose verification operation state characteristic value is less than the motor operation characteristic threshold is recorded as verification stable state, and the verification monitoring result whose verification operation state characteristic value is greater than or equal to the motor operation characteristic threshold is recorded as verification fluctuation state.

[0067] The stability determination verification number is obtained based on the stability determination proportional factor preset in the database and the number of motor stability verification monitoring times, specifically: the product of the stability determination proportional factor and the number of motor stability verification monitoring times is used as the stability determination verification number.

[0068] If the verified stable state number is greater than or equal to the stable determination verification number, it is determined that the commutation gap of the multi-axis servo motor enters the stable stage.

[0069] If the number of verified stable states is greater than or equal to the number of stable judgment verifications, it means that the current operating state of the multi-axis servo motor is stable in multiple continuous monitorings, the fluctuation amplitude is controlled within an acceptable range, and the motor commutation gap has smoothly transitioned to the stable operation stage. It can be considered that the motor has met the requirements for stable operation.

[0070] If the verified stable state number is less than the stable determination verification number, it is determined that the commutation gap of the multi-axis servo motor is still in the fluctuation stage, and the current operating state information of the multi-axis servo motor is re-recorded as fluctuating operation.

[0071] If the number of verified stable states is less than the number of stable judgment verifications, it means that the current operating state of the multi-axis servo motor is not stable enough and there is still a certain degree of fluctuation. The conditions for stable operation are not met and the motor commutation gap is still in the fluctuation stage.

[0072] In this embodiment, the camera exposure of the visual camera in the tool cabinet is performed in combination with the image jitter amplitude prediction result. The specific analysis process is as follows:

[0073] The exposure duration corresponding to each first image jitter amplitude interval stored in the database is extracted, and the exposure duration corresponding to the interval in which the first image jitter amplitude is located is mapped and extracted, and recorded as the camera exposure duration of the visual camera.

[0074] The camera exposure is triggered when the commutation gap of the multi-axis servo motor enters the stable stage.

[0075] The camera exposure of the vision camera in the tool cabinet is performed with the camera exposure duration of the vision camera.

[0076] The visual positioning compensation module is used to determine the image pre-capture duration based on the image jitter amplitude prediction results, conduct image test acquisition, analyze the image jitter evaluation value, determine the compensation offset based on the image jitter amplitude prediction results, and dynamically compensate the visual positioning results.

[0077] In this embodiment, the image jitter evaluation value is analyzed, and the specific analysis process is as follows:

[0078] The image pre-capture duration is determined based on the image jitter amplitude prediction result, and image test capture is performed to obtain each test image frame.

[0079] It should be noted that the database stores a mapping set of first image jitter amplitude and image pre-capture duration. When in use, the first image jitter amplitude acquired in real time is input into the mapping set to extract the image pre-capture duration.

[0080] Image jitter state data is obtained based on each test image frame, including average displacement change, average gray value change and average structural similarity.

[0081] It should be noted that the average displacement change refers to the average value of the displacement change of the feature points in each test image frame, the average grayscale value change refers to the average value of the grayscale value change in each test image frame, and the average structural similarity refers to the average value of the similarity between two adjacent image frames.

[0082] In a specific embodiment, the image shaking state data can be obtained through OpenCV analysis.

[0083] It should be noted that mean displacement change, mean grayscale change, and mean structural similarity are all important metrics for measuring the stability and quality of image sequences, and there is a certain degree of correlation between them. Mean displacement change reflects the degree of change in the position of feature points in the image. Large displacement changes usually indicate significant image jitter or motion. Mean grayscale change reflects fluctuations in image brightness and contrast, which may be caused by changes in lighting or image jitter. Mean structural similarity measures the similarity of structural information between images. Decreased structural similarity may be related to displacement changes, grayscale fluctuations, and changes in image content. Generally, when mean displacement change and mean grayscale change are large, mean structural similarity decreases, indicating that the stability and consistency of the image sequence are deteriorating.

[0084] The reference average displacement change, reference average gray value change and reference average structural similarity stored in the database are extracted.

[0085] The average displacement variation feature allocation coefficient, average gray value variation feature allocation coefficient and average structural similarity feature allocation coefficient preset in the database are extracted.

[0086] It should be noted that the value ranges of the average displacement change feature distribution coefficient, the average gray value change feature distribution coefficient and the average structural similarity feature distribution coefficient are all between 0 and 1, and the sum of the average displacement change feature distribution coefficient, the average gray value change feature distribution coefficient and the average structural similarity feature distribution coefficient is 1. When used, the pre-set value can be directly extracted from the database. The specific extraction method is, for example, to construct a one-to-one mapping set of the average displacement change, the average gray value change and the average structural similarity with the corresponding feature distribution coefficient. When used, the obtained average displacement change, average gray value change and average structural similarity are respectively input into the corresponding mapping set, so as to extract the average displacement change feature distribution coefficient, the average gray value change feature distribution coefficient and the average structural similarity feature distribution coefficient.

[0087] An image shake evaluation value is analyzed based on the image shake state data.

[0088] The image jitter assessment value is a quantitative indicator of the impact of the average displacement change, average grayscale value change, and average structural similarity on the degree of image jitter. The specific analysis process is as follows: the average displacement change and average grayscale value change are compared with the corresponding reference values ​​respectively, and the reference value of average structural similarity is compared with the average structural similarity. The results of each comparison are combined with the corresponding feature allocation coefficient for coupling processing to obtain the image jitter assessment value.

[0089] In a specific embodiment, the image jitter evaluation value is specifically expressed as follows:

[0090]

[0091] Where B is the image jitter evaluation value, Δx is the average displacement change, ΔH is the average grayscale value change, is the average structural similarity, Δx vef is the reference average displacement change, ΔH vef is the reference average gray value change, is the reference average structural similarity, β1 is the average displacement change feature allocation coefficient, β2 is the average gray value change feature allocation coefficient, and β3 is the average structural similarity feature allocation coefficient.

[0092] In this embodiment, the compensation offset is determined in combination with the image jitter amplitude prediction result. The specific analysis process is as follows:

[0093] The image jitter amplitude corresponding to each image jitter evaluation value interval stored in the database is extracted, and the image jitter amplitude corresponding to the interval where the image jitter evaluation value is located is mapped and recorded as the second image jitter amplitude.

[0094] A first image jitter amplitude weight and a second image jitter amplitude weight preset in a database are extracted.

[0095] The image shake correction amplitude is analyzed based on the first image shake amplitude and the second image shake amplitude. The specific analysis process is: the first image shake amplitude and the second image shake amplitude are coupled with corresponding weights to obtain the image shake correction amplitude.

[0096] In a specific embodiment, the product of the first image shake amplitude and the first image shake amplitude weight plus the product of the second image shake amplitude and the second image shake amplitude weight is taken as the numerical result of the image shake correction amplitude.

[0097] The numerical result of the image shake correction amplitude is used as the compensation offset.

[0098] In this embodiment, dynamic compensation is performed on the visual positioning results. The specific analysis steps are as follows:

[0099] A1, during the image test acquisition process, the three-axis vibration acceleration data of the servo motor during operation and the coordinate offset of the image feature points under the synchronized timestamp are collected to establish a time-domain aligned vibration-displacement dataset.

[0100] A2: Perform frequency domain analysis on the vibration signal to extract the dominant frequency component. Analyze the phase difference between the vibration acceleration phase and the displacement of the image feature points based on the dominant frequency component.

[0101] A3: If the phase difference between the vibration acceleration phase and the image feature point displacement is in the first judgment interval, the compensation direction of the compensation offset is opposite to the vibration direction. If the phase difference between the vibration acceleration phase and the image feature point displacement is in the second judgment interval, the compensation direction of the compensation offset is the same as the vibration direction.

[0102] It should be noted that, in this embodiment, the first determination interval is (150° to 210°), and the second determination interval is (-30° to 30°).

[0103] It should be added that the numerical limits of the above-mentioned determination intervals are only examples and are not specifically limited. In specific embodiments, technicians can make targeted limits on the intervals based on actual image conditions.

[0104] It should also be noted that if the phase difference between the vibration acceleration phase and the image feature point displacement is neither in the first determination interval nor in the second determination interval, a prompt message is generated to prompt the operator to perform spectrum detection and wait for confirmation information.

[0105] In this embodiment, the compensation process of visual positioning can be implemented by Python.

[0106] In a specific embodiment, if the phase difference between the vibration acceleration phase and the image feature point displacement is in the first judgment interval, the tool coordinates obtained through visual camera acquisition and analysis are (x0, y0), and the compensation offset is τ, then after visual positioning compensation, the actual output tool coordinates are (x0-τ, y0-τ).

[0107] In another specific embodiment, if the phase difference between the vibration acceleration phase and the displacement of the image feature point is in the second judgment interval, the tool coordinates obtained by the visual camera acquisition and analysis are (x0, y0), and the compensation offset is τ, then after visual positioning compensation, the actual output tool coordinates are (x0+τ, y0+τ).

[0108] See Figure 2 As shown, the second aspect of the present invention provides an AI unmanned tool cabinet management method based on multi-axis servo vision fusion, comprising the following steps:

[0109] S1 collects the operating parameters of the multi-axis servo motor in the tool cabinet, and uses AI to predict the image jitter amplitude. Simultaneously, the commutation gap is determined based on the three-phase current of the multi-axis servo motor.

[0110] S2, after the commutation gap starts, analyze the operating status information of the multi-axis servo motor and determine the commutation gap stable stage. Based on this, the camera exposure of the visual camera in the tool cabinet is executed in combination with the image jitter amplitude prediction result.

[0111] S3, based on the image jitter amplitude prediction result, determine the image pre-capture duration, conduct image test capture, analyze the image jitter evaluation value, and determine the compensation offset in combination with the image jitter amplitude prediction result to dynamically compensate the visual positioning result.

[0112] It will be understood by those skilled in the art that embodiments of the present invention may be provided as methods, systems, or computer program products. Thus, the present invention may take the form of an entirely hardware embodiment, an entirely software embodiment, or an embodiment combining software and hardware. Furthermore, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to magnetic disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0113] The present invention is described with reference to flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to embodiments of the present invention. It should be understood that each process and / or block in the flowcharts and / or block diagrams, as well as combinations of processes and / or blocks in the flowcharts and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowcharts and / or block diagrams. Figure 1 a process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0114] These computer program instructions may also be stored in a computer readable memory that can direct a computer or other programmable data processing device to work in a specific manner, so that the instructions stored in the computer readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 a process or multiple processes and / or boxes Figure 1 The function specified in one or more boxes.

[0115] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operational steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing the instructions executed on the computer or other programmable device for implementing the process. Figure 1 a process or multiple processes and / or boxes Figure 1 A step that specifies a function in one or more boxes.

[0116] Although the preferred embodiments of the present invention have been described, those skilled in the art may make additional changes and modifications to these embodiments once they have learned the basic creative concept. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments and all changes and modifications that fall within the scope of the present invention.

[0117] Obviously, those skilled in the art may make various changes and modifications to the present invention without departing from the spirit and scope of the present invention. Thus, if such changes and modifications fall within the scope of the claims and their equivalents, the present invention is intended to include such changes and modifications.

Claims

1. The AI ​​unmanned tool cabinet management system with multi-axis servo vision fusion is characterized by: The following steps are involved: The motor operation monitoring module is used to collect the operating parameters of the multi-axis servo motor in the tool cabinet, thereby predicting the image jitter amplitude through AI and simultaneously determining the commutation gap based on the three-phase current of the multi-axis servo motor; The camera exposure trigger module is used to analyze the operating status information of the multi-axis servo motor after the commutation gap begins, determine the commutation gap stability stage, and then perform camera exposure of the visual camera in the tool cabinet based on the image jitter amplitude prediction result; The visual positioning compensation module is used to determine the image pre-capture duration based on the image jitter amplitude prediction results, conduct image test acquisition, analyze the image jitter evaluation value, determine the compensation offset based on the image jitter amplitude prediction results, and dynamically compensate the visual positioning results.

2. The multi-axis servo-vision fusion AI unmanned tool cabinet management system according to claim 1, characterized in that: The operating parameters of the multi-axis servo motor in the tool cabinet are collected, and the image jitter amplitude is predicted through AI. The specific analysis method is as follows: During a preset monitoring period, the operating parameters of the multi-axis servo motor in the tool cabinet are collected, including current fluctuation amplitude, vibration acceleration amplitude, motor speed change amplitude, and robot arm strain amplitude; Analyze the operating state characteristic values ​​of the multi-axis servo motor based on its operating parameters; The operating state characteristic value of the multi-axis servo motor is a quantitative index of the degree of influence of the current fluctuation amplitude, the vibration acceleration amplitude, the motor speed change amplitude and the mechanical arm strain amplitude on the operating stability of the multi-axis servo motor. The specific analysis process is: the current fluctuation amplitude, the vibration acceleration amplitude, the motor speed change amplitude and the mechanical arm strain amplitude are respectively compared with the corresponding reference values, and the comparison results are combined with the corresponding characteristic distribution coefficient for coupling processing to obtain the operating state characteristic value of the multi-axis servo motor; An image jitter amplitude is predicted using an AI algorithm according to the operating state characteristic value of the multi-axis servo motor to obtain a first image jitter amplitude.

3. The multi-axis servo-vision fusion AI unmanned tool cabinet management system according to claim 1, characterized in that: The commutation gap determination based on the three-phase current of the multi-axis servo motor is analyzed in detail as follows: Extracting the three-phase current change threshold preset in the database; Monitor the three-phase current of the servo motor. When the amplitude of the current change of any phase exceeds the three-phase current change threshold, trigger current data sampling and recording. When the amplitude of the current change of the three phases is greater than the three-phase current change threshold, and the current of two phases decreases, and the current of the remaining phase increases, this time point is recorded as the commutation start time; Obtain the historical commutation duration of the multi-axis servo motor, obtain the current commutation prediction duration through AI algorithm, and thus derive the commutation end time; The commutation end time is recorded as the commutation gap start time.

4. The multi-axis servo-vision fusion AI unmanned tool cabinet management system according to claim 1, characterized in that: The specific process of analyzing the operating status information of the multi-axis servo motor is as follows: After the commutation gap begins, the operating state of the multi-axis servo motor is continuously monitored to obtain the operating state characteristic value of the multi-axis servo motor corresponding to each monitoring period, which is recorded as each operating state characteristic value of the multi-axis servo motor; Extracting motor operation characteristic thresholds preset in the database; If a certain operating state characteristic value is greater than or equal to the motor operating characteristic threshold, the operating state information of the current multi-axis servo motor is recorded as fluctuating operation; If a certain operating state characteristic value is less than the motor operating characteristic threshold, the current operating state information of the multi-axis servo motor is recorded as stable operation.

5. The multi-axis servo-vision fusion AI unmanned tool cabinet management system according to claim 4, characterized in that: The specific analysis steps for determining the stable stage of the commutation gap are as follows: When the current operating state information of the multi-axis servo motor is stable operation, the operating state characteristic value is subtracted from the motor operating characteristic threshold to obtain the operating state characteristic deviation value; Extract the number of motor stability verification monitoring times based on the characteristic deviation value of the operating status; Performing operation state verification monitoring of the multi-axis servo motor according to the number of motor stability verification monitoring times, obtaining characteristic values ​​of each operation state, and recording them as characteristic values ​​of each verification operation state; Classifying the verification monitoring results into verification fluctuation state and verification stable state based on the motor operation characteristic threshold and each verification operation state characteristic value; Obtaining a stability determination verification number based on a stability determination proportional factor preset in a database and a number of motor stability verification monitoring times; If the number of verified stable states is greater than or equal to the number of stable determination verifications, it is determined that the commutation gap of the multi-axis servo motor has entered a stable stage; If the verified stable state number is less than the stable determination verification number, it is determined that the commutation gap of the multi-axis servo motor is still in the fluctuation stage, and the current operating state information of the multi-axis servo motor is re-recorded as fluctuating operation.

6. The multi-axis servo-vision fusion AI unmanned tool cabinet management system according to claim 1, characterized in that: The camera exposure of the visual camera in the tool cabinet is performed by combining the image jitter amplitude prediction result. The specific analysis process is as follows: Extracting the exposure duration corresponding to each first image jitter amplitude interval stored in the database, and mapping and extracting the exposure duration corresponding to the interval in which the first image jitter amplitude falls, and recording it as the camera exposure duration of the visual camera; Trigger camera exposure when the commutation gap of the multi-axis servo motor enters a stable stage; The camera exposure of the vision camera in the tool cabinet is performed with the camera exposure duration of the vision camera.

7. The multi-axis servo-vision fusion AI unmanned tool cabinet management system according to claim 1, characterized in that: The specific analysis process of analyzing the image jitter evaluation value is as follows: Determine the image pre-capture duration based on the image jitter amplitude prediction result, and perform image test capture to obtain each test image frame; Obtain image jitter state data based on each test image frame, including average displacement change, average gray value change, and average structural similarity; Analyzing an image jitter assessment value based on the image jitter state data; The image jitter assessment value is a quantitative indicator of the impact of the average displacement change, average grayscale value change, and average structural similarity on the degree of image jitter. The specific analysis process is as follows: the average displacement change and average grayscale value change are compared with their corresponding reference values, and the reference value of the average structural similarity is compared with the average structural similarity. The results of each comparison process are combined with the corresponding feature allocation coefficient for coupling processing to obtain the image jitter assessment value.

8. The multi-axis servo-vision fusion AI unmanned tool cabinet management system according to claim 7, characterized in that: The compensation offset is determined by combining the image jitter amplitude prediction result. The specific analysis process is as follows: Extracting the image jitter amplitude corresponding to each image jitter evaluation value interval stored in the database, and mapping the image jitter amplitude corresponding to the interval in which the extracted image jitter evaluation value is located, and recording it as the second image jitter amplitude; Extracting a first image jitter amplitude weight and a second image jitter amplitude weight preset in a database; Analyzing the image shake correction amplitude based on the first image shake amplitude and the second image shake amplitude, wherein the specific analysis process is: coupling the first image shake amplitude and the second image shake amplitude with corresponding weights to obtain the image shake correction amplitude; The numerical result of the image shake correction amplitude is used as the compensation offset.

9. The multi-axis servo-vision fusion AI unmanned tool cabinet management system according to claim 1, characterized in that: The dynamic compensation of the visual positioning results is performed, and the specific analysis steps are as follows: A1, during the image test acquisition process, collect the three-axis vibration acceleration data of the servo motor when it is running and the coordinate offset of the image feature points under the synchronized time stamp to establish a time-domain aligned vibration-displacement dataset; A2: Perform frequency domain analysis on the vibration signal to extract the dominant frequency component. Analyze the phase difference between the vibration acceleration phase and the displacement of the image feature points based on the dominant frequency component. A3: If the phase difference between the vibration acceleration phase and the image feature point displacement is in the first judgment interval, the compensation direction of the compensation offset is opposite to the vibration direction. If the phase difference between the vibration acceleration phase and the image feature point displacement is in the second judgment interval, the compensation direction of the compensation offset is the same as the vibration direction.

10. The method for the AI ​​unmanned tool cabinet management system based on multi-axis servo vision fusion as claimed in any one of claims 1 to 9 is characterized in that: include: S1, collects the operating parameters of the multi-axis servo motor in the tool cabinet, uses AI to predict the image jitter amplitude, and simultaneously determines the commutation gap based on the three-phase current of the multi-axis servo motor; S2, after the commutation gap starts, analyze the operating status information of the multi-axis servo motor to determine the commutation gap stable stage, and then perform camera exposure of the visual camera in the tool cabinet in combination with the image jitter amplitude prediction result; S3, based on the image jitter amplitude prediction result, determine the image pre-capture duration, conduct image test capture, analyze the image jitter evaluation value, and determine the compensation offset in combination with the image jitter amplitude prediction result to dynamically compensate the visual positioning result.

Images