A multi-station instrument visual detection mechanical arm control method, system, device and medium

Abstract

The present application relates to the technical field of intelligent manufacturing and robot autonomous control, and particularly relates to a multi-station instrument visual detection mechanical arm control method, system, equipment and medium, which comprises time synchronization fusion on target data collected in real time to generate a high-dimensional state vector, wherein: the target data comprises physical data and visual state data; the high-dimensional state vector is input into an evaluation network to output fusion features; the fusion features are input into a strategy network to generate a mechanical arm action instruction by the strategy network to perform a multi-station instrument visual detection operation. The present application can effectively solve the technical problems of low sample efficiency, poor real-time performance and feature degradation in multi-station instrument visual detection.

Description

Technical Field

[0001] This invention relates to the field of intelligent manufacturing and autonomous robot control technology, and in particular to a control method, system, equipment and medium for a multi-station instrument vision inspection robotic arm. Background Technology

[0002] In existing industrial robotic arms used for multi-station instrument vision inspection, ultra-wide network structures are often employed for feature mapping to compensate for the insufficient feature recognition capabilities of standard networks. This leads to parameter redundancy and excessive computational load, and the single-step inference latency is insufficient to meet the requirements of high-frequency real-time control. Furthermore, batch normalization operations in deep networks can easily cause the loss or over-smoothing of high-precision signals from underlying physical sensors, resulting in gradient flow instability and feature degradation, thus reducing perception accuracy. Therefore, a new control method is urgently needed to achieve efficient, accurate, and low-latency automated operation. Summary of the Invention

[0003] To solve the above-mentioned technical problems, the present invention provides the following technical solution: In a first aspect, the present invention provides a control method for a multi-station instrument vision inspection robotic arm, including time-synchronized fusion of target data collected in real time to generate a high-dimensional state vector, wherein: the target data includes physical data and visual state data; The high-dimensional state vector is input into the evaluation network, which outputs fused features. The following operations are performed sequentially within the evaluation network: The high-dimensional state vector is decoupled into two parallel processing branches through a parallel branch channel attention mechanism. The main branch performs nonlinear feature extraction, while the attention branch independently calculates the dynamic gain weights of each feature channel. The gain weights output by the attention branch are multiplied channel by channel with the nonlinear features output by the main branch to obtain the recalibrated features. The recalibrated features are subjected to deep nonlinear mapping, and the mapping result is then weighted and fused with the high-dimensional state vector through a gated identity mapping path with learnable scalar parameters to obtain fused features. The learnable scalar parameters are used to dynamically balance the proportion of the high-dimensional state vector and the deep mapping result in the fused features. The fused features are input into the policy network, which generates motion commands for the robotic arm to perform visual inspection operations on the multi-station instrument.

[0004] As a preferred embodiment of the multi-station instrument vision inspection robotic arm control method of the present invention, wherein: the target data collected in real time is time-synchronized and fused to generate a high-dimensional state vector, including: The high-frequency physical signals collected in real time by the robotic arm joint encoder and the six-dimensional force sensor are used as the time reference signal; Within each control cycle, the nearest neighbor timestamp matching strategy is adopted to select the visual state data frame whose timestamp is closest to the current cycle time. A zero-order hold strategy is adopted to continuously reuse the currently selected visual state data before the arrival of the next visual frame, so as to complete the time alignment and fusion of high-frequency physical signals and low-frequency visual states to generate a high-dimensional state vector.

[0005] As a preferred embodiment of the multi-station instrument vision inspection robotic arm control method of the present invention, wherein: the attention branch independently calculates the dynamic gain weight of each feature channel, including: Global average pooling is performed on the input high-dimensional state vector to generate spatial feature vectors; The spatial feature vectors are sequentially passed through a dimension-reducing fully connected layer, a ReLU activation function, a dimension-upgrading fully connected layer, and a Sigmoid activation function to generate dynamic gain weight vectors for each feature channel.

[0006] As a preferred embodiment of the multi-station instrument vision inspection robotic arm control method of the present invention, the mapping result and the high-dimensional state vector are weighted and fused through a gated identity mapping path with learnable scalar parameters, including: The recalibrated features are sequentially input into at least one fully connected layer, and after nonlinear activation, a deep mapping result vector is obtained. A learnable scalar parameter is set, and the high-dimensional state vector and the deep mapping result vector are weighted and summed.

[0007] As a preferred embodiment of the multi-station instrument vision inspection robotic arm control method of the present invention, wherein: the motion commands of the robotic arm are generated by the strategy network, including: Configure the hidden layer dimension of the policy network to 256; Based on the JAX framework, operator fusion and parallel compilation are performed on the forward computation process of the policy network; Within each control cycle, the fused features are input into the compiled policy network, and the angular velocity or torque values ​​of each joint of the robotic arm are output as motion commands.

[0008] As a preferred embodiment of the multi-station instrument vision inspection robotic arm control method of the present invention, the evaluation network and the policy network are jointly trained using a targetless network architecture, including: Set the update to data ratio UTD=1 so that a network parameter update is performed immediately after each environmental interaction is completed.

[0009] As a preferred embodiment of the multi-station instrument vision inspection robotic arm control method of the present invention, the joint training further includes limiting the total number of training environment interaction steps to within 100,000 steps, and using the gradient of a single interaction data to update the strategy network and the evaluation network.

[0010] Secondly, the present invention provides a multi-station instrument vision inspection robotic arm control system, including: a fusion generation module, used to perform time-synchronous fusion of target data collected in real time to generate a high-dimensional state vector, wherein: the target data includes physical data and visual state data; The input / output module is used to input high-dimensional state vectors into the evaluation network and output fused features. The evaluation network performs the following operations sequentially: The high-dimensional state vector is decoupled into two parallel processing branches through a parallel branch channel attention mechanism. The main branch performs nonlinear feature extraction, while the attention branch independently calculates the dynamic gain weights of each feature channel. The gain weights output by the attention branch are multiplied channel by channel with the nonlinear features output by the main branch to obtain the recalibrated features. The recalibrated features are subjected to deep nonlinear mapping, and the mapping result is then weighted and fused with the high-dimensional state vector through a gated identity mapping path with learnable scalar parameters to obtain fused features. The learnable scalar parameters are used to dynamically balance the proportion of the high-dimensional state vector and the deep mapping result in the fused features. The generation and execution module is used to input the fused features into the policy network, which generates motion commands for the robotic arm to perform visual inspection operations on the multi-station instrument.

[0011] Thirdly, the present invention provides a computer device, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the steps of the method described above.

[0012] Fourthly, the present invention provides a computer-readable storage medium having a computer program stored thereon, wherein the computer program, when executed by a processor, implements the steps of the method described above.

[0013] Compared with existing technologies, the beneficial effects of this invention are as follows: By dynamically recalibrating heterogeneous sensor features through a parallel branch channel attention mechanism, the dilution problem of multi-source physical signals in deep networks can be effectively solved. Furthermore, by introducing a gated identity mapping path with learnable scalar parameters, not only can high-precision joint position and force signals be transmitted to the evaluation layer without loss, but the feature degradation and end jitter problems caused by batch normalization can also be avoided. Moreover, by adopting a lightweight strategy network in conjunction with JAX operator-level optimization, the single-step inference latency can be reduced to 0.18ms to meet the requirements of industrial-grade high-frequency real-time control. Attached Figure Description

[0014] To more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings used in the following description of the embodiments will be briefly introduced. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0015] Figure 1 This is a flowchart illustrating a multi-station instrument vision inspection robotic arm control method provided in one embodiment.

[0016] Figure 2 Overall architecture diagram of reinforcement learning control algorithm.

[0017] Figure 3 This is a structural diagram of the parallel branch channel attention mechanism and the learnable gating residual fusion mechanism. Detailed Implementation

[0018] To make the above-mentioned objects, features, and advantages of the present invention more apparent and understandable, specific embodiments of the present invention will be described in detail below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of the present invention, and not all of them. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort should fall within the protection scope of the present invention.

[0019] Example 1, referring to Figures 1-3 This is the first embodiment of the present invention, which provides a control method for a multi-station instrument vision inspection robotic arm, including: In current multi-station instrument visual inspection, there are issues such as the dilution of features from heterogeneous sensors and the difficulty in detecting high-precision objects. This application addresses the technical problems of signal degradation in deep networks and excessive inference latency caused by network parameter redundancy. It provides a method that effectively solves these problems, and the following sections will elaborate on how to implement this multi-station instrument vision inspection robotic arm control method using multiple embodiments. Figure 1 A flowchart of a control method for a multi-station instrument vision inspection robotic arm is shown, including: S100: Perform time-synchronized fusion on the target data collected in real time to generate a high-dimensional state vector, where the target data includes physical data and visual state data.

[0020] In one specific implementation, the equipment used in this method is a six-degree-of-freedom industrial robotic arm, with an industrial-grade eye-on camera and an adjustable-force pneumatic gripper at its end. The workbench is equipped with multiple independent instrument detection stations, and a binocular global vision system is fixedly deployed above the workbench to cover the entire multi-station workspace. Specifically: The high-frequency physical signals acquired in real time by the robotic arm joint encoders and six-dimensional force sensors serve as the time reference signal. It should be noted that high-precision encoders and six-dimensional force sensors built into each joint of the robotic arm will be used to acquire physical signal data such as joint angles, angular velocities, and end-effector contact torques in real time at a high sampling rate. Furthermore, a global binocular vision system / eye-on-hand camera will output visual state data such as the three-dimensional pose of the target instrument and the boundaries of environmental obstacles at a relatively low fixed frame rate.

[0021] Specifically: Within each control cycle, a nearest neighbor timestamp matching strategy is employed to select the visual state data frame whose timestamp is closest to the current cycle time. This involves calculating the absolute difference between the start time of the current control cycle and the timestamps of each visual state data frame, and using the frame with the smallest absolute difference as the matching result.

[0022] Furthermore, a zero-order hold strategy is adopted, continuously reusing the visual state data of the currently selected frame before the arrival of the next visual frame, completing the time alignment and fusion of the high-frequency physical signal and the low-frequency visual state, and generating a high-dimensional state vector. That is, in each high-frequency control cycle, only the visual state data remains unchanged, but the high-frequency physical signal is updated in real time with its own sampling rate.

[0023] In one optional implementation, the high-dimensional state vector can be generated using a time-aligned fusion method based on linear interpolation. Specifically, when the high-frequency physical signals output by the robotic arm joint encoder and the six-dimensional force sensor arrive, the system does not directly reuse the nearest visual frame. Instead, it selects two frames of visual data before and after the current physical signal time, performs linear weighted interpolation according to the time distance, and generates a virtual visual state frame that is strictly aligned with the physical signal time. Finally, the virtual visual frame is spliced ​​with the current physical signal to form a high-dimensional state vector.

[0024] In another alternative implementation, the high-dimensional state vector can also be generated using asynchronous independent sampling and state prediction compensation. That is, the system directly uses the latest available physical signals and visual data in each control cycle, and uses a lightweight time-series prediction model to predict the target instrument pose at the current moment based on the historical sequence of visual data. By replacing the actual visual observation value with the prediction result, it is finally concatenated with the physical signal to form a high-dimensional state vector.

[0025] In one specific implementation, the overall architecture of the reinforcement learning control algorithm is as follows: Figure 2 As shown, specifically: S200: Input the high-dimensional state vector into the evaluation network and output the fused features.

[0026] It should be noted that this method addresses the issue of feature dilution in heterogeneous sensor data during robotic arm path planning by performing a principle-level topology reconstruction of the evaluation network, specifically as follows: Figure 3 As shown in the upper part: S201: The high-dimensional state vector is decoupled into two parallel processing branches through a parallel branch channel attention mechanism. The main branch performs nonlinear feature extraction, while the attention branch independently calculates the dynamic gain weight of each feature channel and multiplies the gain weight output by the attention branch with the nonlinear feature output by the main branch channel by channel to obtain the recalibrated features.

[0027] Specifically, in this step, the attention branch independently calculates the dynamic gain weights for each feature channel, and also includes the following: A1: Perform global average pooling on the input high-dimensional state vector to generate spatial feature vectors.

[0028] A2: The spatial feature vectors are sequentially passed through a dimension-reducing fully connected layer, a ReLU activation function, a dimension-upgrading fully connected layer, and a Sigmoid activation function to generate dynamic gain weight vectors for each feature channel.

[0029] Specifically, the generated dynamic gain weight S is represented as: ; In the formula: This represents the Sigmoid activation function; Represents the ReLU activation function; and These are the dimensionality reduction and dimensionality increase weight matrices for the fully connected layer, respectively. These are spatial feature vectors.

[0030] It should be noted that after completing the above calculations, the dynamic gain weights of the attention branch output are broadcast along the spatial dimension and multiplied channel by channel with the nonlinear feature map of the main branch output to obtain the recalibrated feature output.

[0031] It should be emphasized that when the distance between the end of the robotic arm and the instrument at the current workstation is greater than a set threshold (which can be set separately according to actual needs, so it is not limited), the attention branch increases the gain weight of the feature channel corresponding to the visual data. When the end of the robotic arm contacts the instrument for insertion, removal or rotation, the attention branch increases the gain weight of the feature channel corresponding to the six-dimensional force sensor data.

[0032] Preferably, this step differs from the traditional serial structure. This parallel topology avoids excessive smoothing of the original spatial signal by depth transformation before it enters the attention mechanism. This allows for the preservation of high-fidelity physical features such as the pose of the robotic arm end effector while dynamically enhancing key feature channels.

[0033] It should be noted that, in order to further alleviate the feature degradation phenomenon introduced by the batch normalization layer in deep reinforcement learning, this method also introduces a learnable gated residual fusion mechanism, such as... Figure 3 As shown in the lower part.

[0034] S202: Perform deep nonlinear mapping on the recalibrated features, and then weight and fuse the mapping results with the high-dimensional state vector through a gated identity mapping path with learnable scalar parameters to obtain fused features.

[0035] Specifically, this step involves weighted fusion of the mapping result and the high-dimensional state vector through a gated identity mapping path with learnable scalar parameters, and also includes the following: B1: The recalibrated features are sequentially input into at least one fully connected layer, and after nonlinear activation, a deep mapping result vector is obtained.

[0036] B2: Set a learnable scalar parameter, perform a weighted summation of the high-dimensional state vector and the deep mapping result vector, and use the weighted summation result as the fusion feature. The learnable scalar parameter is used to control the mixing ratio of the high-dimensional state vector and the deep mapping result in the fusion feature.

[0037] It should be noted that the initial value of the learnable scalar parameter can be set based on experience, for example, 0.1.

[0038] Specifically, the calculation result of the following formula is used as the fusion feature, namely: ; In the formula: This is the vector of deep mapping results; For learnable scalar parameters; It is a high-dimensional state vector.

[0039] It should be noted that learnable scalar parameters It acts as a signal flow regulator to ensure that the underlying millimeter-level joint position signals and millinews-level force feedback signals are preserved with high fidelity during depth computation, so as to avoid feature smoothing degradation caused by batch normalization operations.

[0040] Furthermore, in terms of eliminating end-effector jitter and ensuring smooth landing, since the underlying physical signals are fully transmitted, the policy network can output minute braking torques with extreme precision. This enables the robotic arm to achieve smooth deceleration at very close range to the instrument surface, thus completely eliminating the high-frequency jitter phenomenon commonly found in traditional reinforcement learning continuous control and preparing for subsequent fine physical contact.

[0041] Preferably, this design ensures that the high-precision joint position signal at the bottom layer can bypass the complex computational link and be transmitted to the evaluation layer. This not only effectively solves the gradient flow instability problem, but also significantly accelerates the initial convergence slope of the algorithm under small sample constraints.

[0042] S300: The fused features are input into the policy network, which generates motion commands for the robotic arm to perform visual inspection operations on multi-station instruments.

[0043] It is understandable that the motion commands generated by the policy network are used to drive the robotic arm to perform detection operations such as grasping multi-station instruments, plugging and unplugging cables, and status recognition.

[0044] In this specific implementation, step S300 includes the following: C1: Configure the hidden layer dimension of the policy network to 256.

[0045] It should be emphasized that traditional policy networks typically use hidden layers with a large width, such as 2048 dimensions. However, in this method, due to the aforementioned enhanced feature extraction capabilities, the hidden layer dimension of the policy network can be configured to 256.

[0046] C2: Based on the JAX framework, operator fusion and parallel compilation are performed on the forward computation process of the policy network.

[0047] Preferably, through this optimization step, the single-step inference latency of the policy network can be reduced to less than 0.18 milliseconds, meeting the industrial-grade high-frequency control requirements above 500Hz. That is, when a dynamic obstacle appears on the production line, the robotic arm can instantly change the torque of each joint at an extremely high frequency, achieving smooth and flexible real-time obstacle avoidance.

[0048] C3: In each control cycle, the fused features are input into the compiled policy network, and the angular velocity or torque values ​​of each joint of the robotic arm are output as motion commands.

[0049] It should be noted that the motion commands are sent to the actuators of each joint of the robotic arm through a real-time communication interface.

[0050] During implementation, regarding flexible contact and force feedback recalibration, when the robotic arm gripper contacts the instrument components, the relative importance of visual features decreases while force feedback becomes crucial. At this point, the underlying parallel branch channel attention mechanism responds rapidly, dynamically amplifying the feature gain weights of the six-dimensional force sensor channels. In terms of adaptive detection action execution, taking rotating an instrument switch or plugging / unplugging a cable as an example, after the robotic arm's end effector contacts the target, the strategy network dynamically outputs operational angular velocity and displacement fine-tuning commands based on the real-time transmitted torque and contact resistance. If a sudden increase in resistance exceeding a preset safety threshold is detected, the system determines it as mechanical jamming, and the strategy network instantly outputs a reverse unloading action within 0.18ms, effectively preventing forced operation from damaging the instrument components. Regarding visual confirmation of detection results, after the fine action is completed, the robotic arm retreats to a safe observation distance as instructed. The vision system then collects the instrument indicator status or screen readings. The host computer comprehensively determines whether the current workstation's instrument function is qualified and automatically records a multi-sensor fusion detection log, including operational torque and visual screenshots.

[0051] Once all workstations have completed their assembly line inspection tasks, the robotic arm will return to its initial safe standby zero point according to the strategy output. The system will automatically disconnect the servo enable and upload data such as the inspection pass rate, execution time, and anomaly handling logs of each batch to the manufacturing execution system, forming a complete flexible control closed loop.

[0052] In summary, the beneficial effects of the multi-station instrument vision inspection robotic arm control method of this invention are as follows: by dynamically recalibrating the features of heterogeneous sensors through a parallel branch channel attention mechanism, it can effectively solve the dilution problem of multi-source physical signals in deep networks. Furthermore, by introducing a gated identity mapping path with learnable scalar parameters, not only can high-precision joint position and force signals be transmitted to the evaluation layer without loss, but the feature degradation and end effector jitter caused by batch normalization can also be avoided. Moreover, by adopting a lightweight strategy network in conjunction with JAX operator-level optimization, the single-step inference latency can be reduced to 0.18ms to meet the requirements of industrial-grade high-frequency real-time control.

[0053] Example 2, refer to Figure 2 This embodiment, based on Embodiment 1 above, further specifies the joint training method of the evaluation network and the policy network, specifically as follows: Figure 2 The training feedback loop of the targetless network architecture used in this embodiment involves the evaluation network and policy network updating their parameters immediately after each interaction with the environment to form a closed-loop adaptive control. It should be noted that this embodiment is more suitable for industrial field deployment scenarios with limited training samples and requiring rapid convergence.

[0054] The multi-station instrument vision inspection robotic arm control method provided in this embodiment has the same steps as in Embodiment 1, including data acquisition and fusion, feature recalibration and gating residual fusion of the evaluation network, and action generation of the policy network. The difference lies in the following configuration of the network training process in this embodiment: Preferably, the evaluation network and policy network are jointly trained using a target-free network architecture. Specifically, during training, the target network used in traditional reinforcement learning algorithms is removed, and the current evaluation network is used directly to calculate the Q-value target.

[0055] Specifically, by leveraging the advantages of the target network's CrossQ architecture and the learnable gated residual fusion mechanism, the weight parameters of the policy network can be quickly updated without the need for complex reprogramming and teaching, thus rapidly learning the detection strategy of the new instrument.

[0056] Preferably, the update-to-data ratio is set to 1. That is, the system will immediately perform a network parameter update after each environmental interaction.

[0057] Preferably, based on the above settings, the intelligent agent can output fine-tuning instructions for the robotic arm joints according to the recalibration characteristics, and perform actions such as grasping, plugging and status recognition of multi-station instruments.

[0058] Preferably, the total number of interaction steps in the training environment is limited to less than 100,000 steps. Experiments have verified that when the total number of interaction steps reaches 100,000, the policy network can converge to a stable control policy, eliminating the need for the millions of interactions required by traditional methods.

[0059] In summary, this method enables rapid learning of robotic arm control strategies under extremely low sample conditions.

[0060] Example 3: This example also provides a multi-station instrument vision inspection robotic arm control system, including: The fusion generation module is used to perform time-synchronized fusion of the target data collected in real time to generate a high-dimensional state vector, where the target data includes physical data and visual state data. The input / output module is used to input high-dimensional state vectors into the evaluation network and output fused features. The evaluation network performs the following operations sequentially: The high-dimensional state vector is decoupled into two parallel processing branches through a parallel branch channel attention mechanism. The main branch performs nonlinear feature extraction, while the attention branch independently calculates the dynamic gain weights of each feature channel. The gain weights output by the attention branch are multiplied channel by channel with the nonlinear features output by the main branch to obtain the recalibrated features. The recalibrated features are subjected to deep nonlinear mapping, and the mapping result is then weighted and fused with the high-dimensional state vector through a gated identity mapping path with learnable scalar parameters to obtain fused features. The learnable scalar parameters are used to dynamically balance the proportion of the high-dimensional state vector and the deep mapping result in the fused features. The generation and execution module is used to input the fused features into the policy network, which generates motion commands for the robotic arm to perform visual inspection operations on the multi-station instrument.

[0061] This embodiment also provides an electronic device suitable for controlling a multi-station instrument vision inspection robotic arm, including: a memory and a processor; the memory is used to store computer-executable instructions, and the processor is used to execute the computer-executable instructions to implement the multi-station instrument vision inspection robotic arm control method proposed in the above embodiment.

[0062] This embodiment also provides a storage medium on which a computer program is stored. When the program is executed by a processor, it implements the control method for a multi-station instrument vision inspection robotic arm as proposed in the above embodiments.

[0063] The storage medium proposed in this embodiment and the method for controlling a robotic arm for multi-station instrument visual inspection proposed in the above embodiments belong to the same inventive concept. Technical details not described in detail in this embodiment can be found in the above embodiments, and this embodiment has the same beneficial effects as the above embodiments.

[0064] Based on the above description of the implementation methods, those skilled in the art can clearly understand that the present invention can be implemented using software and necessary general-purpose hardware, and of course, it can also be implemented using hardware. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as a computer floppy disk, read-only memory (ROM), random access memory (RAM), flash memory, hard disk, or optical disk, etc., including several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods of the various embodiments of the present invention.

[0065] It should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and not to limit it. Although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can be made to the technical solutions of the present invention without departing from the spirit and scope of the technical solutions of the present invention, and all such modifications or substitutions should be covered within the scope of the claims of the present invention.

Claims

1. A control method for a multi-station instrument vision inspection robotic arm, characterized in that: include: The target data collected in real time is time-synchronized and fused to generate a high-dimensional state vector, where the target data includes physical data and visual state data. The high-dimensional state vector is input into the evaluation network, and the fused features are output, wherein the following operations are performed sequentially in the evaluation network: The high-dimensional state vector is decoupled into two parallel processing branches through a parallel branch channel attention mechanism. The main branch performs nonlinear feature extraction, while the attention branch independently calculates the dynamic gain weight of each feature channel and multiplies the gain weight output by the attention branch with the nonlinear feature output by the main branch channel by channel to obtain the recalibrated features. The recalibrated features are subjected to deep nonlinear mapping, and the mapping result is then weighted and fused with the high-dimensional state vector through a gated identity mapping path with learnable scalar parameters to obtain the fused features. The learnable scalar parameters are used to dynamically balance the proportion of the high-dimensional state vector and the deep mapping result in the fused features. The fused features are input into a policy network, which generates motion commands for the robotic arm to perform visual inspection operations on the multi-station instrument.

2. The control method for a multi-station instrument vision inspection robotic arm as described in claim 1, characterized in that: The step of time-synchronized fusion of the real-time acquired target data to generate a high-dimensional state vector includes: The high-frequency physical signals collected in real time by the robotic arm joint encoder and the six-dimensional force sensor are used as the time reference signal; Within each control cycle, the nearest neighbor timestamp matching strategy is adopted to select the visual state data frame whose timestamp is closest to the current cycle time. A zero-order hold strategy is adopted to continuously reuse the currently selected visual state data before the arrival of the next visual frame, so as to complete the time alignment and fusion of high-frequency physical signals and low-frequency visual states to generate the high-dimensional state vector.

3. The control method for a multi-station instrument vision inspection robotic arm as described in claim 2, characterized in that: The attention branch independently calculates the dynamic gain weights of each feature channel, including: Global average pooling is performed on the input high-dimensional state vector to generate a spatial feature vector; The spatial feature vectors are sequentially passed through a dimension-reducing fully connected layer, a ReLU activation function, a dimension-upgrading fully connected layer, and a Sigmoid activation function to generate dynamic gain weight vectors for each feature channel.

4. The control method for a multi-station instrument vision inspection robotic arm as described in claim 3, characterized in that: The step of weightedly fusing the mapping result with the high-dimensional state vector through a gated identity mapping path with learnable scalar parameters includes: The recalibrated features are sequentially input into at least one fully connected layer, and after nonlinear activation, a deep mapping result vector is obtained. A learnable scalar parameter is set, and the high-dimensional state vector and the deep mapping result vector are weighted and summed, and the weighted sum is used as the fusion feature.

5. The control method for a multi-station instrument vision inspection robotic arm as described in claim 4, characterized in that: The motion commands for the robotic arm generated by the policy network include: Configure the hidden layer dimension of the policy network to 256; Based on the JAX framework, operator fusion and parallel compilation are performed on the forward computation process of the policy network; Within each control cycle, the fused features are input into the compiled policy network, and the angular velocity or torque values ​​of each joint of the robotic arm are output as action commands.

6. The control method for a multi-station instrument vision inspection robotic arm as described in claim 5, characterized in that: The evaluation network and policy network are jointly trained using a target-free network architecture, including: Set the update to data ratio UTD=1 so that a network parameter update is performed immediately after each environmental interaction is completed.

7. The control method for a multi-station instrument vision inspection robotic arm as described in claim 6, characterized in that: The joint training also includes limiting the total number of training environment interaction steps to less than 100,000 steps, and using the gradient of a single interaction data point to update the policy network and the evaluation network.

8. A multi-station instrument vision inspection robotic arm control system, employing the method described in any one of claims 1-7, characterized in that, include: The fusion generation module is used to perform time-synchronized fusion of the target data collected in real time to generate a high-dimensional state vector, where the target data includes physical data and visual state data. The input / output module is used to input the high-dimensional state vector into the evaluation network and output fused features, wherein the following operations are performed sequentially in the evaluation network: The high-dimensional state vector is decoupled into two parallel processing branches through a parallel branch channel attention mechanism. The main branch performs nonlinear feature extraction, while the attention branch independently calculates the dynamic gain weight of each feature channel and multiplies the gain weight output by the attention branch with the nonlinear feature output by the main branch channel by channel to obtain the recalibrated features. The recalibrated features are subjected to deep nonlinear mapping, and the mapping result is then weighted and fused with the high-dimensional state vector through a gated identity mapping path with learnable scalar parameters to obtain the fused features. The learnable scalar parameters are used to dynamically balance the proportion of the high-dimensional state vector and the deep mapping result in the fused features. The generation and execution module is used to input the fused features into the policy network, and the policy network generates motion commands for the robotic arm to perform visual inspection operations of the multi-station instrument.

9. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it implements the steps of the method according to any one of claims 1 to 7.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 7.

Images