Dual-robot collaborative vision anti-collision method, system, device and medium

Abstract

The application relates to the technical field of image processing, and provides a double-robot-arm cooperative visual anti-collision method, system, device and medium. An implementation scheme is as follows: based on joint angle data, the projection area of the double robot arms in an image plane is determined, and a collision risk field in a space domain is constructed according to the projection area; original image features are obtained by performing feature extraction on visual image data through a preset double-robot-arm anti-collision network; the original image features are subjected to frequency domain modulation and space fusion processing based on the collision risk field and an illumination enhancement kernel in the double-robot-arm anti-collision network, so that first enhanced features are obtained; the first enhanced features are subjected to deblurring processing based on a motion clarity kernel in the double-robot-arm anti-collision network, so that second enhanced features are generated; the second enhanced features are subjected to feature extraction and classification prediction based on a detection head and a prediction head in the double-robot-arm anti-collision network, so that a collision prediction result of the double robot arms is obtained. The reliability of anti-collision can be improved by the application.

Description

Technical Field

[0001] This invention relates to the field of image processing technology, and in particular to a dual-robotic-arm collaborative vision anti-collision method, system, device and medium. Background Technology

[0002] With the continuous improvement of industrial automation, collaborative operations of dual robotic arms are widely used in assembly manufacturing, flexible production, and precision operations. To ensure the safe operation of dual robotic arms in a shared workspace, collision avoidance detection technology has become a key supporting element.

[0003] In existing technologies, simplified geometric envelopes (such as capsules or spheres) are typically constructed for each link of a dual robotic arm, and environmental point cloud data is acquired using a depth camera. The minimum spatial distance between these envelopes is then calculated to determine the potential for collisions. This approach reduces reliance on visible light conditions to some extent and shows promising application prospects in structured scenes.

[0004] However, in complex industrial scenarios, the reliability of existing dual-arm visual collision avoidance methods remains insufficient. On the one hand, industrial environments often experience low lighting or drastic lighting changes, leading to a decline in visual image quality; on the other hand, the high-speed movement of the robotic arm easily generates dynamic blur, making it difficult to accurately extract key structural and pose features. Due to the combined effects of low lighting and dynamic blur, existing methods are prone to low feature extraction accuracy, resulting in false alarms or missed alarms in collision risk assessment, failing to meet the requirements for high-reliability collision avoidance. Summary of the Invention

[0005] This invention provides a dual-robotic-arm collaborative vision-based collision avoidance method, system, device, and medium, which can solve at least one of the above-mentioned technical problems.

[0006] In a first aspect, embodiments of the present invention provide a dual-robotic-arm collaborative vision-based collision avoidance method, comprising: Acquire the joint angle data and visual image data of the dual robotic arms at the current moment; Based on the joint angle data, the projection area of ​​the dual robotic arms in the image plane is determined, and a collision risk field in the spatial domain is constructed according to the projection area; The visual image data is processed by a pre-set dual-robotic arm anti-collision network to extract features and obtain the original image features. Based on the collision risk field and the illumination enhancement kernel in the dual robotic arm anti-collision network, the original image features are subjected to frequency domain modulation and spatial fusion processing to obtain the first enhanced feature. Based on the motion clarity kernel in the dual robotic arm anti-collision network, the first enhanced feature is deblurred to generate the second enhanced feature; Based on the detection head and prediction head in the dual-arm collision avoidance network, feature extraction and classification prediction are performed on the second enhanced feature to obtain the collision prediction result of the dual-arm.

[0007] Secondly, embodiments of the present invention provide a dual-robotic arm collaborative vision collision avoidance system, comprising: The data acquisition module is used to acquire the joint angle data and visual image data of the dual robotic arms at the current moment; A construction module is used to determine the projection area of ​​the dual robotic arms in the image plane based on the joint angle data, and to construct a collision risk field in the spatial domain based on the projection area; The feature extraction module is used to extract features from the visual image data through a preset dual-robotic arm anti-collision network to obtain the original image features; The illumination enhancement kernel module is used to perform frequency domain modulation and spatial fusion processing on the original image features based on the collision risk field and the illumination enhancement kernel in the dual robotic arm anti-collision network to obtain the first enhanced feature. The motion clarity kernel module is used to deblur the first enhanced feature based on the motion clarity kernel in the dual robotic arm anti-collision network to generate a second enhanced feature. The prediction module is used to extract and classify the second enhanced feature based on the detection head and prediction head in the dual-arm collision avoidance network to obtain the collision prediction result of the dual-arm.

[0008] Thirdly, embodiments of the present invention also provide a computer device, including: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, the instructions being executed by the at least one processor to enable the at least one processor to perform the method described in any one of the embodiments of the present invention.

[0009] Fourthly, embodiments of the present invention also provide a non-transitory computer-readable storage medium storing computer instructions, wherein the computer instructions are used to cause a computer to perform the method described in any one of the embodiments of the present invention.

[0010] This invention employs the technical solution to acquire joint angle data and visual image data of dual robotic arms at the current moment. Based on this, firstly, the projection area of ​​the robotic arms in the image plane is determined using the joint angle data, and a collision risk field in the spatial domain is further constructed. This introduces prior constraints related to the real spatial position during feature extraction, enabling the model to focus on high-risk areas and reduce interference from irrelevant regions. Secondly, the original image features are frequency-domain modulated and spatially fused using an illumination enhancement kernel to obtain the first enhanced feature, which effectively compensates for feature degradation under low illumination conditions and improves the image's detail representation. Then, the first enhanced feature is further deblurred using a motion clarity kernel to weaken the dynamic blurring effect introduced by the high-speed movement of the robotic arms and restore key structural and pose information. Finally, the second enhanced feature is discriminated and predicted using a detection head and a prediction head, thereby obtaining accurate and reliable collision prediction results. Therefore, by specifically suppressing and compensating for adverse factors such as low light and dynamic blur, the accuracy of feature extraction can be effectively improved, enabling the extracted features to more realistically and completely represent the spatial positional relationship and motion state of the two robotic arms. On this basis, when the detection head and prediction head make judgments based on the above features, their recognition capabilities are enhanced, thereby reducing misjudgments and missed judgments caused by feature distortion or missing information, and ultimately improving the reliability of the collision prediction results of the two robotic arms.

[0011] It should be understood that the description in this section is not intended to identify key or essential features of the embodiments of the present invention, nor is it intended to limit the scope of the invention. Other features of the invention will become readily apparent from the following description. Attached Figure Description

[0012] The accompanying drawings are provided for a better understanding of this solution and do not constitute a limitation of the invention. Wherein: Figure 1 This is a flowchart of a dual-robotic arm collaborative vision anti-collision method according to an embodiment of the present invention; Figure 2 This is a structural block diagram of a dual-robotic arm collaborative vision anti-collision system according to an embodiment of the present invention; Figure 3 This is a schematic block diagram of a computer device used to implement the methods of the embodiments of the present invention. Detailed Implementation

[0013] The following description, in conjunction with the accompanying drawings, illustrates exemplary embodiments of the present invention, including various details to aid understanding. These details should be considered merely exemplary. Therefore, those skilled in the art will recognize that various changes and modifications can be made to the embodiments described herein without departing from the scope of the invention. Similarly, for clarity and brevity, descriptions of well-known functions and structures are omitted in the following description.

[0014] This invention provides a dual-arm collaborative vision collision avoidance method, system, device, and medium. The executing entity of this dual-arm collaborative vision collision avoidance method can be the dual-arm collaborative vision collision avoidance system provided in this invention, or a computer device integrating the dual-arm collaborative vision collision avoidance system. The dual-arm collaborative vision collision avoidance system can be implemented in hardware or software, and the computer device can be a terminal or a server.

[0015] Figure 1 This is a flowchart of a dual-robotic arm collaborative vision anti-collision method according to an embodiment of the present invention.

[0016] like Figure 1 As shown, the dual-robotic-arm collaborative vision-based collision avoidance method may include: S110, acquire the joint angle data and visual image data of the dual robotic arms at the current moment; S120, based on joint angle data, determines the projection area of ​​the two robotic arms in the image plane, and constructs a collision risk field in the spatial domain according to the projection area; S130 extracts features from visual image data through a pre-set dual-arm anti-collision network to obtain the original image features; S140, based on the collision risk field and the illumination enhancement kernel in the dual robotic arm anti-collision network, performs frequency domain modulation and spatial fusion processing on the original image features to obtain the first enhanced feature; S150, based on the motion clarity kernel in the dual-robotic arm anti-collision network, the first enhanced feature is deblurred to generate the second enhanced feature; S160, based on the detection head and prediction head in the dual-arm collision avoidance network, performs feature extraction and classification prediction on the second enhanced feature to obtain the collision prediction result of the dual-arm.

[0017] For example, joint angle data refers to the set of angle parameters used to characterize the physical posture of each joint of the dual robotic arms at various times.

[0018] For example, for a six-DOF dual-arm system, joint angle data ,in Let t be the rotational joint angle (joint angle vector) of the left robotic arm at time t. Let t be the rotational joint angle of the right robotic arm at time t; T is the matrix transpose operator.

[0019] For example, visual image data refers to a two-dimensional pixel matrix that includes a dual-robotic arm operation scene, acquired through a vision sensor. For instance, a red-green-blue (RGB) image acquired at time t in a low-light environment. , where H, W and C are the height, width and length of the visual image, respectively; R is a real number.

[0020] In this example, firstly, the encoder output signals corresponding to each joint are read from the joint drive units in the dual-arm control system to obtain the joint angle data at the current moment; and the data of different joints are uniformly timestamped to ensure the consistency of the data in the time dimension. For example, at a certain moment, the corresponding angle values ​​are read by the absolute encoders of the six joints respectively, and uniformly marked as the data set of the same sampling moment, which is used to characterize the overall posture of the robotic arm at that moment.

[0021] Second, the acquired joint angle data undergoes preprocessing operations, including outlier detection, noise filtering, and unit conversion, to improve data stability and usability. Outlier detection removes abrupt changes caused by communication delays or hardware jitter; noise filtering smooths angle variation curves; and unit conversion converts angle values ​​to radians or a standardized format.

[0022] Third, visual image data at the current moment is acquired by an industrial camera set up in the working environment, and the camera parameters, including exposure time, gain and white balance, are adaptively adjusted according to the ambient lighting conditions to ensure that recognizable image information can still be obtained in low light or complex lighting environments.

[0023] Fourth, the acquired visual image data is preprocessed, including image denoising, resizing, and pixel value normalization. Denoising reduces random noise interference in the image, resizing unifies the image resolution, and normalization adjusts pixel values ​​to a uniform distribution range.

[0024] Fifth, the preprocessed joint angle data and visual image data can be matched and aligned according to timestamps, and a unified data input structure can be constructed so that subsequent steps can perform joint processing based on the state information at the same time. For example, joint angle data acquired within the same sampling period can be bound with image frames at the corresponding time points to form a set of combined data containing robotic arm posture information and environmental visual information, which can be used for subsequent collision risk analysis and feature extraction.

[0025] For example, a collision risk field refers to a spatial probability distribution map generated after the physical configuration of the robotic arm is projected onto the image plane, used to quantify the potential collision risk level of each pixel in the image space. The collision risk field can be represented as... Used to map image pixel coordinates At any moment Spatial domain collision threat level.

[0026] For example, the generated collision risk field This is a single-channel feature heatmap with the same resolution as the original image, and its pixel values ​​are distributed in... Within the interval. Specifically, the pixel values ​​of the robotic arm link projection area and its adjacent spatial envelope area approach [value missing]. (Marked as high risk), while the pixel values ​​of background areas far from the robotic arm's range of motion tend to be close to (Marked as low risk).

[0027] In this example, firstly, the joint angle data is converted into the pose representation of each link of the dual robotic arms in three-dimensional space based on the forward kinematics model. Then, the pose representation is mapped to the image plane coordinate system by combining the camera intrinsic and extrinsic parameter matrices to obtain the projection area of ​​each link of the dual robotic arms in the image plane. Subsequently, the Euclidean distance from each pixel in the image plane to the edge of the projection area is calculated, and the distance is converted into the initial risk probability of each pixel using a preset spatial decay function (such as the Gaussian kernel function) to achieve a smooth decay of the risk level as the spatial distance increases. Finally, the initial risk probabilities are weighted and accumulated according to the physical size of each link of the robotic arms and the motion priority weight (such as the weight of the end effector being higher than that of the base link) to construct a collision risk field that reflects the global spatial risk characteristics.

[0028] For example, a dual-manipulator collision avoidance network refers to an end-to-end deep learning model architecture built for collaborative dual-manipulator operation scenarios. It integrates illumination enhancement, dynamic deblurring, object detection, and collision relationship discrimination functions within a unified network framework. This network includes at least an input layer, a backbone feature extraction module, a feature enhancement module, a detection branch (detection head), and a collision prediction branch (prediction head).

[0029] Specifically, the backbone feature extraction module performs multi-scale feature encoding on the input visual image data, preferably using a YOLO-based backbone network structure. The feature enhancement module is embedded in different layers of the backbone network, including an illumination enhancement kernel for low-light compensation and a motion sharpness kernel for dynamic blur recovery. The illumination enhancement kernel is placed in the shallow high-resolution feature stage to achieve global brightness and contrast enhancement, while the motion sharpness kernel is placed in the mid-level feature stage to achieve local contour recovery. The detection branch outputs the category information and spatial location of the robotic arm and environmental targets, while the collision prediction branch performs a probability assessment of potential collision relationships based on the detection results and enhanced features. All modules are jointly trained using a unified loss function, ensuring that the feature enhancement process directly serves the collision detection task.

[0030] For example, in one specific implementation, YOLOv11 is used as the backbone network. E-Collision Blocks (light enhancement kernels) are inserted in Stage 1 (first stage) and Stage 2 (second stage) for light enhancement, and D-Collision Blocks (motion clarity kernels) are inserted in Stage 3 (third stage) and Stage 4 (fourth stage) for deblurring. A detection head and a collision prediction head are set at the output end to realize the real-time anti-collision function in low-light dynamic environment.

[0031] For example, raw image features refer to the feature matrix containing low-level semantic information obtained after visual image data has undergone preliminary downsampling and extraction via a convolutional neural network. For instance, raw image features... It has 32 channels and a resolution that is 1 / 4 of the original image. It captures the basic outline of the scene, but its response value is extremely low in low light.

[0032] In this example, visual image data is input into a pre-trained deep convolutional network structure (e.g., YOLOv11 backbone network). First, the input image is scale-normalized and pixel value-standardized to eliminate brightness and scale differences under different acquisition conditions. Then, the image is progressively feature-extracted through multiple convolutional layers in the backbone network. Low-level feature information such as edges and textures is obtained in the shallow convolutional stage, local structure and shape information is extracted in the middle convolutional stage, and high-level semantic information is extracted in the deep convolutional stage. Batch normalization and non-linear activation functions (e.g., sigmoid function or rectified linear unit function, ReLU) are used between each convolutional layer to enhance the non-linearity and stability of feature representation. Based on this, a feature pyramid network structure is constructed to fuse feature maps at different scales through top-down and lateral connections, so that high-resolution features have stronger semantic information while preserving spatial details, thus forming a multi-scale fused original image feature representation.

[0033] Furthermore, a Region of Interest Align (ROI) operation can be introduced during feature extraction to perform fine-grained feature alignment on the regions where key parts of the robotic arm are located, so as to avoid spatial information shifts caused by quantization errors and thus improve the feature representation accuracy of key structures. At the same time, a lightweight fully connected structure can be combined to recalibrate the channels of the fused features to highlight important channel responses and suppress redundant information.

[0034] For example, the illumination enhancement kernel refers to a frequency domain feature enhancement module embedded in the dual-arm anti-collision network. It guides the original image features based on the collision risk field and adaptively modulates the original amplitude spectrum in the frequency domain, thereby achieving targeted enhancement of low-light areas and collision-critical areas.

[0035] The illumination enhancement kernel includes a Fourier transform module, a multilayer perceptron module, an amplitude modulation module, a spatial gating module, an inverse Fourier transform module, and a feature fusion module.

[0036] Specifically, the Fourier transform module performs a discrete Fourier transform on the original image features to separate the original amplitude spectrum and the original phase spectrum, and maps the collision risk field to the frequency domain to obtain the frequency domain risk field; the amplitude modulation module performs nonlinear enhancement processing on the original amplitude spectrum based on the enhancement coefficients (determined by the frequency domain risk field, learnable channel enhancement parameters, and temperature parameters) and the amplitude compensation amount generated by the multilayer perceptron module to generate an enhanced amplitude spectrum; the inverse Fourier transform module performs an inverse transform on the enhanced amplitude spectrum and the original phase spectrum to obtain spatial domain enhanced features; the spatial gating module generates a spatial gating mask based on the collision risk field; and the feature fusion module combines the spatial gating mask, the original image features, and the spatial domain enhanced features to obtain the first enhanced feature, thereby avoiding information distortion caused by over-enhancement.

[0037] For example, in one specific implementation, the E-Collision Block first performs a fast Fourier transform on the input features to decompose the features into frequency components. Then, based on the collision risk field, it applies a higher gain to the frequency components corresponding to high-risk areas. The spatial features are then recovered through an inverse transform, and the enhanced results are output in combination with the original features, so that the end effector and linkage area of ​​the robotic arm still have high contrast in low-light environments.

[0038] For example, the first enhanced feature refers to the feature matrix in which the brightness and contrast of the original image features are restored and the response value of the key area is significantly improved after processing by the illumination enhancement kernel.

[0039] In this example, the original image features are first transformed in the frequency domain to decompose them into low-frequency structural information and high-frequency detail information. Then, the collision risk field is mapped to a weight matrix of the same scale as the feature map and jointly modeled with the illumination enhancement kernel to enhance the feature response corresponding to the high-risk region and suppress the low-risk region. Finally, the enhanced frequency domain features are reconstructed into a unified feature representation through spatial fusion operation to obtain the first enhanced feature.

[0040] For example, the motion clarity kernel refers to a spatial domain deblurring and contour enhancement module embedded in a dual-arm collision avoidance network. It locates dynamically blurred regions in the first enhancement feature by constructing a saliency-guided mechanism and achieves accurate restoration of key structures through an adaptive convolutional sampling strategy.

[0041] The motion clarity kernel includes an edge extraction module, a motion estimation module, a saliency fusion module, a deformable convolution module, an attention processing module, and a feature fusion output module.

[0042] Specifically, the edge extraction module is used to extract the edge intensity map from the first enhancement feature; the motion estimation module is used to calculate the motion difference feature based on the dense optical flow field between the first enhancement features at adjacent time points; the saliency fusion module is used to fuse the normalized edge intensity map and the motion difference feature to generate a saliency map, which is used to indicate the key contour region of collision; the deformable convolution module is used to calculate the spatial gradient of the saliency map, and adjust the initial sampling offset based on the spatial gradient to obtain the target sampling offset, so that the sampling points are shifted to the high saliency region to obtain the aligned contour feature; the attention processing module is used to calculate the saliency association value based on the spatial gradient, and combine the query feature vector, key feature vector and value feature vector to generate the attention feature; the feature fusion output module is used to fuse and normalize the attention feature and the aligned contour feature to obtain a clear second enhancement feature and suppress background redundancy information.

[0043] For example, in one specific implementation, the D-Collision Block first uses the Sobel Operator (Sobel) to extract the edges of the robotic arm, then combines optical flow estimation to obtain the motion-blurred region, generates a weight map through saliency fusion, and uses this weight map to modulate the sampling offset of the deformable convolution, so that the convolution kernel is preferentially aligned with the edges of the robotic arm links and the position of the end tool, thereby effectively restoring the blurred contour and improving the collision detection accuracy.

[0044] For example, the second enhanced feature refers to the feature matrix after the first enhanced feature has been processed by the motion clarity kernel, in which the blurred edges are reconstructed and the contour clarity meets the detection standard.

[0045] In this example, gradient analysis is performed on the motion-blurred regions in the first enhancement feature to identify feature regions with high edge degradation. Subsequently, deconvolution or feature compensation processing is applied to these regions using a motion clarity kernel to restore edge structure information and local texture details. The restored features are then normalized and fused to obtain the second enhancement feature. For example, during the high-speed grasping process of a robotic arm, image blurring is caused by shutter lag. This step restores the edge contour of the end effector, making it more clearly distinguishable in the feature space.

[0046] For example, a detection head refers to a network branch used to regress object category confidence and predicted bounding box coordinates from augmented features. For instance, a standard YOLO detection head outputs a set of bounding boxes containing the locations of robotic arm links, end effectors, and obstacles. .

[0047] For example, the prediction head refers to the logical prediction branch used to determine whether there is a risk of physical conflict between pairs of entities within a specific bounding box region. For instance, a classifier composed of a multilayer perceptron (MLP) receives fine features extracted by ROIAlign and outputs the collision probability.

[0048] For example, the collision prediction result refers to the binary or multivariate discrimination vector ultimately output by the system, used to quantify the safety status of each preset detection pair of the dual robotic arms. For instance, the collision prediction result is represented as a vector. ,in The preset number of target pairs to be detected (e.g., a value of 6); if Then, "1" indicates that the preset second detection pair (such as the left robotic arm end effector and the right robotic arm link) is at the current moment. There is a risk of collision, while "0" indicates that the other detection pairs are in a safe state.

[0049] In this example, firstly, the detection head is invoked to perform target-level feature parsing on the second enhanced feature. Then, regression analysis is used to determine the spatial locations (bounding boxes) of key parts of the dual robotic arms and obstacles in the environment. ) and category information. Then, based on the bounding box The system guides the process by extracting fine-grained region features of each target from the second enhanced features using the ROIAlign operation. These features are then input into the prediction head, where a multilayer perceptron (MLP) is used to perform nonlinear modeling of the spatial topological relationships between target pairs, outputting collision probability values ​​for each detection pair. Finally, a preset risk threshold is used to binarize the collision probability values, setting detection pairs above the threshold to "1" and those below to "0", thus generating a vector-based collision prediction result.

[0050] According to the above implementation method, by acquiring the joint angle data and visual image data of the dual robotic arms at the current moment, and determining the projection area of ​​the dual robotic arms in the image plane based on the joint angle data, a collision risk field in the spatial domain is further constructed. This maps and fuses the kinematic state of the robotic arms with spatial geometric relationships, providing risk guidance information with spatial prior constraints for subsequent visual analysis. Subsequently, feature extraction is performed on the visual image data through a preset dual robotic arm anti-collision network to obtain the original image features. Then, the original image features are further combined with the collision risk field and the illumination enhancement kernel to perform frequency domain modulation and spatial fusion processing to obtain the first enhanced feature. This strengthens the feature response of key risk areas, thereby improving the feature expression capability and robustness under complex illumination conditions. Then, the first enhanced feature is deblurred based on the motion clarity kernel to generate the second enhanced feature, thereby effectively suppressing the blurring interference introduced by the high-speed movement of the robotic arms or visual acquisition jitter, and improving the clarity and distinguishability of features in dynamic scenes. Finally, based on the detection head and prediction head, further feature extraction and classification prediction of the second enhanced feature are performed to obtain the collision prediction results of the dual robotic arms. This achieves end-to-end discrimination of potential collision risks, effectively improving the accuracy, robustness and real-time performance of collision prediction in the dual robotic arm collaborative operation scenario, and providing a more reliable decision-making basis for safety control.

[0051] In one implementation, based on joint angle data, the projection area of ​​the dual robotic arms in the image plane is determined, and a collision risk field in the spatial domain is constructed according to the projection area. This includes: calculating the joint angle data using a preset forward kinematics model to obtain the pose data of each link envelope of the dual robotic arms in the reference coordinate system; using each pose data as input to a preset camera projection model to project the vertices of each link envelope onto the image plane to obtain the projection area of ​​each link envelope; calculating the shortest distance from each pixel in the image plane to each projection area; calculating the risk value of each pixel using an exponential function on preset link priority weights, preset size coefficients, and each shortest distance; and determining the collision risk field in the spatial domain based on the risk values ​​of each pixel.

[0052] For example, the forward kinematics model is a mapping equation from joint space to three-dimensional Cartesian space established based on structural parameters such as the physical length and offset of each link in the dual robotic arms (e.g., Denavit-Hartenberg parameters, DH parameters). The acquired real-time joint angle data will be used as the basis for this mapping. Substituting into the equation, we obtain the homogeneous transformation matrix of each link relative to the base coordinate system through matrix multiplication. , For the first The pose matrix (pose data) of a link relative to the base coordinate system contains the rotation and translation information of the link in three-dimensional space.

[0053] For example, In the formula, For dual robotic arms in Joint angle data at any given time; Let be the vector composed of the rotational joint angles of each joint of the left robotic arm at time t; Let be the vector composed of the rotational joint angles of the right robotic arm at time t; For transpose operator; It is a 12-dimensional real number space.

[0054] For example, a camera projection model is constructed using the camera's intrinsic and extrinsic parameter matrices. This model is used to define the vertices of the link envelope in three-dimensional pose. Pixel coordinates mapped to a two-dimensional image plane The formula for calculating the projection area is as follows: ; In the formula, For the first The projection area of ​​each link in the image plane; The pose matrix varies with joint angle data. Changes in real time; Let be the camera projection model, where π is the projection function. For three-dimensional real space, It is a two-dimensional real number space; Link X is the set of three-dimensional vertices of the envelope (such as a cylinder or simplified cuboid); X is the vertex coordinate vector of the link envelope in three-dimensional space. For example, a robotic arm link that is originally cylindrical in three-dimensional space appears as a polygonal color block area in the image after being processed by the camera projection model.

[0055] For example, for any pixel in the image Calculate its relationship with the projection area The minimum of the Euclidean distances between all pixels in the set is taken as the shortest distance for that pixel.

[0056] The distance calculation formula is: In the formula, Represents pixels To the projection area The shortest physical pixel distance at the edge (shortest distance); For pixels Other pixels besides; express Norm.

[0057] For example, the formula for calculating the risk value is: In the formula, Image coordinates exist Risk value at any given moment; For the first The preset priority weights of each link are set, and the weight setting of the robotic arm end effector (e.g., 0.8) is usually higher than that of the base link (e.g., 0.2). A preset size factor (the value is determined according to actual needs, for example, 0.6) is used to adjust the slope of risk attenuation with distance; It is a natural exponential function used to realize the risk value as a function of distance. It increases while decreasing nonlinearly.

[0058] For example, the risk value of all pixels in the image plane. The collision risk field is mapped to a single-channel probability distribution map, forming a collision risk field in the spatial domain, which guides subsequent frequency domain enhancement. For example, the generated collision risk field is visually represented as a heatmap, in which the robotic arm and its trajectory are highlighted, while unrelated background areas are black (low risk).

[0059] For example, in a specific embodiment, the result is a spatial risk probability distribution map in which the projection areas corresponding to the left robotic arm end tool and the right robotic arm link are high-risk areas (risk value 0.7~1.0), and the background area is a low-risk area (risk value 0~0.3).

[0060] According to the above implementation method, the joint angle data are calculated using a preset forward kinematics model to obtain the pose data of each link envelope of the dual robotic arms in the reference coordinate system. The pose data is then input into a preset camera projection model to map the vertices of the link envelopes onto the image plane, thus obtaining the projection area of ​​each link envelope. By uniformly mapping the kinematic pose information of the robotic arms with the visual imaging model, the link distribution in three-dimensional space can be accurately expressed in the image plane, thereby achieving an effective conversion of spatial information to the visual domain. Based on this, the shortest distance from each pixel in the image plane to each projection area is further calculated. Combined with preset link priority weights and size coefficients, the shortest distance is weighted using an exponential function to obtain the risk value of each pixel. Then, a collision risk field in the spatial domain is constructed based on these risk values. Thus, by fusing link priority weights and size coefficients, the influence of different links in risk assessment is differentiated, improving the refined expression and spatial discrimination accuracy of the collision risk field. This provides spatial guidance for subsequent frequency domain enhancement, ensuring that enhancement resources are preferentially allocated to areas with high collision risk.

[0061] In one implementation, based on the collision risk field and the illumination enhancement kernel in the dual-arm anti-collision network, the original image features are subjected to frequency domain modulation and spatial fusion processing to obtain the first enhanced feature. The illumination enhancement kernel includes a Fourier transform module, a multilayer perceptron module, an amplitude modulation module, a spatial gating module, an inverse Fourier transform module, and a feature fusion module. The fusion module includes: calling the Fourier transform module to perform a discrete Fourier transform on the original image features to obtain the original amplitude spectrum and the original phase spectrum; performing a Fourier transform on the collision risk field to obtain the frequency domain risk field; performing network processing on the frequency domain risk field through the multilayer perceptron module to generate an amplitude compensation amount; and then, based on the frequency... The system calculates enhancement coefficients based on the collision risk field, preset learnable channel enhancement parameters, and preset temperature parameters. It then calls the amplitude modulation module to modulate the original amplitude spectrum using the enhancement coefficients and amplitude compensation to generate an enhanced amplitude spectrum. Next, it calls the spatial gating module to perform convolution and nonlinear activation processing on the collision risk field, generating a spatial gating mask. Finally, it performs an inverse Fourier transform on the enhanced amplitude spectrum and the original phase spectrum using the inverse Fourier transform module to obtain spatial domain enhancement features. Based on the spatial gating mask, it weights the original image features to obtain weighted image features, and uses these weighted image features and spatial domain enhancement features as input to the feature fusion module to obtain the first enhancement feature.

[0062] For example, the Fourier transform module refers to a processing unit used to transform image features in the spatial domain to a frequency domain representation. Taking the original image as input features, before calling the Fourier transform module to process the input features, the input features can be preprocessed, as shown below: ; ; In the formula, The original image features are the input. The representation layer is normalized to stabilize the feature distribution and accelerate network convergence; These are the features after normalization; and They represent and The convolution operation is used to extract local spatial features and adjust the channel dimensions to adapt to the subsequent Fourier transform. These are the features of the original image after preprocessing.

[0063] For example, in low-light working scenarios, the original image features The numerical values ​​fluctuate wildly and contain a lot of noise. Through preprocessing, the features are mapped to a standard distribution space with a mean of 0 and a variance of 1. At the same time, redundant channel noise is removed, providing a clean "feature basis" for entering the frequency domain.

[0064] For example, the process of calling the Fourier transform module to perform a discrete Fourier transform on the preprocessed original image features is shown in the following function expression: ; Let the original amplitude spectrum be denoted as The original phase spectrum is ; In the formula, Represents the Discrete Fourier Transform operator; This represents the frequency domain coordinates and channel index, where p and q are the horizontal and vertical frequency coordinates, respectively, and c is the characteristic channel; Let m be the pixel coordinates in the spatial domain, where m is the vertical index and n is the horizontal index; c is the feature channel. This is the original amplitude spectrum, representing the energy response of each frequency component; The original phase spectrum represents the structural layout of the image; This is used for phase angle calculation, which is used to extract phase information from complex number features.

[0065] For example, the implementation process of performing a Fourier transform on the collision risk field is shown in the following function expression: In the formula, For frequency domain risk fields; This is a collision risk field.

[0066] In this example, the above steps decouple the brightness information (amplitude spectrum) and contour position information (phase spectrum) of the image, so that the system can enhance only the brightness component without destroying the geometry of the robotic arm.

[0067] For example, fine-grained amplitude compensation is generated based on the frequency domain risk distribution, and its calculation expression is as follows: ; In the formula, For amplitude compensation; FreLU A nonlinear activation function designed for frequency characteristics; This is a frequency domain risk field.

[0068] For example, if the frequency response of the region at the end of the robotic arm is extremely weak, the MLP module will calculate a specific amplitude increment to compensate for signal loss caused by low light.

[0069] For example, the calculation expression for the nonlinear gain implementation process by calling the amplitude modulation module is as follows: ; In the formula, To enhance the amplitude spectrum; Enhance the strength parameter of the preset learnable channel (e.g., 0.6). The preset temperature parameter (the value is determined according to actual needs, for example, 0.6) is used to control the smoothness of the enhancement coefficient; For activation functions; Generated by a lightweight MLP: ; To perform a flattening operation on the original amplitude spectrum A.

[0070] For example, setting For frequency components located in high-risk areas, due to... The larger the amplitude, the greater the enhancement coefficient, resulting in a 30% to 50% energy enhancement of the end effector features in the amplitude spectrum, thus "visually illuminating" the dark areas.

[0071] For example, the spatial gating module is invoked to generate a mask from the spatial domain risk field for fine-tuning the enhancement weights. Its calculation expression is as follows: ; In the formula, The generated spatial gating mask has a value range of [0,1]. For the Sigmoid function; This is a collision risk field.

[0072] For example, mask The region where the robotic arm is located is given a high weight (e.g., 0.9), while the background region is given a low weight (e.g., 0.2). This is used to guide the accurate spatial delivery of the enhanced features and prevent background noise from being amplified synchronously.

[0073] For example, the signal is recovered using an inverse Fourier transform module and then weighted and fused with the original features. The calculation expression is as follows: ; ; In the formula, This represents the inverse Fourier transform operator; Enhanced features for the spatial domain; i is the imaginary unit; The original phase spectrum; This is element-wise multiplication; This is the first enhanced feature obtained; Features of the original image; This is for layer normalization processing.

[0074] In this example, the details of frequency domain repair are shown. Compared with the original information after masking. Deep coupling. The final result is... It ensures high definition of key parts of the robotic arm while preserving the continuity of the original background, providing high-quality visual input for subsequent collision prediction.

[0075] In one implementation, based on the motion clarity kernel in the dual-arm anti-collision network, the first enhanced feature is deblurred to generate a second enhanced feature. The motion clarity kernel includes an edge extraction module, a motion estimation module, a saliency fusion module, and a deformable convolution module. The process includes: calling the edge extraction module to extract edge gradient features from the first enhanced feature at the current time, obtaining an edge intensity map; calculating motion difference features using the dense optical flow field between the first enhanced feature at the current time and the first enhanced feature at the previous time in the motion estimation module; calling the saliency fusion module to normalize the edge intensity map and the motion difference features respectively, and performing channel splicing and convolution activation on the normalized edge intensity map and motion difference features to generate a saliency map; calculating the spatial gradient of the saliency map using the deformable convolution module, updating the initial sampling offset based on the spatial gradient and preset learnable modulation coefficients to obtain a target sampling offset; performing a deformable convolution operation on the first enhanced feature using the target sampling offset to obtain an aligned contour feature; and performing feature enhancement processing on the first enhanced feature based on the aligned contour feature to obtain the second enhanced feature.

[0076] For example, the edge extraction module is invoked, and a multi-directional convolution operation is performed on the first enhanced feature at the current time step using a preset gradient operator (Sobel operator) to obtain an edge intensity map reflecting the intensity of the feature space transition. Its calculation expression is as follows: ; ; ; In the formula, The first enhanced feature at the current moment; and These are the gradient components in the horizontal and vertical directions, respectively; This is the obtained edge intensity map.

[0077] For example, in collaborative operations between two robotic arms, the boundary between the robotic arm links and the background exhibits significant numerical differences at the feature level. Through this step, the boundary profile of the robotic arm is determined in the edge intensity map. The curve in the middle is a high-response arc.

[0078] For example, the motion estimation module calculates the dense optical flow field between the features of the current time step and the previous time step, and measures the local motion deviation based on the transformation operator. The calculation expression is as follows: ; In the formula, Features of motion differences; Features of the current moment; Features of the previous time step; The dense optical flow field is estimated by the Recurrent All-Pairs Field Transforms Lite (RAFT Lite) model. This is a bilinear interpolation sampling operator.

[0079] In this example, when the robotic arm's end effector sweeps across the field of view at a high speed, the target's large displacement within the shutter time causes blurring. A significant residual exists between the current frame's features and the aligned features of the previous frame in the blurred region. This residual is obtained through... Quantified.

[0080] For example, the edge intensity map and motion difference features are normalized, and their calculation expression is as follows: ; ; In the formula, Edge intensity map; These are characteristics of motion differences.

[0081] For example, the saliency fusion module is invoked to perform nonlinear fusion of structural and motion information to locate the contour region critical for collision perception. Its calculation expression is as follows: ; In the formula, This is a saliency plot, with values ​​ranging from [0,1]. This indicates the minimum-maximum normalization process; This indicates a concatenation operation at the channel level. Use the Sigmoid activation function; This is a 1×1 convolution operation.

[0082] In this example, the generated saliency map A high response value (e.g., 0.85) is exhibited at the edges of links in a high-speed, critical robotic arm, while a low response value is exhibited against a static background or in a large area of ​​solid color, which is used to guide the offset direction of subsequent convolution kernels.

[0083] For example, a deformable convolution module finely modulates the sampling position of the convolution kernel based on the spatial gradient of the saliency map. Its calculation expression is as follows: ; ; In the formula, The spatial gradient of the saliency map at spatial pixel coordinates; For the convolution kernel The initial learning offset of each sampling point; The learnable modulation coefficient (e.g., 0.7); This is the updated target sampling offset; The coordinates of the target center position for performing the convolution operation; It is a fixed offset vector at the nth position in the preset convolutional grid.

[0084] In this example, if the significance gradient Pointing to the actual boundary of the robotic arm, this step shifts the sampling points of the convolution kernel from the originally blurred area to the gradient direction (i.e., the direction of the clear outline), thereby achieving "sampling point alignment".

[0085] For example, a convolution operation is performed on the first enhanced feature using the target sampling offset to reconstruct a sharp feature. The calculation expression is as follows: ; In the formula, To align contour features at position The output value (i.e., the aligned contour feature); For convolution sampling grid; These are the corresponding convolution weight coefficients; This is the first enhancement feature; After target sampling offset modulation, in the first enhanced feature The actual spatial coordinates of the feature sampling performed on the above.

[0086] In this example, for the "burr" edges caused by dynamic blur, this step makes the reconstructed features sharp and continuous again at the robot arm contour by sampling at the aligned position.

[0087] For example, the aligned contour features are combined with an attention mechanism to extract global correlation information and generate the second enhanced feature of the final output.

[0088] In one implementation, a second enhanced feature is obtained by performing feature enhancement processing on a first enhanced feature based on alignment contour features. The motion clarity kernel further includes an attention processing module and a feature fusion output module, comprising: calling the attention processing module to map the first enhanced feature into a query feature vector, a key feature vector, and a value feature vector through a projection matrix; calculating the saliency correlation value between pixels, the query feature vector, and the key feature vector based on spatial gradients, and calculating attention weights; performing weighted processing on the value feature vector based on the attention weights to obtain attention features; using the attention features and alignment contour features as inputs to the feature fusion output module to perform convolution processing on the attention features, and fusing and normalizing the convolutional attention features with the alignment contour features to obtain the second enhanced feature.

[0089] For example, the query, key, and value are defined as follows: ; In the formula, The projection matrix; To query feature vectors; The key feature vector; The eigenvectors are eigenvalues. This is the first enhancement feature.

[0090] For example, the saliency correlation between pixels is calculated to enhance the global correlation of the contour region. The calculation process is as follows: ; In the formula, For position arrive The significance of the correlation value; and Let i and j represent the spatial gradient values ​​of the saliency map S at positions i and j, respectively. The significance variance coefficient is the preset value.

[0091] It should be noted that the significance variance coefficient is set according to actual needs, with a value range of 0.1-2.0. For example, in a dual-robotic arm collaborative grasping scenario, when the ends of the two robotic arms gradually approach each other, the significance variance coefficient can be set to 1.2, causing the risk value of the approaching area to increase rapidly, thereby forming a more obvious high-response area in the feature map; while in the long-distance collaborative operation stage, it can be reduced to 0.6 to avoid falsely triggering high-risk judgments at long distances.

[0092] For example, saliency-weighted attention weights: ; In the formula, Attention weights; For learnable weights (e.g., 0.5); Let i be the query feature vector at position i; Let j be the key feature vector at position j; For position Key feature vector at the location; The feature dimension of the query / key vector; It is a natural exponential function.

[0093] For example, the value feature vector is weighted based on the attention weights to obtain attention features. .

[0094] For example, based on aligned contour features, global correlation information is extracted using an attention mechanism to generate the final output. Its calculation expression is as follows: ; In the formula, This is the second enhancement feature; These are attention features generated by the attention processing module, incorporating saliency correlation values. This represents depthwise separable convolution; This is the first enhancement feature; For layer normalization operators; This is a 1×1 convolution operation.

[0095] In this example, precisely aligned local details ( This module performs multi-path fusion with saliency attention features that provide a global collision view. If the robotic arm's end effector is blurry but has a high saliency correlation with an obstacle, this module will focus on strengthening the global feature connection between the two. The resulting second enhanced feature exhibits extremely high contour fidelity in complex motion environments, improving the detection contrast of key collision areas by approximately 35%, effectively solving the visual safety perception challenge in high-speed operating environments.

[0096] In one implementation, based on the detection head and prediction head in the dual-arm collision avoidance network, feature extraction and classification prediction are performed on the second enhanced feature to obtain the collision prediction result of the dual-arm, including: detecting and processing the second enhanced feature through the detection head to obtain a set of bounding boxes; aligning the bounding box set and the second enhanced feature with regional interests to obtain feature vectors for each region; inputting the feature vectors of each region into the multilayer perceptron of the prediction head for prediction processing to obtain the probability value of collision occurring in each region; and using each probability value as the collision prediction result of the dual-arm.

[0097] For example, the detection head is invoked to perform multi-scale convolution processing on the input second enhanced features, and regression is used to obtain the bounding box set of key parts of the robotic arm and obstacles in the image. The calculation expression is as follows: ; In the formula, For a moment The set of bounding boxes; For the first Feature information of each detection target; These are the pixel coordinates of the center point of the bounding box; and These are the width and height of the bounding box, respectively; The category confidence score for this target.

[0098] In this example, in a dimly lit working environment, the system accurately located the bounding box positions of the left robotic arm end gripper, the right robotic arm third link, and the edge of the work platform based on the enhanced second feature.

[0099] For example, the bounding box set and the second enhanced feature are processed by Region Interest Alignment (ROIAlign). Bilinear interpolation maps target regions of different scales to fine-grained feature representations of fixed dimensions, resulting in feature vectors for each region. The calculation expression is as follows: ; In the formula, For the first The fixed-dimensional region feature vector corresponding to each target (e.g.) size); The second augmentation feature is the input; This represents the corresponding bounding box.

[0100] In this example, regardless of whether the robotic arm appears as a large-scale area in the foreground or a small-scale area in the background, this step allows the system to extract local feature blocks with uniform semantic specifications, providing consistent input for subsequent collision determination.

[0101] For example, the extracted feature vectors of each region are input into the multilayer perceptron of the prediction head for nonlinear mapping processing to model the spatial conflict risk of the preset detection pairs, thus obtaining the probability value of collision occurring in each region. The calculation expression is as follows: ; In the formula, For the first The probability values ​​for physical collisions between preset detection pairs are , and their values ​​range from . ; Use the Sigmoid activation function; Indicates feature concatenation operation; and These are the region features of the two targets in the corresponding detection pair.

[0102] For example, for the detection pair of "left arm end" and "right arm link", MLP analyzes the spatial overlap trend and deep activation response of the two features and outputs a continuous value (such as 0.92) to characterize the mathematical probability of the detection pair colliding.

[0103] For example, the obtained probability values ​​are binarized based on a preset risk threshold to obtain the quantized collision prediction result for the dual robotic arms. The calculation expression is as follows: ; ; In the formula, Let be the collision prediction result vector at time t; The preset risk assessment threshold (e.g., 0.85); This is the preset total number of detection pairs; This is the first test pair; This is the second test pair; For the first Group detection pairs.

[0104] For example, if the calculated probability value of the "left arm end - right arm link" pair is 0.92, since it exceeds the threshold of 0.85, the system will remove the corresponding vector element. Set it to "1" to output the collision prediction result. The result clearly indicated that the second detection pair posed an immediate risk of collision. The system then triggered an emergency braking or obstacle avoidance algorithm based on this result to ensure the safe collaboration of the two robotic arms in a low-light dynamic environment.

[0105] In one implementation, after extracting and classifying the second enhanced feature based on the detection head and prediction head in the dual-arm collision avoidance network to obtain the collision prediction result of the dual-arm, the method further includes: acquiring the real detection label, real collision label, and real edge mask corresponding to the visual image data; calculating the target detection loss based on the prediction result of the second enhanced feature by the detection head and the real detection label, and calculating the cross-entropy loss based on the collision prediction result and the real collision label to obtain the collision perception guidance loss; calculating the amplitude constraint and regularization penalty based on the original amplitude spectrum separated by the illumination enhancement kernel during processing, the enhanced amplitude spectrum generated by modulation, and the learnable parameters to determine the illumination enhancement regularization loss; calculating the boundary alignment penalty based on the spatial gradient of the second enhanced feature and the real edge mask to obtain the edge preservation loss; weighting and summing the target detection loss, collision perception guidance loss, illumination enhancement regularization loss, and edge preservation loss to construct the total joint loss function; and updating the learnable parameters of the dual-arm collision avoidance network based on the gradient of the total joint loss function to jointly train the dual-arm collision avoidance network.

[0106] For example, the true detection label, true collision label, and true edge mask corresponding to the visual image data are obtained. This can be represented as: ; In the formula, The input is the training sample image (i.e., visual image data). The actual detection labels are manually annotated object bounding boxes; A true collision label to characterize whether physical conflict occurs between each target; This is the actual edge mask corresponding to the outline of the robotic arm and obstacles; This is a deep learning training dataset that includes input images and corresponding multi-path ground truth annotations.

[0107] For example, during the offline annotation phase, annotators draw the precise location bounding box of the robotic arm in low-light sample images. Record the Boolean value indicating whether the arms are actually in contact at that moment. And use an edge detection algorithm to generate a binary mask image of the robotic arm's contour. .

[0108] For example, the target detection loss is calculated based on the prediction results of the detection head and the actual detection labels, and the cross-entropy loss is calculated based on the collision prediction results and the actual collision labels to obtain the collision perception guidance loss. The calculation expression is as follows: ; ; In the formula, The target detection loss includes classification, regression, and confidence loss components. The classification loss measures the class determination bias between the predicted target class and the actual detected label. The bounding box regression loss measures the degree of spatial coordinate overlap between the predicted bounding box and the true detection label (e.g., calculated using the CIoU loss function). The target confidence loss is used to measure the accuracy of the probability that the predicted region contains the real target. To guide loss for collision perception; To predict the bounding box; To predict the collision probability (i.e., the collision prediction result in the previous example); This is the binary cross-entropy loss function.

[0109] In this example, if the system predicts the robotic arm position to be 5 pixels off from the actual position, then This will result in a large penalty gradient; if the system fails to predict the impending collision (predicted probability is 0.1 while the actual probability is 1), then... This will result in significant losses, forcing the network to focus on visual features that lead to missed detections.

[0110] For example, amplitude constraints and regularization penalties are calculated based on the original amplitude spectrum, enhanced amplitude spectrum, and learnable parameters separated from the illumination enhancement kernel. The calculation expression is as follows: ; In the formula, To enhance the regularization loss due to illumination; For the first The original amplitude spectrum of the layer; The modulated amplitude spectrum; For learnable channel enhancement strength parameters; The preset penalty coefficient (e.g., 0.01); It is the Frobenius norm.

[0111] For example, this loss term is used to prevent excessive enhancement during frequency domain modulation. If the enhanced amplitude spectrum... Compared to the original spectrum If the increase is too large (leading to overexposure), the first norm will increase to incur a penalty, ensuring that the lighting restoration is within a reasonable physical range.

[0112] For example, the boundary alignment penalty is calculated based on the spatial gradient of the second enhanced feature and the true edge mask to obtain the edge preservation loss. Its calculation expression is as follows: ; In the formula, Preserve loss at the edge; This is a gradient magnitude map of the second enhancement feature at each pixel. This is element-wise multiplication; To prevent extremely small constants with a denominator of zero (where, ); This is the true edge mask.

[0113] In this example, the loss term serves to "guide" the optimization direction of the motion clarity kernel. If the enhanced features are within the ground truth edge mask... If the gradient at a certain point is not significant enough (i.e., the outline is blurred), the ratio decreases, and the loss... Increasing this amount prompts deformable convolutions to further correct the sampling offset to align with the edges.

[0114] For example, the weighted sum of the above loss components is used to construct the total joint loss function. Its calculation expression is as follows: ; In the formula, The total joint loss function; , , These are preset weighting coefficients. Each weighting coefficient is set according to the operational safety requirements. For example, Set it to 0.2; Set to 0.3; usually, a larger value is set. (e.g., 0.5) to prioritize the accuracy of collision detection, thereby achieving deep coupling of the functions of each module.

[0115] For example, the parameters of each module in the dual-robotic arm collision avoidance network are updated using the backpropagation algorithm based on the gradient of the overall joint loss function. The gradient derivation logic is as follows: ; In the formula, These are the learnable parameters of the network as a whole; The light intensity enhancement coefficient; The offset modulation coefficients of the motion clarity kernel; This is the saliency attention parameter.

[0116] In this example, during training iterations, if the collision detection accuracy is insufficient, the generated gradient signal will be fed back to the illumination enhancement kernel and the motion sharpness kernel along the computation graph. Even if the underlying image enhancement module does not have direct "aesthetic" supervision, it will automatically learn, guided by the gradient, how to specifically repair low-light and blurry features in order to "detect collisions more accurately." Finally, after multiple iterations, an intelligent anti-collision network system that can still stably output reliable safety decisions in harsh visual environments is obtained.

[0117] For example, the learnable parameters of the dual-arm collision avoidance network include parameters for the detection head, prediction head, illumination enhancement kernel, and motion clarity kernel. The parameters of the illumination enhancement kernel refer to the set of values ​​used to adjust the frequency domain gain intensity, including at least learnable channel-related enhancement intensity parameters and temperature coefficients controlling enhancement smoothness. The parameters of the motion clarity kernel refer to the set of modulation coefficients used to guide spatial sampling and global correlation, including at least learnable modulation coefficients based on saliency gradients and saliency-weighted attention weight coefficients. The parameters of the detection head and prediction head refer to the set of logical weights used for target resolution and safety decision-making, including at least convolutional kernel weights for regressing bounding box coordinates, mapping matrices for extracting region features, and multilayer perceptron weights for collision probability discrimination.

[0118] In this example, during the joint training phase of the dual-robotic arm collision avoidance network, all the above parameters are uniformly encapsulated into a set of independent variables to be optimized. The gradient of the total joint loss function with respect to each component parameter in the set is calculated using an automatic differentiation framework, and gradient descent is performed to update the parameters according to a preset learning rate.

[0119] For example, if there is a significant deviation between the collision prediction probability output by the system and the true label during training iterations (such as a risk of missed detection), the gradient signal will be backpropagated to the motion clarity kernel through the prediction head. At this point, the learnable modulation coefficients in the motion clarity kernel will adaptively adjust, causing the sampling points of the deformable convolution to more accurately align with the physical contours of the robotic arm. Simultaneously, the channel enhancement intensity parameters in the illumination enhancement kernel will also be updated synchronously, enhancing the visual contrast of key collision regions by strengthening specific frequency components. Through this joint update mechanism of multi-module parameters, the entire anti-collision network ultimately achieves a synergistic optimal mapping between visual enhancement effect and anti-collision discrimination accuracy under extreme conditions such as low light and dynamic blur.

[0120] In another example, the gradient chain derivative formula (with E-block parameters) D block parameters (For example) ; ; Optimization algorithm update formula: In the formula, The learning rate is initially set to 0.001. This is the set of learnable parameters for the neural network in the current iteration period (round t) (including the weights and bias terms of each convolutional kernel). This is the set of neural network parameters for the next iteration (round t+1) after updating according to the gradient of the total joint loss function. The total loss is calculated using an automatic differentiation framework (such as PyTorch). The gradient propagation path for all learnable parameters of the network covers the "collision prediction branch, D-Collision Block, E-Collision Block, YOLO11 backbone, and detector head". The SGD optimization algorithm, combined with a momentum factor (set to 0.9), is used to iteratively update all parameters, ensuring that the parameter update direction always points to minimizing the total loss. Finally, the updated values ​​of each learnable parameter are obtained. For example, the E-block... Updated from 0.6 to 0.62 (enhanced frequency in high-risk areas), D block. The version was updated from 0.7 to 0.73 (to improve contour alignment accuracy). This enabled coordinated updates of all network parameters, enhanced the linkage optimization between kernel parameters and detection and collision judgment parameters, and ensured that the enhancement effect directly improved collision detection performance.

[0121] For example, optimization effect verification and parameter tuning steps can also be set up. Through verification and tuning, it can be ensured that the joint optimization objective function can effectively guide network training and make the model performance meet the needs of industrial applications. Specifically, a test set (including dual robotic arm operation images in low-light and dynamic blur scenes and corresponding real labels), the optimized network model, and evaluation metrics (collision detection accuracy, false alarm rate, and system latency) can be set up.

[0122] Collision detection accuracy formula False alarm rate formula In the formula, TP represents a true positive, TN represents a true negative, FP represents a false positive, and FN represents a false negative. For; collision detection accuracy; This refers to the false alarm rate.

[0123] The test set images are input into the optimized network to obtain collision detection results and system latency data. The collision detection accuracy and false alarm rate are calculated and compared with preset targets (accuracy ≥ 98%, false alarm rate ≤ 1%, latency ≤ 10ms). If the targets are not met, the weight coefficients are adjusted (e.g., increasing the accuracy). The learning rate is adjusted to 0.6 (e.g., reduced to 0.0005), and the above steps are repeated for multiple rounds of iterative optimization. For example, after 100 iterations, the collision detection accuracy reaches 98.5%, the false alarm rate is 0.8%, and the system latency is 8ms, all meeting the preset targets. In this way, it can be ensured that the network can achieve high-precision, low-latency collision detection of dual robotic arms in low-light dynamic environments.

[0124] Figure 2 This is a structural block diagram of a dual-robotic arm collaborative vision anti-collision system according to an embodiment of the present invention.

[0125] like Figure 2 As shown, the dual-robotic-arm collaborative vision collision avoidance system may include: The data acquisition module is used to acquire the joint angle data and visual image data of the dual robotic arms at the current moment; A construction module is used to determine the projection area of ​​the dual robotic arms in the image plane based on the joint angle data, and to construct a collision risk field in the spatial domain based on the projection area; The feature extraction module is used to extract features from the visual image data through a preset dual-robotic arm anti-collision network to obtain the original image features; The illumination enhancement kernel module is used to perform frequency domain modulation and spatial fusion processing on the original image features based on the collision risk field and the illumination enhancement kernel in the dual robotic arm anti-collision network to obtain the first enhanced feature. The motion clarity kernel module is used to deblur the first enhanced feature based on the motion clarity kernel in the dual robotic arm anti-collision network to generate a second enhanced feature. The prediction module is used to extract and classify the second enhanced feature based on the detection head and prediction head in the dual-arm collision avoidance network to obtain the collision prediction result of the dual-arm.

[0126] In one embodiment, the illumination enhancement kernel includes a Fourier transform module, a multilayer perceptron module, an amplitude modulation module, a spatial gating module, an inverse Fourier transform module, and a feature fusion module. The illumination enhancement kernel module is specifically used for: The Fourier transform module is invoked to perform discrete Fourier transform on the original image features to obtain the original amplitude spectrum and the original phase spectrum, and the collision risk field is subjected to Fourier transform to obtain the frequency domain risk field. The frequency domain risk field is processed by the multilayer perceptron module to generate amplitude compensation. The enhancement coefficient is calculated based on the frequency domain risk field, the preset learnable channel enhancement parameters, and the preset temperature parameters. The amplitude modulation module is then invoked to modulate the original amplitude spectrum using the enhancement coefficient and the amplitude compensation amount to generate an enhanced amplitude spectrum. The spatial gating module is invoked to perform convolution and nonlinear activation processing on the collision risk field to generate a spatial gating mask. The inverse Fourier transform module is used to perform inverse Fourier transform on the enhanced amplitude spectrum and the original phase spectrum to obtain spatial domain enhancement features. The original image features are weighted based on the spatial gating mask to obtain weighted image features, and the weighted image features and the spatial domain enhancement features are used as inputs to the feature fusion module to obtain the first enhancement feature.

[0127] In one embodiment, the motion sharpness kernel includes an edge extraction module, a motion estimation module, a saliency fusion module, and a deformable convolution module. The motion sharpness kernel module is specifically used for: The edge extraction module is invoked to extract edge gradient features from the first enhanced feature at the current time, thereby obtaining an edge intensity map; Motion difference features are calculated using the dense optical flow field between the first enhanced feature at the current moment and the first enhanced feature at the previous moment in the motion estimation module. The saliency fusion module is invoked to normalize the edge intensity map and the motion difference feature respectively, and channel splicing and convolution activation are performed on the normalized edge intensity map and motion difference feature to generate a saliency map. The deformable convolution module is used to calculate the spatial gradient of the saliency map, and the initial sampling offset is updated based on the spatial gradient and the preset learnable modulation coefficients to obtain the target sampling offset. Perform a deformable convolution operation on the first enhanced feature using the target sampling offset to obtain aligned contour features; The first enhanced feature is subjected to feature enhancement processing based on the aligned contour feature to obtain the second enhanced feature.

[0128] In one embodiment, the motion clarity kernel further includes an attention processing module and a feature fusion output module, wherein the motion clarity kernel module is specifically used for: The attention processing module is invoked to map the first enhanced feature into a query feature vector, a key feature vector, and a value feature vector through a projection matrix. Based on the spatial gradient, the saliency correlation value between pixels is calculated, along with the query feature vector and the key feature vector, and the attention weight is calculated. The attention features are obtained by weighting the value feature vector based on the attention weights. The attention features and the alignment contour features are used as inputs to the feature fusion output module to perform convolution processing on the attention features, and the convolution-processed attention features are fused and normalized with the alignment contour features to obtain the second enhanced feature.

[0129] In one implementation, the building module includes: The first calculation unit is used to calculate the joint angle data through a preset positive kinematics model to obtain the pose data of each link envelope of the dual robotic arms in the reference coordinate system. The projection unit is used to take the pose data as input to a preset camera projection model, so as to project the vertices of each link envelope onto the image plane to obtain the projection area of ​​each link envelope. The second calculation unit is used to calculate the shortest distance from each pixel in the image plane to each of the projection regions; The third calculation unit is used to calculate the risk value of each pixel by using an exponential function to calculate the preset link priority weight, the preset size coefficient, and each of the shortest distances. The collision risk field determination unit is used to determine the collision risk field of the spatial domain based on the risk value of each pixel.

[0130] In one implementation, the prediction module includes: The detection processing unit is used to detect and process the second enhanced feature through the detection head to obtain a bounding box set; The region interest alignment unit is used to perform region interest alignment on the bounding box set and the second enhanced feature to obtain each region feature vector; The prediction processing unit is used to input the feature vectors of each region into the multilayer perceptron of the prediction head for prediction processing to obtain the probability value of collision occurring in each region. The collision prediction result unit is used to use each of the probability values ​​as the collision prediction result of the dual robotic arms.

[0131] In one implementation, after the prediction module, the system further includes: The label mask acquisition module is used to acquire the real detection label, real collision label and real edge mask corresponding to the visual image data; The collision perception guidance loss calculation module is used to calculate the target detection loss based on the prediction result of the detection head on the second enhanced feature and the real detection label, and to calculate the cross-entropy loss based on the collision prediction result and the real collision label, so as to obtain the collision perception guidance loss. The illumination enhancement regularization loss calculation module is used to calculate the amplitude constraint and regularization penalty based on the original amplitude spectrum separated by the illumination enhancement kernel during the processing, the enhanced amplitude spectrum generated by modulation, and the learnable parameters, and to determine the illumination enhancement regularization loss. The edge preservation loss calculation module is used to calculate the boundary alignment penalty based on the spatial gradient of the second enhanced feature and the real edge mask to obtain the edge preservation loss. The overall joint loss function construction module is used to perform a weighted summation of the target detection loss, the collision perception guidance loss, the illumination enhancement regularization loss, and the edge preservation loss to construct the overall joint loss function. The dual-arm collision avoidance network training module is used to update the learnable parameters of the dual-arm collision avoidance network based on the gradient of the total joint loss function, so as to jointly train the dual-arm collision avoidance network.

[0132] The specific functions and examples of each module and submodule of the system in this embodiment of the invention can be found in the relevant descriptions of the corresponding steps in the above method embodiments, and will not be repeated here.

[0133] The acquisition, storage, and application of user personal information involved in the technical solution of this invention all comply with the provisions of relevant laws and regulations and do not violate public order and good morals.

[0134] This invention also provides a computer device, comprising: At least one processor; and a memory communicatively connected to said at least one processor; The memory stores instructions that can be executed by the at least one processor, which, when executed by the at least one processor, enables the at least one processor to perform the method described in any one of the embodiments of the present invention.

[0135] The beneficial effects of the computer device in this embodiment of the invention are equivalent to the beneficial effects of the above-described dual-robotic arm collaborative vision anti-collision method, and will not be repeated here.

[0136] This invention also provides a non-transitory computer-readable storage medium storing computer instructions, wherein the computer instructions are used to cause a computer to perform the method described in any one of the embodiments of this invention.

[0137] The beneficial effects of the storage medium of the present invention are equivalent to those of the above-described dual-robotic arm collaborative vision anti-collision method, and will not be repeated here.

[0138] Figure 3 A schematic block diagram of an example computer device 800 that can be used to implement embodiments of the present invention is shown. Computer device 800 is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. Computer device 800 may also represent various forms of mobile devices, such as personal digital assistants, cellular phones, smartphones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely illustrative and are not intended to limit the implementation of the invention described and / or claimed herein.

[0139] like Figure 3As shown, the computer device 800 includes a computing unit 801, which can perform various appropriate actions and processes according to a computer program stored in a read-only memory (ROM) 802 or a computer program loaded from a storage unit 808 into a random access memory (RAM) 803. The RAM 803 may also store various programs and data required for the operation of the computer device 800. The computing unit 801, ROM 802, and RAM 803 are interconnected via a bus 804. An input / output (I / O) interface 805 is also connected to the bus 804.

[0140] Multiple components in computer device 800 are connected to I / O interface 805, including: input unit 806, such as keyboard, mouse, etc.; output unit 807, such as various types of monitors, speakers, etc.; storage unit 808, such as disk, optical disk, etc.; and communication unit 809, such as network card, modem, wireless transceiver, etc. Communication unit 809 allows computer device 800 to exchange information / data with other devices through computer networks such as the Internet and / or various telecommunications networks.

[0141] The computing unit 801 can be a variety of general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of the computing unit 801 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special-purpose artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, a digital signal processor (DSP), and any suitable processor, controller, microcontroller, etc. The computing unit 801 performs the various methods and processes described above, such as the dual-arm cooperative visual collision avoidance method. For example, in some embodiments, the dual-arm cooperative visual collision avoidance method can be implemented as a computer software program, which is tangibly contained in a machine-readable medium, such as storage unit 808. In some embodiments, part or all of the computer program can be loaded and / or installed on the computer device 800 via ROM 802 and / or communication unit 809. When the computer program is loaded into RAM 803 and executed by the computing unit 801, one or more steps of the dual-arm cooperative visual collision avoidance method described above can be performed. Alternatively, in other embodiments, computing unit 801 may be configured to perform a dual-arm collaborative vision-based collision avoidance method by any other suitable means (e.g., by means of firmware).

[0142] Various embodiments of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), systems-on-a-chip (SoCs), payload-programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments may include implementations in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which may be a dedicated or general-purpose programmable processor, capable of receiving data and instructions from a storage system, at least one input device, and at least one output device, and transmitting data and instructions to the storage system, the at least one input device, and the at least one output device.

[0143] The program code used to implement the methods of the present invention can be written in any combination of one or more programming languages. This program code can be provided to a processor or controller of a general-purpose computer, special-purpose computer, or other programmable data processing device, such that when executed by the processor or controller, the program code causes the functions / operations specified in the flowcharts and / or block diagrams to be implemented. The program code can be executed entirely on the machine, partially on the machine, as a standalone software package partially on the machine and partially on a remote machine, or entirely on a remote machine or server.

[0144] In the context of this invention, a machine-readable medium can be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, apparatus, or device. A machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. Machine-readable media can include, but are not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fibers, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

[0145] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device for displaying information to the user (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor); and a keyboard and pointing device (e.g., a mouse or trackball) through which the user provides input to the computer. Other types of devices can also be used to provide interaction with the user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including sound input, voice input, or tactile input).

[0146] The systems and technologies described herein can be implemented in computing systems that include backend components (e.g., as a data server), or computing systems that include middleware components (e.g., an application server), or computing systems that include frontend components (e.g., a user computer with a graphical user interface or web browser through which a user can interact with implementations of the systems and technologies described herein), or any combination of such backend, middleware, or frontend components. The components of the system can be interconnected via digital data communication of any form or medium (e.g., a communication network). Examples of communication networks include local area networks (LANs), wide area networks (WANs), and the Internet.

[0147] Computer systems can include clients and servers. Clients and servers are generally located far apart and typically interact via communication networks. Client-server relationships are created by computer programs running on the respective computers and having a client-server relationship with each other. Servers can be cloud servers, servers in distributed systems, or servers incorporating blockchain technology.

[0148] It should be understood that the various forms of processes shown above can be used to reorder, add, or delete steps. For example, the steps described in this invention can be executed in parallel, sequentially, or in different orders, as long as the desired result of the technical solution disclosed in this invention can be achieved, and this is not limited herein.

[0149] The specific embodiments described above do not constitute a limitation on the scope of protection of this invention. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the principles of this invention should be included within the scope of protection of this invention.

Claims

1. A dual-robotic arm collaborative vision-based collision avoidance method, characterized in that, include: Acquire the joint angle data and visual image data of the dual robotic arms at the current moment; Based on the joint angle data, the projection area of ​​the dual robotic arms in the image plane is determined, and a collision risk field in the spatial domain is constructed according to the projection area; The visual image data is processed by a pre-set dual-robotic arm anti-collision network to extract features and obtain the original image features. Based on the collision risk field and the illumination enhancement kernel in the dual robotic arm anti-collision network, the original image features are subjected to frequency domain modulation and spatial fusion processing to obtain the first enhanced feature. Based on the motion clarity kernel in the dual robotic arm anti-collision network, the first enhanced feature is deblurred to generate the second enhanced feature; Based on the detection head and prediction head in the dual-arm collision avoidance network, feature extraction and classification prediction are performed on the second enhanced feature to obtain the collision prediction result of the dual-arm.

2. The method according to claim 1, characterized in that, The illumination enhancement kernel, based on the collision risk field and the dual-robotic arm anti-collision network, performs frequency domain modulation and spatial fusion processing on the original image features to obtain the first enhanced feature. The illumination enhancement kernel includes a Fourier transform module, a multilayer perceptron module, an amplitude modulation module, a spatial gating module, an inverse Fourier transform module, and a feature fusion module, comprising: The Fourier transform module is invoked to perform discrete Fourier transform on the original image features to obtain the original amplitude spectrum and the original phase spectrum, and the collision risk field is subjected to Fourier transform to obtain the frequency domain risk field. The frequency domain risk field is processed by the multilayer perceptron module to generate amplitude compensation. The enhancement coefficient is calculated based on the frequency domain risk field, the preset learnable channel enhancement parameters, and the preset temperature parameters. The amplitude modulation module is then invoked to modulate the original amplitude spectrum using the enhancement coefficient and the amplitude compensation amount to generate an enhanced amplitude spectrum. The spatial gating module is invoked to perform convolution and nonlinear activation processing on the collision risk field to generate a spatial gating mask. The inverse Fourier transform module is used to perform inverse Fourier transform on the enhanced amplitude spectrum and the original phase spectrum to obtain spatial domain enhancement features. The original image features are weighted based on the spatial gating mask to obtain weighted image features, and the weighted image features and the spatial domain enhancement features are used as inputs to the feature fusion module to obtain the first enhancement feature.

3. The method according to claim 1, characterized in that, The motion clarity kernel in the dual-robotic arm anti-collision network is used to deblur the first enhanced feature to generate a second enhanced feature. The motion clarity kernel includes an edge extraction module, a motion estimation module, a saliency fusion module, and a deformable convolution module. The edge extraction module is invoked to extract edge gradient features from the first enhanced feature at the current time, thereby obtaining an edge intensity map; Motion difference features are calculated using the dense optical flow field between the first enhanced feature at the current moment and the first enhanced feature at the previous moment in the motion estimation module. The saliency fusion module is invoked to normalize the edge intensity map and the motion difference feature respectively, and channel splicing and convolution activation are performed on the normalized edge intensity map and motion difference feature to generate a saliency map. The deformable convolution module is used to calculate the spatial gradient of the saliency map, and the initial sampling offset is updated based on the spatial gradient and the preset learnable modulation coefficients to obtain the target sampling offset. Perform a deformable convolution operation on the first enhanced feature using the target sampling offset to obtain aligned contour features; The first enhanced feature is subjected to feature enhancement processing based on the aligned contour feature to obtain the second enhanced feature.

4. The method according to claim 3, characterized in that, The first enhanced feature is enhanced based on the aligned contour feature to obtain the second enhanced feature. The motion clarity kernel further includes an attention processing module and a feature fusion output module, including: The attention processing module is invoked to map the first enhanced feature into a query feature vector, a key feature vector, and a value feature vector through a projection matrix. Based on the spatial gradient, the saliency correlation value between pixels is calculated, along with the query feature vector and the key feature vector, and the attention weight is calculated. The attention features are obtained by weighting the value feature vector based on the attention weights. The attention features and the alignment contour features are used as inputs to the feature fusion output module to perform convolution processing on the attention features, and the convolution-processed attention features are fused and normalized with the alignment contour features to obtain the second enhanced feature.

5. The method according to claim 1, characterized in that, The step of determining the projection area of ​​the dual robotic arms in the image plane based on the joint angle data, and constructing a collision risk field in the spatial domain based on the projection area, includes: The joint angle data are calculated using a preset positive kinematics model to obtain the pose data of each link envelope of the dual robotic arms in the reference coordinate system. Each of the pose data is used as input to a preset camera projection model to project the vertices of each link envelope onto the image plane, thereby obtaining the projection area of ​​each link envelope. Calculate the shortest distance from each pixel in the image plane to each of the projection regions; The risk value of each pixel is obtained by calculating the preset link priority weight, the preset size coefficient, and each of the shortest distances using an exponential function. Based on the risk value of each pixel, the collision risk field of the spatial domain is determined.

6. The method according to claim 1, characterized in that, The collision prediction result of the dual robotic arms is obtained by extracting features and classifying and predicting the second enhanced features based on the detection head and prediction head in the dual robotic arm collision avoidance network, including: The second enhanced feature is detected and processed by the detection head to obtain a set of bounding boxes; The bounding box set and the second enhanced feature are aligned with region interests to obtain feature vectors for each region. The feature vectors of each region are input into the multilayer perceptron of the prediction head for prediction processing to obtain the probability value of collision in each region. Each of the probability values ​​is used as the collision prediction result of the dual robotic arms.

7. The method according to claim 1, characterized in that, After extracting and classifying the second enhanced features based on the detection head and prediction head in the dual-robotic arm collision avoidance network to obtain the collision prediction result of the dual-robotic arm, the method further includes: Obtain the true detection label, true collision label, and true edge mask corresponding to the visual image data; The target detection loss is calculated based on the prediction result of the detection head for the second enhanced feature and the real detection label, and the cross-entropy loss is calculated based on the collision prediction result and the real collision label to obtain the collision perception guidance loss. Based on the original amplitude spectrum separated by the illumination enhancement kernel during the processing, the enhanced amplitude spectrum generated by modulation, and the learnable parameters, the amplitude constraint and regularization penalty are calculated to determine the illumination enhancement regularization loss. The boundary alignment penalty is calculated based on the spatial gradient of the second enhanced feature and the real edge mask to obtain the edge preservation loss; The target detection loss, the collision perception guidance loss, the illumination enhancement regularization loss, and the edge preservation loss are weighted and summed to construct a total joint loss function. Based on the gradient of the total joint loss function, the learnable parameters of the dual-robotic arm collision avoidance network are updated to jointly train the dual-robotic arm collision avoidance network.

8. A dual-robotic arm collaborative vision collision avoidance system, characterized in that, include: The data acquisition module is used to acquire the joint angle data and visual image data of the dual robotic arms at the current moment; A construction module is used to determine the projection area of ​​the dual robotic arms in the image plane based on the joint angle data, and to construct a collision risk field in the spatial domain based on the projection area; The feature extraction module is used to extract features from the visual image data through a preset dual-robotic arm anti-collision network to obtain the original image features; The illumination enhancement kernel module is used to perform frequency domain modulation and spatial fusion processing on the original image features based on the collision risk field and the illumination enhancement kernel in the dual robotic arm anti-collision network to obtain the first enhanced feature. The motion clarity kernel module is used to deblur the first enhanced feature based on the motion clarity kernel in the dual robotic arm anti-collision network to generate a second enhanced feature. The prediction module is used to extract and classify the second enhanced feature based on the detection head and prediction head in the dual-arm collision avoidance network to obtain the collision prediction result of the dual-arm.

9. A computer device, characterized in that, include: At least one processor; and a memory communicatively connected to the at least one processor; The memory stores instructions that can be executed by the at least one processor to enable the at least one processor to perform the method of any one of claims 1-7.

10. A non-transitory computer-readable storage medium storing computer instructions, characterized in that, The computer instructions are used to cause the computer to perform the method according to any one of claims 1-7.

Images