4128 results about "Depth map" patented technology

The invention relates to the field of defect detection, in particular to a PCB defect detection method and system based on image recognition. The method comprises the following steps: collecting a multidirectional PCB detection image, carrying out pixel-level registration correction and adaptive pixel stability compensation, and constructing a space-time stability compensation image sequence; performing reverse pyramid structure division on the space-time stability compensation image sequence, performing normalized similarity probability calculation, and constructing an initial region classification result; based on an initial region classification result, depth image visual analysis is carried out, pseudo defect comprehensive elimination optimization is carried out, and a pseudo defect purification high-confidence image is constructed; pCB connection defect identification is carried out on the pseudo defect purification high-confidence image, global defect point distribution marking is carried out, and a defect point space distribution diagram is constructed. According to the invention, high-credibility, high-precision and high-closed-loop PCB defect detection is realized.

The invention relates to an optimization method and device for sparse view angle three-dimensional Gaussian splash, and belongs to the technical field of three-dimensional reconstruction in computer vision, and the method comprises the steps: collecting a sparse view angle image; a multi-view stereoscopic vision model based on deep learning generates a geometrically consistent depth map for the sparse view image, converts the depth map into point clouds and fuses the point clouds to obtain dense point clouds; sampling dense point clouds by adopting voxel-guided farthest point sampling to obtain initialized point clouds, and constructing a three-dimensional Gaussian field; rendering the three-dimensional Gaussian field through an enhanced geometric renderer to obtain a rendering depth and a rendering normal; constructing a multi-level geometric regularization loss function, and optimizing the three-dimensional Gaussian field; and performing optimization adjustment on the three-dimensional Gaussian field based on a shape-scale constraint criterion and a two-stage adaptive opacity constraint strategy to obtain an optimized three-dimensional Gaussian field. According to the method, the problems of initialization failure, insufficient geometric supervision and element out-of-control of 3D Gaussian splashing under the sparse view angle are solved.

The embodiment of the invention provides an unhooking robot control method based on multi-source visual fusion, which is applied to the technical field of robot control and comprises the following steps: acquiring an RGB image, a depth image, an infrared image and IMU data through a multi-source sensing system mounted at the tail end of a robot; carrying out feature fusion identification by adopting a double-branch neural network, and outputting the boundary contour of the lifting hook and the three-dimensional coordinates of the optimal grabbing point; the visual coordinates are unified to a robot base coordinate system through a registration correction mechanism; a Transform prediction model is constructed based on the visual and inertial signals, and future pose changes of the lifting hook are estimated; a feedforward control track is generated to counteract swing of the lifting hook, and track correction is carried out in combination with visual servo feedback; and a joint instruction is generated through path planning and inverse kinematics solution, and the mechanical arm is driven to complete precise unhooking operation. According to the method, the recognition precision, the anti-interference capability and the operation success rate of unhooking operation in complex illumination and dynamic environments are effectively improved.

The invention provides a posture recognition algorithm for any object under a monocular camera and an application system, and the algorithm comprises the steps: S1, constructing a target three-dimensional model, carrying out the multi-view annular shooting image collection of a target, and generating a dense grid model through feature extraction, matching, posture calculation and a multi-view geometric method; s2, generating an image depth map, and predicting depth information of a target in a motion process based on a monocular image sequence; s3, extracting a target image mask, and generating a target area mask graph through an image encoder, a prompt encoder and a mask decoder; and S4, executing attitude estimation, performing attitude initialization, correction and screening by combining the three-dimensional model, the depth map and the mask map, and outputting a six-degree-of-freedom attitude result of the target. According to the method, the target is subjected to annular shooting modeling through the method based on multi-view geometry, the three-dimensional model of the target is generated, attitude estimation is achieved in combination with the image mask and the depth map, the generalization ability of an attitude estimation algorithm in an actual scene is improved, and the application range of the attitude estimation algorithm in the actual scene is widened.

The invention provides a mountainous area tunnel construction safety intelligent monitoring and early warning method and system, and relates to the technical field of construction safety monitoring, and the method comprises the steps: collecting visible light and depth images of tunnel surrounding rock, and carrying out the segmentation and extraction of crack features through a depth attention network after image preprocessing and data fusion; extracting parameter time sequence data based on the crack spatial position and the type feature; determining fracture evolution characteristics and critical state parameters by combining wavelet transform and stress-rate coupling analysis; and adopting deep reinforcement learning to calculate the instability probability and generate early warning information. According to the invention, intelligent identification, instability prediction and risk early warning of tunnel surrounding rock cracks are realized, and the safety monitoring accuracy and early warning timeliness are improved.

The invention provides a multi-view three-dimensional point cloud reconstruction method and device based on DPE-SE depth estimation, and relates to the technical field of computer vision and three-dimensional reconstruction. The method comprises the following steps: acquiring multi-view image data; preprocessing the image; inputting the preprocessed image into a DPE-SE-based depth estimation model, carrying out key constraint on an edge region through a semantic edge guiding mechanism, carrying out adaptive propagation updating on a weak texture region, realizing accurate depth estimation, and generating a multi-view depth result; then geometric consistency check and multi-scale depth fusion are performed on a multi-view depth result, and a dense depth map is constructed; and finally, performing three-dimensional back projection reconstruction and point cloud optimization processing, and outputting high-quality point cloud data containing three-dimensional coordinates and confidence information. According to the method, the problems of edge mismatching and depth voids are remarkably improved in complex illumination, weak texture and shielding environments, the continuity and structural integrity of the point cloud are improved, and technical support is provided for unmanned aerial vehicle surveying and mapping, building detection and digital twin modeling.

Provided is a defect detection method and device, computer equipment and a storage medium. The method includes: acquiring an RGB image, a depth image and a sample label of a detection object sample; performing feature map extraction and feature map fusion on the RGB image and the depth image by a feature extraction network of the defect detection model, to obtain a fused feature map; performing defect detection based on the fused feature map by a feature reconstruction network of the defect detection model, to obtain a defect score map, wherein the defect score map being obtained by fusing a global defect score map which is generated based on a global defect detection network with a local defect score map which is generated by a local defect detection network; and updating parameters of the defect detection model based on the defect score map and the sample label.

The invention discloses a three-dimensional scene reconstruction method and device based on large model geometric prior, and a medium, and aims to solve the problems that a conventional 3DGS is liable to have artifacts and detail loss in geometric discontinuity, data redundancy and illumination variation scenes, and predicts a dense depth map and a normal map from a monocular image by using a pre-trained large model. The position and form of the Gaussian kernel are constrained as additional geometric priori; a primitive adjustment strategy based on kernel density estimation is introduced in the training stage, small Gaussian primitives with similar structures and adjacent spaces are combined into a large Gaussian primitive, the rendering quality is kept, redundancy is reduced, and the volume of the model is reduced; an exposure coefficient is adaptively estimated for each input image, an exposure compensation image loss function is constructed, and floating artifacts caused by illumination differences at shooting moments are eliminated. Experiments show that compared with the prior art, the method improves the three-dimensional reconstruction precision and real-time rendering quality of complex illumination and less-texture areas in a public data set and an unmanned aerial vehicle aerial photography scene.

The invention discloses an operation and maintenance manipulator intelligent control method and system based on visual identification, and relates to the technical field of intelligent manipulator control, and the method comprises the steps: collecting RGB image data and depth image data of an operation and maintenance operation area, and obtaining a standardized image matrix and a mapping relation matrix; inputting the standardized image matrix into an improved ResNet residual network model, generating a comprehensive feature descriptor, and calculating a spatial position coordinate and an attitude angle of the target equipment based on the mapping relation matrix; based on the current joint angle state of the manipulator, an improved Jacobian matrix inverse kinematics algorithm is used for solving a target angle sequence of each joint, a preset operation mode library is matched based on the comprehensive feature descriptor, and a grabbing force parameter and a motion speed parameter are determined; and converting the target angle sequence into a control instruction, and sending the control instruction to each joint driver of the manipulator to drive the manipulator to complete action planning. According to the invention, full-process automation from environment perception to task execution is realized.

The invention discloses a double-station robot sorting optimization method and system based on digital twinning and a terminal, double improvement of sorting efficiency and safety is achieved by constructing a digital twinning driven collaborative operation system, and the method comprises the steps that firstly, a laser radar and a polarization camera are used for forming a composite sensing unit; geometric morphology, material reflection characteristics and spatial pose data of a target object are synchronously obtained, and are input into a digital twin engine after time synchronization processing; an engine constructs a virtual sorting scene containing a material attribute database based on a physical rendering technology, sub-millimeter-level space registration is achieved through feature fusion of a binocular stereoscopic vision depth map and geometric parameters, high-precision digital twin stations are generated, a system plans a space-time constraint trajectory of a double-station robot in a virtual environment, and a target object is obtained. And an integrated discrete event simulation engine performs operation time sequence conflict prediction, and when a space overlapping risk is detected, an alternative path containing a dynamic obstacle avoidance strategy is generated through a trajectory re-planning algorithm.

Disclosed in the present invention is a fruit stem pose recognition method used for a pomelo fruit picking apparatus, the method comprising the following steps: preprocessing image data of a scene to be recognized, so as to obtain preprocessed image data, the image data comprising an RGB map and a depth map; inputting the preprocessed image data into an optimized instance segmentation model to obtain a depth map of a single pomelo fruit; on the basis of the depth map of a single pomelo fruit, recognizing an elliptical fruit pose feature region, and further using the Hough ellipse detection algorithm to obtain an ellipse tilt angle of the fruit pose feature region; and, by means of the ellipse tilt angle of the fruit pose feature region, calculating and obtaining the position and orientation of a pomelo fruit stem. The method integrates a plurality of advanced technologies to achieve optimization of a whole process from image data preprocessing to fruit stem pose accurate recognition, thus improving the recognition accuracy and working efficiency of automated pomelo fruit picking apparatuses, and reducing the mis-picking rate and the fruit damage rate.

The invention relates to the technical field of constructional engineering, and discloses a concrete crack intelligent identification and analysis platform based on image and point cloud fusion, the platform operates a concrete crack intelligent identification and analysis method, and the method comprises the following steps: S1, synchronously collecting image data and point cloud data of a concrete structure in the same scene; s2, establishing a unified world coordinate system and generating a depth map corresponding to the image; s3, generating image domain crack candidates; s4, generating a depth domain crack candidate; s5, performing weighted fusion on the image domain crack candidate and the depth domain crack candidate to generate a fusion crack response; s6, extracting a crack skeleton; s7, obtaining a crack three-dimensional model; and S8, selecting an optimal view angle to trigger re-acquisition of the crack area. Through a closed-loop feedback mechanism, an optimal view angle is selected for re-acquisition by calculating a comprehensive utility value after preliminary acquisition, so that information insufficiency caused by illumination, angle or sparse data is effectively made up.

The invention discloses a tunnel disease recognition model training method and system based on a point cloud and an image, and the method comprises the steps: synchronously collecting tunnel point cloud and image data, carrying out the calibration and registration, achieving the spatial alignment, preprocessing the point cloud, generating a gray-scale image and a depth image, collecting continuous images through a line-scan digital camera, generating a spliced image, and carrying out the recognition of tunnel diseases. Partitioning a large-size image after multi-image space-time synchronization; a multi-branch network is constructed, point cloud geometry and image texture features are extracted by using Point Net / 3DCNN and CNN / Transform respectively, and semantic collaborative fusion is realized through an intermediate layer fusion module; parameters are adjusted according to errors through self-adaptive training, manual labeling dependence is reduced in combination with transfer learning and the like, and convergence is accelerated through online iteration, joint loss and self-adaptive weight to improve generalization; the performance of the model is evaluated through field testing and indexes in various tunnel environments, and the structure is optimized according to data to ensure that engineering is feasible and efficient. The model can accurately identify various diseases on different tasks, and can adapt to complex and changeable working conditions in tunnel detection.

The invention discloses a shield muck volume estimation method and system based on image processing, and particularly relates to the technical field of tunnel engineering monitoring, and the method comprises the steps: obtaining muck RGB and depth images through an image collection device, generating high-precision three-dimensional point cloud data through the combination of laser scanning and a multispectral technology, and calculating the muck volume through a slicing method. And fusing the mass flow and water content data of the belt weigher, dynamically generating a calibration factor, and correcting a volume estimation result in real time. The system comprises an image acquisition module, a point cloud processing module, a volume calculation module and a dynamic calibration module, has the characteristics of automation, high precision, real-time feedback and the like, effectively solves the problems of large error and slow response of a traditional manual metering and single weighing mode, and is suitable for continuous and accurate monitoring of the volume of muck in the shield construction process.

The invention belongs to the technical field of unmanned aerial vehicle autonomous navigation, provides an unmanned aerial vehicle trajectory planning method and an unmanned aerial vehicle design applying the method, and aims to realize real-time and efficient environmental perception and unmanned aerial vehicle autonomous obstacle avoidance trajectory generation. A group of primitive sets is predefined in a three-dimensional state space to explore the whole search space so as to realize complete coverage of a feasible region, multi-mode perception input of'depth image, current state and target direction 'is adopted, and the depth image is acquired by a depth camera; the current state is obtained by the airborne vision positioning module; multi-modal sensing input is processed by a deep learning network, future expected position, speed and acceleration information is calculated according to output of the deep learning network and serves as input of a bottom layer controller of the unmanned aerial vehicle for trajectory tracking, and finally obstacle avoidance flight in a complex environment is achieved. The method is mainly applied to unmanned aerial vehicle design and manufacturing occasions.

The invention discloses a multi-modal three-dimensional target detection method and device based on image-guided depth completion and multi-stage iteration fusion, and the method comprises the steps: firstly, predicting a dense depth map through an image-guided depth completion module by using the context information of an image, and carrying out the image-guided depth completion; the depth map is fused with a sparse depth map generated by the laser radar in a mask guiding manner, so that a high-quality complemented depth map is generated, and the accuracy of subsequent view angle conversion is improved; and then, through a multi-stage iterative fusion module, iterative fine-grained fusion is carried out on the converted image aerial view features and point cloud aerial view features, so that modal conflicts are effectively relieved, and the expression ability of fusion features is enhanced. According to the invention, through accurate depth information completion and efficient multi-modal feature fusion, the precision and robustness of three-dimensional target detection can be significantly improved, and especially the effect is more obvious when a long-distance target or a blocked target and other difficult targets are processed.

The invention discloses a joint denoising method for robot visual motion prediction, and the method comprises the steps: constructing a unified generative model through fusing an image and a depth map collected by a depth camera, motion data collected by CAN line communication of a Piper mechanical arm, and a tactile image collected by a Gelsight Mini tactile sensor; the method comprises two steps of data acquisition and input coding, and joint denoising and generation: firstly, multi-modal data are coded into low-dimensional potential representation, and then future images, depth maps, tactile data and robot actions are cooperatively predicted through a joint denoising framework based on Transform. A mask self-attention mechanism is innovatively introduced, information interaction between modes is dynamically adjusted, action generation is guided through tactile feedback, and the force control precision is improved. The model adopts a de-noising diffusion probability loss function to jointly optimize multi-modal prediction, so that the output consistency is ensured. According to the method, the robustness and the accuracy of flexible operation of the robot are remarkably improved.

The invention belongs to the technical field of unmanned aerial vehicle autonomous navigation, and discloses a multi-modal unmanned aerial vehicle trajectory prediction method and system based on bipolar optimization, and the method comprises the steps: collecting and preprocessing multi-modal data, synchronously obtaining an RGB image, a depth image and unmanned aerial vehicle state information, and carrying out the normalization processing; multi-modal feature extraction and fusion, wherein RGB, depth and IMU features are fused through a deep convolutional network and a cross-modal attention mechanism; generating an environment cost map, constructing a differentiable grid map based on the depth information, and adding a navigation guide item; performing bipolar optimization trajectory prediction, predicting key points through an upper neural network, and dynamically optimizing the trajectory by combining safety and exploratory performance through a lower differentiable optimizer; and performing safety assessment and output, and outputting a track or triggering an obstacle avoidance mechanism after comprehensively assessing the collision risk and the dynamic feasibility. According to the invention, through multi-modal fusion and bipolar optimization, the sensing precision, the real-time performance and the security are significantly improved.

The invention discloses an automobile part quality detection method and system based on artificial intelligence visual inspection, and belongs to the field of artificial intelligence machine visual inspection, and the method comprises the steps: firstly, carrying out the registration of a collected RGB image and a depth image, and extracting a part region through a saliency detection network; two-dimensional key points are extracted based on an RGB region image and are matched with key points of a three-dimensional model, an initial three-dimensional attitude is obtained by adopting a PnP algorithm, iterative registration is performed with the three-dimensional model in combination with a point cloud generated by a depth image, and a fine three-dimensional attitude is obtained. And calculating a geometric transformation matrix from the part to a standard front view attitude according to the attitude, and performing attitude correction on the RGB and depth region image. And then matching the corrected image with a standard template image by using a feature detection and matching network so as to correct the position of the detection window. And finally, the three-dimensional size of the part is calculated in the corrected detection window in combination with the depth value, and tolerance judgment is carried out. And the precision, the robustness and the automation level of online detection of the automobile parts can be obviously improved.

The invention relates to the field of computer systems, and provides an intelligent identification method for flaws of a plush fabric based on multi-modal fusion. RGB, infrared and depth images during movement of the plush fabric are collected, multi-modal local features are extracted through a parallel encoder, and the multi-modal local features are fused into a joint feature map after semantic projection alignment; a lightweight YOLOv11 model is constructed, dynamic characteristics and a multi-scale mixed attention mechanism are combined, and flaw types, sizes and positions are recognized; setting a dynamic compensation mechanism to output defect labels and quality grades; according to the invention, through synchronous acquisition and feature alignment of multi-modal images, in combination with a lightweight YOLOv11 model and a multi-scale attention mechanism, multi-modal cooperative flaw detection of the plush fabric in a continuous moving state is realized.

The invention discloses a laser radar and multispectral polarization imaging fused target identification system and method. The system comprises a sensor configuration and data preprocessing module, a cross-modal feature extraction and fusion module and a multi-task output and optimization module. The sensor configuration and data preprocessing module performs time-space synchronization processing on the collected original optical signals and laser signals in the environment to obtain multispectral image data and laser radar point cloud data; the cross-modal feature extraction and fusion module performs feature extraction and fusion enhancement processing on the multispectral image data and the laser radar point cloud data, and outputs high-dimensional semantic enhancement point cloud representation containing image semantics and point cloud geometry; and the multi-task output and optimization module processes the high-dimensional semantic enhanced point cloud representation and outputs a three-dimensional target recognition result, target speed information and a pixel-level depth map. Through module design and data processing, the defects in the prior art are overcome, and the accuracy of target recognition is improved.

The invention provides a multi-modal information fused steel pipe inner surface defect area segmentation method and system, and the method comprises the steps: obtaining a steel pipe inner surface defect RGB image and a depth map at the same time, and carrying out the preprocessing and marking; constructing a support data set and a query data set; the support and query image feature extractor is used for respectively extracting support image multi-scale aggregation features FS, support image multi-mode semantic features IS, query image multi-scale aggregation features FQ and query image multi-mode semantic features IQ by sharing the multi-mode feature extraction backbone network; the FS, the IS, the FQ and the IQ are input into a multi-feature fusion device, a graph semantic guide module is supported to generate class guide features FA by using the FS and the MS, a similar prior feature generation module is supported to generate similar prior features FM by using the IS, the IQ and the MS, the FA, the FM and the FQ are spliced on a channel dimension, and a fusion feature graph is output and decoded by a multi-mode decoder to obtain defect area segmentation output. The method can be used for segmenting the defect area on the inner surface of the steel pipe.

The invention discloses an industrial robot intelligent obstacle avoidance control method and system based on visual identification, and relates to the technical field of robot obstacle avoidance control, and the method comprises the steps: carrying out the calibration of a multi-mode visual sensor, obtaining an initial depth map and point cloud information, and combining the visual identification and depth compensation technology; recognizing and positioning obstacles in the operation area, and constructing a dynamic environment map layer; and the current robot state is collected, kinematics calculation and collision distance analysis are carried out, whether obstacle avoidance operation needs to be executed or not is judged, if yes, an obstacle avoidance path is generated in combination with the dynamic environment map layer and the target point location, feasibility verification is carried out after the path is generated, and the path passing the verification serves as an execution track to be issued to the control module. According to the invention, the recognition precision and depth perception integrity of the industrial robot on obstacles in a complex environment are improved, high feasibility of path planning and high-reliability obstacle avoidance capability in a dynamic environment are realized, and the intelligent decision-making level and operation safety of the system are remarkably enhanced.

The invention relates to the technical field of virtual reality, in particular to an indoor structure reconstruction method based on a panoramic image scene understanding algorithm, and the method comprises the steps: S11, obtaining a plurality of equidistant columnar projection panoramic images of a current indoor scene through a panoramic camera or a panoramic image splicing algorithm; s12, performing semantic segmentation on the collected panoramic image by using SAM, deducing indoor ceiling, floor and wall areas of the panoramic image on the basis of a semantic segmentation result, and generating a multi-channel semantic graph; and S13, inputting the acquired RGB panoramic images and the multi-channel semantic map into a pre-trained panoramic image depth estimation model to obtain a depth map corresponding to each panoramic image. The method is used for automatically generating an indoor structure model, the scheme comprises the core steps of semantic segmentation and layout reasoning, depth estimation and point cloud construction, structure optimization and indoor model reconstruction and the like, and a complete indoor panoramic image scene understanding scheme is formed.

Embodiments of the present disclosure provide a three-dimensional scene reconstruction method and apparatus, a device, a medium, and a program product. The three-dimensional scene reconstruction method comprises: acquiring a scene image collected for a three-dimensional scene; on the basis of the scene image, determining sparse point cloud data and camera parameter information corresponding to the scene image; performing model initialization on the basis of the sparse point cloud data to obtain a current three-dimensional data model; performing rendering on the basis of the camera parameter information and the current three-dimensional data model to obtain a current color map, a current depth map and a current normal map from the perspective of the scene image, and determining a pseudo normal map from the perspective of the scene image on the basis of the current depth map; and on the basis of the current color map, the current normal map, the pseudo normal map, and an actual color map, training the current three-dimensional data model to obtain a trained target three-dimensional data model. By means of the technical solution provided by the embodiments of the present disclosure, higher-quality automatic reconstruction of three-dimensional scenes can be achieved, reducing reconstruction costs, and improving the level of detail and rendering quality of scene models.

The invention provides a method for detecting the thickness increment of a sprayed coating, and relates to the technical field of industrial spraying. The method comprises the following steps: acquiring a depth image of a current sprayed coating, and determining first point cloud data of the current sprayed coating according to the depth image; registering the first point cloud data with second point cloud data, wherein the second point cloud data is depth point cloud data of an initial spraying coating corresponding to the current spraying coating; determining the offset of matching points in the first point cloud data and the second point cloud data according to a registration result; and determining the thickness increment of the current sprayed coating according to the offset of the matching point. According to the invention, the thickness increment of the sprayed coating can be accurately and rapidly detected.

The invention discloses an electronic cabin assembly quality detection method based on three-dimensional vision. The method comprises the following steps: firstly, generating data for comparison during detection for a standard electronic cabin; shooting all to-be-detected objects in the to-be-detected electronic cabin to obtain a depth image and a color image of each to-be-detected object; according to the depth image and the color image of the to-be-detected object, combining with the data of the standard electronic cabin for comparison during detection; and whether screws of the electronic cabin to be detected are neglected and not installed in place, whether cable plugs are neglected and not installed in place, whether connectors and cable plugs in the wiring unit are wrongly matched, and whether the bending radius of cables is too small are detected in sequence. Through fusion of depth information, three-dimensional quantitative detection of the assembly quality is realized, and the detection precision, reliability and automation level are significantly improved.

The invention discloses an intelligent water quality monitoring and evaluating system based on image recognition, which relates to the technical field of environment monitoring, and comprises a fusion module for controlling a polarizer array to rotate based on an environment parameter set, capturing water surface optical images in different polarization directions, and performing image reconstruction through a polarization characteristic tensor decomposition method, generating a high-definition water surface image after reflection suppression; the three-dimensional diagram generation module is used for acquiring a morphology stripe image of the water surface through a DLP projection method, and generating a three-dimensional depth diagram in combination with the high-definition water surface image; the pollution identification module is used for correcting the three-dimensional depth map based on the real-time water temperature data, identifying spatial distribution of pollutants through a dynamic structure element opening and closing algorithm and generating a three-dimensional pollution distribution map; according to the method, mirror reflection noise is suppressed through multi-angle polarization image fusion, and the distinguishability of pollutant characteristics is remarkably enhanced while the image resolution is kept.

The invention belongs to the technical field of computer vision and automatic driving, and discloses a monocular depth prediction method based on double-stage jump attention and dynamic position coding, which comprises the following steps: S1, fusing shallow 1 / 8-1 / 16 features and middle 1 / 32 global features respectively through two series-connected jump attention modules, so as to obtain two jump attention modules; s2, performing depth classification probability prediction on the multi-scale fusion features, performing weighted summation in combination with non-uniform depth candidate values, and outputting a continuous weighted depth map; and S3, generating a dynamic position code based on the weighted depth map interpolation, adding the dynamic position code with the feature after Transform coding, and outputting the final depth feature of geometric perception. Therefore, the semantic conflict of multi-scale feature fusion is reduced, the depth prediction precision is improved, and the accuracy and reliability of a prediction result are enhanced.

The invention discloses a robot arm control method and device, equipment, a medium and a product, and the method comprises the steps: obtaining a pre-training dynamic model and a real visual depth map of a real robot arm visual angle, and the pre-training dynamic model is used for completing a specified task; determining a joint control value according to the real vision depth map and a pre-training kinetic model; hybrid action control is carried out based on the joint control value, and remote target navigation is carried out on the response action of the real robot arm through a visual navigation model; and the controlled simulation target image and the actual image are obtained, feature matching and closed-loop estimation are carried out based on the simulation target image and the actual image, and pose error compensation is carried out on the action of the real robot arm. The motion is observed and deduced through a real visual depth map, then real and simulated mixed motion control is carried out to reduce a visual and dynamic gap, and finally pose error compensation is carried out. And the pose error of the arm is reduced, and a start pose guarantee is provided for downstream control.