278 results about "Vision Disparity" patented technology

The invention particularly relates to a binocular vision-based three-dimensional reconstruction method, which comprises the following steps of: calibrating a binocular camera based on an improved Zhang Zhengyou calibration method to obtain internal and external parameters and a distortion coefficient of the camera; performing stereo correction on the image by using the internal and external parameters of the camera and the distortion coefficient obtained by calibration, so that the binocular image meets an epipolar constraint condition; a multi-strategy optimized semi-global stereo matching algorithm is adopted to process the image after stereo correction, and a disparity map is generated; based on the generated disparity map, generating a three-dimensional point cloud through a triangulation principle; carrying out anti-interference processing and registration optimization on the three-dimensional point cloud; and performing global splicing on the three-dimensional point clouds subjected to anti-interference processing and registration optimization based on a sequential registration error sharing strategy to complete three-dimensional reconstruction. According to the method, the key problems of large calibration error, weak texture matching failure, point cloud noise sensitivity and registration accumulative error in a traditional method are solved, and the reconstruction precision and stability are remarkably improved.

The invention relates to the technical field of image data processing and intelligent control, and discloses a patch board spot welding track intelligent control method and system based on visual perception. According to the method, synchronous image streams are collected through a binocular vision sensor, and epipolar correction image pairs are generated through timestamp synchronization, ROI extraction and epipolar correction processing; calculating a disparity map by adopting a stereo matching algorithm, and constructing a compensated three-dimensional point cloud model based on deformation vector field fusion multi-frame point cloud data of a radial basis kernel function; welding spot position error vectors are generated by extracting welding spot feature points and performing spatial filtering optimization; and carrying out inverse kinematics calculation on the mechanical arm by adopting a damping least square method, carrying out safety constraint optimization in combination with prospective collision risk assessment and real-time pose data, and generating a trajectory compensation instruction. According to the method, the problems of dynamic deformation compensation and motion safety in patch plate welding are solved, the submillimeter welding spot positioning precision is achieved, and the welding quality stability and the system robustness are improved.

The invention provides a robust stereo matching method fusing monocular semantic prior and multi-dimensional expert aggregation, relates to the technical field of computer vision and stereo matching, establishes a robust stereo matching framework fusing monocular semantic prior and multi-dimensional expert aggregation, and comprises a monocular branch and a binocular branch, inputting a stereo image into a robust stereo matching framework based on fusion of monocular semantic prior and multi-dimensional expert aggregation for matching calculation, wherein the matching calculation comprises the following steps: extracting semantic prior features of the input image by using a monocular branch; the method comprises the following steps: extracting a geometric enhancement feature map of a stereo image by using binocular branches, generating a matching cost body based on the geometric enhancement feature map, and carrying out refined iterative updating on the matching cost body to obtain a matched disparity map. According to the method, monocular and binocular depth estimation is cooperatively supported in a unified network architecture, and a parallel monocular depth prior path is utilized to actively guide and strengthen a core binocular matching process.

The invention relates to the technical field of intelligent maintenance of tunnel electromechanical facilities, in particular to a tunnel lamp recognition and spatial positioning method based on binocular vision, which comprises the following steps: synchronously acquiring tunnel environment images through left and right cameras subjected to polar horizontal correction, inputting the images into a lightweight tunnel lamp detection model, and detecting an area image containing a lamp bounding box; extracting feature points in the bounding box by adopting a multi-scale adaptive FAST feature extraction algorithm, describing the feature points by rotating an invariant binary descriptor, and generating a disparity map in combination with an adaptive Hamming distance matching algorithm; calculating three-dimensional space coordinates of the lamp according to the disparity map by using a binocular camera triangulation principle and combining camera parameters; when the ambient illumination is lower than a threshold value, a multispectral image fusion module is started to enhance the feature matching robustness; and when continuous matching fails, visible light communication assisted positioning is started. According to the invention, the problem of low positioning precision of the lamp in a complex tunnel environment is solved, and the reliability and efficiency of identification and positioning are improved.

The invention discloses a photoelectric fusion beam control method and device based on a reconfigurable intelligent reflecting surface. The method comprises the following steps: acquiring internal and external parameter matrixes of a binocular camera and a binocular-RIS coordinate transformation matrix; capturing a binocular image stream of a detection target by using a binocular camera and executing stereo matching to obtain a disparity map and depth data of a binocular image; detecting a target image in the binocular image and outputting bounding box information of a center point of the target image; calculating target image coordinates through the depth data and the bounding box information, and converting the target image coordinates into three-dimensional space coordinates under an RIS coordinate system so as to determine the spatial position of the target; generating an RIS phase codebook according to the target spatial position, wherein the RIS phase codebook comprises a near-field phase compensation item; and loading the phase codebook to the RIS unit to form a target beam so as to perform directional tracking on a target. According to the technical scheme of the invention, high-precision beam control and stable positioning and tracking of the target can be realized in a complex multi-target environment.

The invention provides a short-baseline binocular three-dimensional human body posture reconstruction method and system, and the method comprises the steps: collecting original left and right image pairs, and carrying out the detection of a human body region; carrying out human body image pair matching and scale transformation on the detected human body region; parallax features are obtained based on the processed left and right human body area image pairs, feature fusion is carried out on the parallax features and human body features extracted by the three-dimensional human body posture estimation sub-network, a parallax image initial result is obtained based on feature fusion information, a loss function is established with the human body features to carry out parallax optimization, and an optimized parallax image and a scene depth image are obtained; human body features are obtained based on left and right human body area image pairs, feature fusion is carried out on the human body features and parallax features, two-dimensional human body joint points are obtained through two-dimensional posture estimation according to feature fusion results, and a three-dimensional human body posture initial result is obtained based on binocular camera parameters and a triangulation calculation method; and establishing a loss function with the scene depth map and human body prior supervision, and performing three-dimensional human body posture optimization to obtain an optimized three-dimensional human body posture.

The invention belongs to the technical field of data measurement, and discloses a high-speed railway ballastless track construction measurement method, which comprises the following steps of: carrying out internal reference and external reference calibration on a binocular camera of a track detection trolley; obtaining related data containing a binocular image, and carrying out space-time alignment processing on the related data so as to construct a prediction state vector; carrying out stereo matching on the binocular image to generate a prediction disparity map, and carrying out three-dimensional reconstruction on the prediction disparity map to obtain a track point cloud map; and extracting a track inner side point coordinate sequence and a track height program sequence from the track point cloud picture, and generating a corresponding track gauge table and a corresponding smoothness table. According to the invention, vision, three-dimensional reconstruction and inertial navigation technologies are integrated, and integrated detection of gauge, elevation and smoothness parameters is realized.

The invention discloses a pavement disease size accurate quantification method based on binocular vision, and the method specifically comprises the steps: collecting pavement image data through a vehicle-mounted binocular camera, achieving the real-time detection of a disease target through an improved RT-DETR model, and outputting the disease type and detection frame information; after a pavement disease target is detected, pixel-level segmentation is carried out on a disease area in the detection frame based on a semantic segmentation model, and a disease contour mask is extracted; an improved IGEV-Stereo stereo matching algorithm is used for calculating a disparity map, depth information is output based on camera calibration parameters and a disparity estimation result, and conversion from pixel coordinates to three-dimensional coordinates is achieved; and the binocular depth information and a disease detection segmentation result are combined to realize accurate size quantification of typical road surface diseases with different characters. Through the binocular stereoscopic vision technology, automatic detection and precise quantification of pavement diseases can be realized, the cost can be effectively reduced while the disease quantitative evaluation precision is improved, and decision support is provided for road management and maintenance.

The invention discloses a target recognition method based on multi-view vision and a target recognition model training method, belongs to the technical field of target recognition, and solves the problem that global information is easy to lose due to the fact that a binocular vision method adopted in an existing method mainly depends on target detection after disparity maps and image splicing in the aspect of feature extraction. The method comprises the steps of performing pre-recognition processing on a target scene image based on an improved deep belief network model, responding to an auxiliary adjustment instruction by a multi-view vision acquisition system, fusing a pre-recognition result and a scene compensation image based on a spatial stamp alignment algorithm, and performing scene compensation. The target recognition model extracts global information of a scene fusion set in a parallel mode of combining a bidirectional feature pyramid network architecture with a YOLO algorithm; according to the invention, accurate identification of the target object type and the target object position in the three-dimensional space of the target scene is realized by adopting a cooperative cooperation mode of the improved deep belief network model and the target identification model, and the problems of global information loss and information insufficiency of the scene fusion set during target identification are avoided.

The invention discloses a water level measurement method based on binocular stereoscopic vision. The method comprises the following steps that model selection is conducted on a binocular camera according to an actual measurement scene; horizontally erecting a binocular camera on a river bank, adjusting a pitch angle to enable left and right views to cover a water surface area under a water level full range, and synchronously acquiring water surface images of the left and right views by using the binocular camera; acquiring a corresponding disparity map pixel by pixel by adopting an iterative optimization semi-global stereo matching method; based on a calibration result of the binocular camera, obtaining a three-dimensional point cloud coordinate value of the water surface image by using a relationship between parallax and depth; performing plane fitting on the point cloud by using an RANSAC algorithm to generate a horizontal plane equation; calculating the distance from the optical center of the left camera to the horizontal plane, and converting the elevation of the camera to obtain the water level. According to the invention, the shore-based arrangement of the instrument can be realized, the water level can be measured remotely in an inclined manner, the measurement does not depend on any reference object in contact with a water body such as a water gauge, the frame-while-frame measurement can be realized in severe environments such as a high flood period, and the device is particularly suitable for water level emergency monitoring.

The invention discloses a position monitoring method and system based on different-source image registration and binocular positioning, and relates to the field of position monitoring. The method comprises the following steps: performing target detection and instance segmentation on image data of a dangerous area acquired by left and right binocular cameras to construct a mask image set; matching and processing the mask image of the target to obtain a target matching and correction image; introducing angle fusion score screening to obtain an optimal feature point, carrying out image registration on the optimal feature point according to the bit difference degree, and carrying out fusion display; calculating an optimal binocular parallax to obtain a target three-dimensional coordinate, and generating a point cloud; and carrying out background integration on the personnel target point cloud image according to the obtained crane space position to form a complete real-time personnel monitoring point cloud image. Through a binocular vision technology combining infrared and visible light cameras, a YOLOv8 algorithm and three-dimensional point cloud modeling, real-time monitoring of a hoisting operation dangerous area is realized, and operation safety and monitoring efficiency are remarkably improved.

The invention relates to the technical field of computer vision, and particularly provides a tunnel video stream three-dimensional modeling system based on binocular stereo matching and SLAM (Simultaneous Localization and Mapping), which comprises a multi-modal image acquisition unit, a binocular infrared camera and an RGB (Red, Green and Blue) camera are configured to synchronously acquire an infrared image pair, an RGB image pair and inertial measurement unit data of a tunnel environment, the multi-source data is uploaded to the cloud processing platform in real time through the wireless transmission module; based on a dynamic switching mechanism of environmental perception, performing adaptive fusion pose estimation of a feature point method and a direct method on input multi-modal image data; the stereo matching and dense reconstruction unit is used for calculating a disparity map by adopting an improved self-adaptive window stereo matching algorithm, generating a three-dimensional point cloud in combination with the pose information output by the pose estimation unit, and constructing a global consistency point cloud model of the tunnel scene through a time sequence point cloud registration and splicing algorithm; and a multi-algorithm target detection and semantic fusion unit. The method can meet the requirements of precision and robustness of tunnel reconstruction.

The invention provides a civil engineering construction period structure safety monitoring method and system based on image recognition, and belongs to the technical field of combination of computer vision and civil engineering safety monitoring. The method comprises the following steps: S1, collecting binocular image data of a construction site, and constructing a data set of a building material; s2, performing target detection and instance segmentation on the building material in the data set based on an improved Mask RCNN algorithm to obtain a mask and space coordinates; s3, carrying out stereo matching on the binocular image, generating a disparity map by adopting an SGM algorithm, and converting the disparity map into three-dimensional point cloud data; s4, calculating the volume of the building material based on the point cloud data, and optimizing a volume measurement result in combination with a correction coefficient k of the shooting angle theta; and S5, estimating the weight of the building material according to the material density and the corrected volume, and outputting a load distribution diagram. The method can replace traditional manual measurement, remarkably improves the construction load statistical efficiency and safety, and is suitable for real-time monitoring and early warning in a complex construction site environment.

The invention discloses a mechanical arm vision distance measurement method and a distance measurement system thereof, and relates to the technical field of intelligent vision perception, a camera is installed on a mechanical arm to collect images, and through an optimization matching algorithm, the images shot by the left eye and the right eye are rapidly compared to generate a clear and accurate disparity map; the motion state of the mechanical arm is detected in real time through a vibration compensation mechanism, a shot image is corrected, and it is ensured that the distance measuring process is stable and reliable; and in combination with the corrected disparity map, the accurate distance from the target object to the camera is obtained by utilizing a depth calculation formula, and the result is transmitted to a control system of the mechanical arm. The depth value is fed back to the mechanical arm controller in real time through depth calculation and linkage with a control system, the precision of dynamic path planning and action execution of the mechanical arm is improved, and the mechanical arm shows higher anti-interference capacity in a dynamic vibration environment.

The invention relates to the technical field of computer vision, in particular to a three-dimensional reconstruction method and device based on optical polarization and stereoscopic vision principles. The three-dimensional reconstruction method sequentially comprises the steps of image preprocessing, polarization parameter extraction, binocular stereo matching, disparity map generation and optimization, normal vector correction, depth information fusion and three-dimensional point cloud reconstruction and modeling generation. A polarization normal vector field is constrained and corrected as a'skeleton ', and the azimuth angle pi ambiguity problem which puzzles polarization three-dimensional reconstruction for a long time is solved; high-frequency surface normal details contained in corrected polarization information are used as textures to enhance and fill up depth information loss of binocular vision in weak texture and repeated texture areas, the inherent limitation of a single sensing technology in a three-dimensional reconstruction task is overcome, and the three-dimensional reconstruction of the surface of an object, especially a diffuse reflection object, is realized. And high-precision and high-integrity three-dimensional shape recovery is realized.

The invention provides a three-dimensional calibration method using a speckle image as a calibration object and combining weight parameters. The method comprises the following steps: acquiring a standard grid image and the speckle image through a camera; determining angular point positions in the standard grid image through an angular point detection algorithm; calibrating the stereoscopic vision system by using the obtained angular points to obtain an initial calibration result; calculating 3D coordinates of the camera calibration board in the world by using the disparity map; determining a subset of the reference image corresponding to the checkerboard angular points in the target image through a digital image cross-correlation algorithm; calculating a weight parameter of each calibration point based on the re-projection error; solving internal and external parameters of the camera by using the weight parameters, radial alignment constraint and a least square method; optimizing the main point of the camera by using the weight parameters; and if the re-projection error or the number of iterations does not meet the requirement, repeatedly calculating the weight parameter of the calibration point and the subsequent steps, otherwise, obtaining a camera calibration result, and obtaining the distortion curved surface of the camera lens.

The invention provides a super-resolution binocular image generation method and system based on geometric structure consistency, and the method comprises the steps: extracting the deep features of a low-resolution binocular image through employing a convolutional neural network or a Transform model, achieving the information interaction of a left image and a right image in combination with a cross attention module, and constructing a pixel incidence matrix of the left image and the right image; acquiring a pixel corresponding relation of the left image and the right image by using the pixel incidence matrix, and constructing a continuous parallax field; performing spatial warping on the deep features based on a continuous parallax field to obtain warping features, and aligning the deep features of the left and right images; and merging the deep features and the warping features after spatial alignment, inputting the merged features into a feature up-sampling module based on implicit two-dimensional expression, and outputting a high-resolution binocular image of the same scene. The invention provides a binocular image super-resolution technology comprising binocular image feature extraction, continuous parallax field construction based on implicit two-dimensional expression, left and right image feature space alignment and feature upsampling based on implicit two-dimensional expression.

The invention discloses an underwater fish body length measurement method, system and device based on binocular vision and a medium, and relates to the technical field of underwater measurement. The method comprises the following steps: performing fish body tracking on a collected fish school video, and determining an ID of each fish and corresponding position information; judging the straightening state based on the tracking frame corresponding to the ID of each fish, and when the judgment result is that the fish body is straightened, selecting a frame with the longest center line as a fish body straightening reference frame; carrying out ROI cutting and masking on the basis of the fish body straightening reference frame to obtain an ROI area map; inputting the left view and the right view in the ROI into a GMFlow model for parallax matching to obtain a parallax image which only contains the characteristics of the whole body of the target fish and the periphery of which forms a black frame; and calculating the length of the fish body according to the coordinates of the central points of the four corners of the fish body in the disparity map and calibration parameters of the binocular camera. According to the invention, efficient and accurate fish body measurement can be realized through step-by-step processing of the fish school image sequence.

The invention provides a road three-dimensional lane line detection method and system based on binocular vision, and relates to the technical field of automatic driving, and the method comprises the steps: carrying out the online calibration of internal and external parameters of a camera and the time sequence alignment of the binocular image and IMU attitude data through collecting the forward binocular image and IMU attitude data of a vehicle; processing the binocular image after time sequence alignment, and determining a dense disparity map; generating a road point cloud based on the dense disparity map and camera parameters, and constructing an adaptive terrain model; constructing a depth enhanced BEV feature map according to the depth feature of the road point cloud and the texture feature of the binocular image, predicting candidate parameters of a three-dimensional lane line, and screening through various constraints; and then executing extended Kalman filtering and trajectory optimization to obtain three-dimensional lane line parameters. According to the invention, rapid and accurate detection of the three-dimensional lane line can be realized, the accuracy of three-dimensional lane line detection under a complex terrain is improved, and the deployment cost is reduced.

The invention discloses a control method and device of a multi-mode intelligent manipulator, and relates to the technical field of manipulators and artificial intelligence. Capturing left and right images of the target object by using a binocular vision technology; outputting the category and boundary coordinates of the target object through a YOLO model; extracting center coordinates of a target object in the left and right images; matching objects consistent with the target object in category and similar to the target object in coordinate in the left and right images; calculating parallax and depth information between the left image and the right image to combine three-dimensional space coordinates of the target object; smoothing the historical coordinates of each target object to output a target detection result of the YOLO model; inputting the target detection result into a task decision model and converting the target detection result into a structured instruction; the grabbing force is executed, calculated and adjusted according to the structured instruction; according to the control method and device for the multi-mode intelligent manipulator, through deep learning and an intelligent control algorithm, the manipulator can autonomously recognize a target and optimize an execution path.

The application provides a three-dimensional reconstruction method based on binocular stereo matching, and relates to the technical field of binocular stereo vision. The method combines binocular stereo matching algorithm, triangulation algorithm and surface texture mapping, and can realize surface reconstruction of scene objects in different environments. After stereo calibration is performed on a binocular camera, the internal and external parameters of the binocular camera are obtained, stereo correction is performed, then an image disparity map of the corresponding scene object is generated according to an optimized semi-global stereo matching algorithm, after the disparity information is obtained, the point cloud information converted by the object disparity is triangulated on the surface according to the triangulation algorithm, and finally three-dimensional surface reconstruction of the object is realized through the texture mapping technology. The application can reduce the influence of noise in the disparity map generation process, i.e. the semi-global stereo matching algorithm, improve the accuracy of the disparity information, and complete the three-dimensional surface reconstruction task of the object with little sacrifice of time performance.

The invention relates to the field of medical instruments, and provides a cannula pose recognition method and system based on binocular vision, and the method comprises the steps: firstly, building a binocular camera system with a shooting range covering eyeballs, collecting a group of binocular images, and respectively recognizing cannulas and cannula holes in the two images; then, a disparity map is generated based on the casing pipe and the casing pipe hole in the two images, the disparity map is converted into a depth map in combination with the internal reference of the binocular camera system, and the boundary point cloud of the casing pipe hole is obtained; and finally, determining a fitting plane of the casing position and the casing hole according to the boundary point cloud, and taking a normal vector as a casing posture. According to the method, the operation complexity of aligning the instrument to the cannula in the robot-assisted surgery can be reduced, additional arrangement of a marker is not needed, the precision and speed of cannula pose recognition are effectively improved, eyeball movement is directly recognized through cannula pose changes, the compensation effect of eyeball movement is enhanced, and the safety of the robot-assisted surgery is improved.

The invention discloses a distance measuring method and device based on binocular vision and laser, and intelligent wearable equipment. The method comprises the following steps: collecting a first image and a second image of a target object at different visual angles through a binocular vision device of the intelligent wearable equipment; extracting, matching and screening feature points of a target object based on the first image and the second image to obtain key point pairs, and calculating parallax according to the key point pairs; calculating a depth value of the intelligent wearable device and the target object according to the parallax; laser ranging values of the intelligent wearable device and the target object are collected through a laser device of the intelligent wearable device; fusing the depth value and the laser ranging value by adopting a Kalman filtering algorithm, and outputting a fused ranging result; the method solves the problems that a distance measurement scheme in the prior art is complex in algorithm and high in cost.

The present invention discloses a binocular vision ranging method and system based on angular rotation consistency, belonging to the field of vision ranging. Aiming at the problems of slow processing efficiency and low accuracy in existing vision ranging, the present invention provides a binocular vision ranging method based on angular rotation consistency, comprising the following steps: collecting two images and detecting a target to be measured; extracting feature points; performing feature point matching and removing mismatches; calculating the average disparity and the distance of the target to be measured. By calculating the rotation angles of paired feature points matched in two images, the present invention uses the consistency of feature point rotation to remove mismatched feature points, without the need to calculate a complex homography matrix, nor to perform repeated iterations. Only the rotation angles between paired feature points need to be calculated once, the rotation angles are divided into intervals, and removal is performed according to the quantity within the intervals, greatly improving the speed of screening mismatched feature points and enhancing real-time performance while ensuring accuracy.

The embodiment of the invention discloses an aircraft height measurement method based on binocular vision, and the method comprises the steps: obtaining internal parameters and distortion parameters of a first camera and a second camera, and obtaining a first image and a second image collected by the first camera and the second camera; calibrating the first image and the second image according to the internal reference and the distortion parameter of the first camera and the second camera to obtain a first calibration image and a second calibration image; binocular matching is carried out on the first calibration image and the second calibration image, and a full-image disparity map is obtained based on the first calibration image; performing airport runway semantic segmentation based on the first calibration image to obtain a semantic segmentation map of the airport runway; performing 3D coordinate reduction operation on the 2D pixels according to the full-image disparity map and the semantic segmentation map to obtain a 3D point cloud of the airport runway; fitting a runway plane equation according to the 3D point cloud, converting the runway plane equation to an airframe coordinate system according to the relative position of the first camera and the airframe, and obtaining an aircraft height measurement result according to the runway plane equation under the airframe coordinate system.

The invention relates to the field of binocular stereo vision, and discloses a binocular stereo matching method and system based on multi-scale iterative optimization and related equipment. The method comprises the steps of performing semantic structure feature extraction on a corrected left view and a corrected right view through a feature extraction network; determining a cost space pyramid and a probability matrix according to the left view feature map and the right view feature map output by the feature extraction network; extracting a multi-scale context feature of the left view through a context sensing network, and initializing the multi-scale context feature to obtain a hidden state and input of a recursive network; during recursive network iteration, according to the probability matrix, a multi-range search strategy is adopted to index local cost from the cost space pyramid; and inputting the local cost into the recursive network, combining the hidden state obtained by the context features and the input, and carrying out iterative updating on the parallax value to obtain a parallax map, thereby improving the accuracy of parallax calculation.

The invention discloses an indexable blade cutting edge passivation radius measuring method based on binocular vision, and belongs to the technical field of binocular vision measurement. The method comprises the following steps: calibrating left and right cameras by using a binocular vision system to obtain internal and external parameters and distortion coefficients of the cameras; secondly, acquiring an indexable blade cutting edge passivation image for image correction, so that the corrected images are in the same plane and are parallel to each other; secondly, improving the blade image quality through a wavelet packet transformation image preprocessing method, and reducing the noise influencing the image quality; thirdly, obtaining an image disparity map through a region-based optimization SGBM stereo matching algorithm, and changing the disparity map into a depth map according to a triangulation principle; and finally, measuring the passivation radius of the cutting edge of the blade on the basis of the obtained depth map, and comparing the passivation radius of the cutting edge of the blade with the passivation radius of the cutting edge of the blade measured by an Alcona three-dimensional detection instrument so as to judge the precision of the research.

The invention discloses a two-dimensional detection and SGBM three-dimensional ranging method based on improved YOLOv13, belongs to the field of computer vision, and particularly relates to the two-dimensional detection and SGBM three-dimensional ranging method based on the improved YOLOv13. The objective of the invention is to solve the problem of low cross-view-angle distance measurement precision in a three-dimensional space perception level of the existing method. The method comprises the following steps of: obtaining a trained YOLOv13-SHSA network model; the binocular camera obtains a left view and a right view; inputting the left view into the trained network model, and outputting three detection results by three detection heads of the trained network model; processing the three detection results to obtain detection frames and confidence coefficients of a plurality of targets in the left view; obtaining a disparity map of the left and right views; calculating a depth value corresponding to each pixel point in the disparity map according to the disparity map and the binocular camera parameters; and calculating the distance of the target object based on the depth map, the detection frames of the multiple targets in the left view and the confidence coefficient.

The invention discloses a binocular fisheye stereo matching method constructed by fusing monocular depth prior and multi-scale spherical projection cost. The binocular fisheye stereo matching method comprises the following specific steps: S1, image input and feature extraction; s2, monocular depth priori estimation; s3, constructing a cost matrix of multi-scale spherical projection scanning; s4, the deep attention network is matched; s5, performing initial parallax estimation; s6, obtaining a final fine disparity map based on a ConvGRU disparity optimization network, and the method can accurately and efficiently carry out stereo matching without image correction, and improves the matching precision and enhances the robustness of an algorithm to a distorted image by introducing monocular depth estimation prior and a multi-scale spherical projection scanning strategy; the method also avoids the quality loss caused by image correction, retains the original view field information of the fisheye image, gives consideration to the precision and efficiency through a multi-scale spherical scanning strategy, adapts to fisheye distortion characteristics, introduces an attention mechanism and a GRU structure, and improves the expression capability of a parallax estimation network.

The invention belongs to the field of underwater operation, and particularly discloses a binocular vision servo method for autonomous operation of an underwater vehicle-double-arm manipulator, and the method comprises the steps: obtaining a two-dimensional image and a depth image in front of the underwater vehicle-double-arm manipulator; inputting the two-dimensional image into a lightweight target detector to obtain a target recognition frame; designing a target three-dimensional coordinate observation value acquisition method based on the shape of an identification frame so as to reduce the influence of a parallax hole; solving an equipment motion matrix based on the three-dimensional coordinates of the key points in the images at the adjacent moments; a nonlinear Kalman filtering method based on a motion matrix is designed by taking the three-dimensional coordinates of the target as state vectors, and continuous estimation of the target position is realized; a target three-dimensional coordinate is used as a reference, an aircraft-manipulator is driven to move, and a two-dimensional image is used for driving a tail end joint of the manipulator to rotate, so that an actuator of the manipulator can effectively clamp the target after the manipulator reaches the target. According to the invention, the autonomy and intelligence of underwater operation can be effectively improved.