585 results about "Stereo matching" patented technology

Disclosed in the present invention is a salient contour matching-based method for target measurement in a severe imaging environment. The method specifically comprises: (1) acquiring a binocular image of a target; (2) establishing a global-local joint constraint-based background light estimation model, and removing a scattering effect of a medium in an imaging environment to obtain a restored left eye image and a restored right eye image; (3) learning an original image, and on the basis of a residual between a network reconstructed image and the original image, obtaining target localization prediction maps of the left eye image and the right eye image; and (4) respectively extracting contour lines of the target in the left eye image and the right eye image, constructing feature matching descriptors of contour points, performing stereo matching on the two sets of contour lines by minimizing matching cost, and performing three-dimensional reconstruction on the contour lines in light of calibrated intrinsic and extrinsic parameters to complete the measurement of a key size. According to the present invention, the key sizes of different targets in a severe environment can be accurately measured, thereby providing an effective solution for the problem of measuring the sizes of targets in a severe environment.

The invention discloses a distribution network tree barrier real-time analysis method and system based on dynamic vision and SLAM. The method comprises the following steps: generating a dynamic visual baseline by cooperatively controlling the translation and flight displacement of an unmanned aerial vehicle holder, and constructing a bionic binocular model to simulate a time sequence image into a binocular image pair; generating a depth point cloud through epipolar correction and stereo matching; key targets are recognized and extracted through a semantic segmentation network, and semantic point clouds are generated; establishing a dimensionality reduction motion model by utilizing pan-tilt stability augmentation, and fusing a visual inertial odometer and RTK data by adopting a filtering or optimization algorithm to realize centimeter-level pose estimation; and finally, performing optimization processing on the semantic point cloud, completing three-dimensional reconstruction based on multi-modal fusion, and outputting a risk assessment result through tree line spacing calculation and safety margin analysis. According to the invention, accurate, efficient and automatic routing inspection and risk early warning of the distribution network tree obstacles are realized.

The invention provides an adaptive welding seam detection and three-dimensional reconstruction method based on deep learning and binocular vision. The adaptive welding seam detection and three-dimensional reconstruction method comprises the steps of S1, collecting samples and making a training data set; s2, the picture of the sample to be welded is processed, a feature region is recognized, the image quality of the region to be welded is analyzed and evaluated through wavelet transform and local variance, and the noise level and the contrast ratio are calculated; s3, dynamically generating edge detection parameters and model fitting parameters according to the image quality; s4, using an edge detection algorithm to extract edge point cloud of the welding seam area; s5, performing RANSAC linear fitting, weighted least square fitting and polynomial curve fitting on the edge point cloud in parallel; s6, selecting an optimal fitting result based on an image quality adaptive dynamic scoring model; and S7, carrying out three-dimensional coordinate conversion in combination with the three-dimensional matching model IGEV-Stereo, and outputting a final welding seam three-dimensional coordinate. According to the invention, automatic detection of the position and size of the welding seam can be efficiently and accurately realized.

A slope monitoring method based on optical flow estimation and binocular vision belongs to the technical field of deformation monitoring and measurement, and adopts the technical scheme that a binocular stereoscopic vision system is built, and camera acquisition parameters are calibrated; transmitting the image to a server in real time through the Internet of Things; distortion correction and stereo matching are carried out to obtain a depth map; monitoring points are manually or automatically arranged on the corrected image, and dense optical flow estimation tracking pixel point movement is carried out; world coordinates of observation points and movement conditions of each frame are obtained, and displacement changes of slope monitoring points are obtained by combining coordinates before and after displacement; and obtaining a displacement rate by combining monitoring time and displacement, judging slope safety, obtaining a displacement track, and performing linear interpolation to obtain a slope deformation cloud picture, thereby realizing the monitoring purpose. The slope displacement monitoring method has the advantages that slope monitoring can be carried out more flexibly and efficiently, the safety of workers is improved, the slope monitoring cost is reduced, real-time slope displacement monitoring is achieved, and the intensive degree of slope monitoring points is greatly improved.

The invention discloses a three-dimensional vision calibration and positioning method and system based on an event camera, and belongs to the field of positioning. The method comprises the steps of constructing an array containing a plurality of unique frequency coding light sources, constructing a binocular event camera system, collecting light source array light signal event streams, extracting pixel positions, frequencies and timestamp information, determining pixel coordinate centroids of the light sources through frequency and pixel clustering analysis, and establishing frequency and centroid mapping. Auxiliary light sources with the same coding rule are arranged on the surface of a target object, and after a binocular system collects light signals of the auxiliary light sources, the center-of-mass coordinates of the light sources are matched according to the mapping relation. And based on event camera calibration parameters and binocular external parameters, correcting and stereoscopically matching centroid coordinates, and calculating three-dimensional coordinates of the auxiliary light source through triangulation. And in combination with the known geometric topological relation of the auxiliary light source on the surface of the object, the spatial pose of the target object is solved by using a rigid transformation algorithm, and high-precision positioning is completed. According to the embodiment, the calibration precision can be improved in a complex application environment.

The invention particularly relates to a binocular vision-based three-dimensional reconstruction method, which comprises the following steps of: calibrating a binocular camera based on an improved Zhang Zhengyou calibration method to obtain internal and external parameters and a distortion coefficient of the camera; performing stereo correction on the image by using the internal and external parameters of the camera and the distortion coefficient obtained by calibration, so that the binocular image meets an epipolar constraint condition; a multi-strategy optimized semi-global stereo matching algorithm is adopted to process the image after stereo correction, and a disparity map is generated; based on the generated disparity map, generating a three-dimensional point cloud through a triangulation principle; carrying out anti-interference processing and registration optimization on the three-dimensional point cloud; and performing global splicing on the three-dimensional point clouds subjected to anti-interference processing and registration optimization based on a sequential registration error sharing strategy to complete three-dimensional reconstruction. According to the method, the key problems of large calibration error, weak texture matching failure, point cloud noise sensitivity and registration accumulative error in a traditional method are solved, and the reconstruction precision and stability are remarkably improved.

The invention discloses a monocular and binocular cooperative localization and mapping method and device for underwater refraction compensation, and the method comprises the steps: extracting checkerboard angular points, compensating an underwater light path through employing a two-time vector correction model based on a Snell law, and precisely calibrating the internal and external parameters of a binocular camera. According to the method, an underwater real light path is effectively restored, and calibration errors caused by medium refraction are remarkably reduced. Then epipolar correction is performed on the corrected left and right images, so that the geometric consistency of stereo matching is ensured; and finally, monocular and binocular poses are fused, and seven-degree-of-freedom similarity transformation global alignment is applied, so that the problem of monocular scale uncertainty is effectively solved, and high-precision and high-robustness three-dimensional mapping is realized by means of binocular real scale introduction. Key links such as camera pre-calibration, refraction compensation, image correction, monocular tracking, binocular depth measurement, scale alignment and the like are connected in series to form an integrated process, so that the underwater vision positioning and mapping precision is improved.

The invention discloses a multi-view underwater three-dimensional point cloud splicing method based on sparse and dense point cloud fusion, and relates to the technical field of structure underwater detection. The method comprises the steps that a binocular camera surface image of an underwater structure is received, the binocular camera comprises a left camera and a right camera, dense point clouds are generated based on stereo matching and triangulation, and the corresponding relation between the dense point clouds and image pixels of the left camera is output; and based on the received left camera image, a sparse point cloud is generated by adopting a motion structure recovery algorithm and utilizing multi-view triangulation, and a corresponding relation between the sparse point cloud and the pixels of the left camera image is output. According to the method, when the point cloud of multi-view reconstruction is spliced, the surface features of the underwater structure can be well restored, feature extraction is not needed, the method has high accuracy and robustness, and high-precision point cloud splicing can be achieved in the underwater complex environment, so that the method has good application prospects in actual underwater structure detection and image splicing tasks.

The invention relates to the technical field of image data processing and intelligent control, and discloses a patch board spot welding track intelligent control method and system based on visual perception. According to the method, synchronous image streams are collected through a binocular vision sensor, and epipolar correction image pairs are generated through timestamp synchronization, ROI extraction and epipolar correction processing; calculating a disparity map by adopting a stereo matching algorithm, and constructing a compensated three-dimensional point cloud model based on deformation vector field fusion multi-frame point cloud data of a radial basis kernel function; welding spot position error vectors are generated by extracting welding spot feature points and performing spatial filtering optimization; and carrying out inverse kinematics calculation on the mechanical arm by adopting a damping least square method, carrying out safety constraint optimization in combination with prospective collision risk assessment and real-time pose data, and generating a trajectory compensation instruction. According to the method, the problems of dynamic deformation compensation and motion safety in patch plate welding are solved, the submillimeter welding spot positioning precision is achieved, and the welding quality stability and the system robustness are improved.

The invention provides a robust stereo matching method fusing monocular semantic prior and multi-dimensional expert aggregation, relates to the technical field of computer vision and stereo matching, establishes a robust stereo matching framework fusing monocular semantic prior and multi-dimensional expert aggregation, and comprises a monocular branch and a binocular branch, inputting a stereo image into a robust stereo matching framework based on fusion of monocular semantic prior and multi-dimensional expert aggregation for matching calculation, wherein the matching calculation comprises the following steps: extracting semantic prior features of the input image by using a monocular branch; the method comprises the following steps: extracting a geometric enhancement feature map of a stereo image by using binocular branches, generating a matching cost body based on the geometric enhancement feature map, and carrying out refined iterative updating on the matching cost body to obtain a matched disparity map. According to the method, monocular and binocular depth estimation is cooperatively supported in a unified network architecture, and a parallel monocular depth prior path is utilized to actively guide and strengthen a core binocular matching process.

The invention discloses a tunnel surrounding rock multi-source data dynamic fine rapid sensing method, which comprises the following steps: based on a mobile terminal multi-view image sequence, arranging shooting positions according to a preset space interval and an orthogonal angle, generating a sparse point cloud through a motion recovery structure algorithm, and generating a dense point cloud model in combination with multi-view stereo matching optimization; and for the dense point cloud model, delimiting a local neighborhood based on k-nearest neighbor search, resolving a neighborhood point covariance matrix through principal component analysis, extracting a feature vector corresponding to a minimum feature value as a normal vector, and constructing a rock mass surface microscopic geometric feature field. The invention provides a dynamic fine rapid sensing method based on multi-source data, and aims to improve the efficiency, precision and timeliness of tunnel surrounding rock information acquisition and provide accurate surrounding rock information support for tunnel construction and support design.

The invention relates to the field of power transmission line channel risk hidden danger monitoring, and provides a data fusion power transmission line channel risk hidden danger monitoring method and system, and the method comprises the steps: collecting the multi-modal sensing data of a power transmission line channel, and generating a multi-modal data flow of a unified time-space coordinate; constructing a three-dimensional space point cloud through a phase unwrapping and stereo matching fusion algorithm, and fusing multi-modal data to generate a space probability tensor; extracting risk semantic latent variables, constructing a Bayesian network and identifying potential risks; performing tensor product on the potential risk and the environmental data to generate a dynamic risk enhancement feature matrix, and constructing a nonlinear dynamic threshold curved surface through quantum annealing and Gaussian process regression; a mechanical equation is constructed, Gaussian kernel density estimation and numerical simulation are combined, the evolution trajectory of the risk in the space-time dimension is predicted, and a risk thermodynamic diagram and early warning information are generated; and generating a structured risk early warning report by adopting a natural language processing method. And the accuracy of power transmission line channel risk hidden danger monitoring is improved.

The invention discloses a photoelectric fusion beam control method and device based on a reconfigurable intelligent reflecting surface. The method comprises the following steps: acquiring internal and external parameter matrixes of a binocular camera and a binocular-RIS coordinate transformation matrix; capturing a binocular image stream of a detection target by using a binocular camera and executing stereo matching to obtain a disparity map and depth data of a binocular image; detecting a target image in the binocular image and outputting bounding box information of a center point of the target image; calculating target image coordinates through the depth data and the bounding box information, and converting the target image coordinates into three-dimensional space coordinates under an RIS coordinate system so as to determine the spatial position of the target; generating an RIS phase codebook according to the target spatial position, wherein the RIS phase codebook comprises a near-field phase compensation item; and loading the phase codebook to the RIS unit to form a target beam so as to perform directional tracking on a target. According to the technical scheme of the invention, high-precision beam control and stable positioning and tracking of the target can be realized in a complex multi-target environment.

The present application provides an iris recognition method and apparatus, an electronic device, and a storage medium. The method comprises: obtaining an iris image group acquired for an object to be recognized, wherein the iris image group comprises at least two iris images acquired at different acquisition angles for a same eye region of said object; performing stereo matching on the at least two iris images to obtain a disparity map between the at least two iris images, wherein the disparity map comprises disparity elements, and each disparity element represents a displacement amount in a specified direction between two pixel points corresponding to a same eye region element in the at least two iris images; in one iris image among the at least two iris images, determining a pupil edge on the basis of the displacement amounts; determining an iris region in the iris image on the basis of the pupil edge, and performing feature extraction on the iris region to obtain an iris feature; and performing identity recognition on said object on the basis of the iris feature.

The invention relates to the technical field of three-dimensional reconstruction, and provides a local feature adaptive fusion and high-fidelity new view angle synthesis method based on three-dimensional reconstruction. According to the method, a local feature module is introduced into a point cloud network, robust matching of local features is realized, a self-adaptive matching strategy is combined, high-resolution images can be efficiently processed, and the precision and stability of three-dimensional reconstruction are remarkably improved. The method is suitable for application scenes such as three-dimensional rendering, three-dimensional reconstruction and new view angle synthesis. The specific process comprises dense reconstruction and initial rendering; and carrying out missing region repairing and iterative optimization. Generating a high-precision point cloud based on an improved stereo matching algorithm, and rendering an initial image of a target view angle in combination with camera parameters; a conditional video generation model is adopted, and holes, distortion and artifacts in initial rendering are repaired; and carrying out progressive complementation on the occlusion region by using the view angle consistency constraint, and improving the reconstruction integrity by jointly optimizing the point cloud geometry and texture. According to the method, the final picture quality is remarkably improved through feature matching optimization and efficient processing capacity. Local feature branches and global context information are coordinated, and the problem of matching ambiguity in a complex scene is solved; the adaptive fusion strategy gives consideration to high-resolution image processing efficiency and detail retention; and the detail fidelity and the overall quality of the rendering result are effectively improved.

The invention relates to the technical field of structure health monitoring, and provides a structure unmarked three-way displacement real-time measurement method based on deep learning stereoscopic vision, which comprises the following steps: erecting and calibrating a binocular camera; obtaining a structure monitoring image sequence, and automatically extracting a to-be-monitored region of the structure through the semantic segmentation model; tracking coordinates of feature points of the to-be-monitored region of the structure through a deep learning feature point detection model; obtaining feature point coordinates of sub-pixel precision through sub-pixel refinement; three-dimensional coordinates of the feature points at all moments are obtained through stereo matching and three-dimensional reconstruction; and obtaining three-direction displacement information of the structure according to the three-dimensional coordinates and the three-dimensional point displacement. According to the invention, three-dimensional displacement measurement of the structure can be carried out without manually selecting a to-be-monitored area of the structure and designing a mark, the cost is low, the precision is high, and the engineering practicability is high.

The invention belongs to the technical field of data measurement, and discloses a high-speed railway ballastless track construction measurement method, which comprises the following steps of: carrying out internal reference and external reference calibration on a binocular camera of a track detection trolley; obtaining related data containing a binocular image, and carrying out space-time alignment processing on the related data so as to construct a prediction state vector; carrying out stereo matching on the binocular image to generate a prediction disparity map, and carrying out three-dimensional reconstruction on the prediction disparity map to obtain a track point cloud map; and extracting a track inner side point coordinate sequence and a track height program sequence from the track point cloud picture, and generating a corresponding track gauge table and a corresponding smoothness table. According to the invention, vision, three-dimensional reconstruction and inertial navigation technologies are integrated, and integrated detection of gauge, elevation and smoothness parameters is realized.

The invention discloses a pavement disease size accurate quantification method based on binocular vision, and the method specifically comprises the steps: collecting pavement image data through a vehicle-mounted binocular camera, achieving the real-time detection of a disease target through an improved RT-DETR model, and outputting the disease type and detection frame information; after a pavement disease target is detected, pixel-level segmentation is carried out on a disease area in the detection frame based on a semantic segmentation model, and a disease contour mask is extracted; an improved IGEV-Stereo stereo matching algorithm is used for calculating a disparity map, depth information is output based on camera calibration parameters and a disparity estimation result, and conversion from pixel coordinates to three-dimensional coordinates is achieved; and the binocular depth information and a disease detection segmentation result are combined to realize accurate size quantification of typical road surface diseases with different characters. Through the binocular stereoscopic vision technology, automatic detection and precise quantification of pavement diseases can be realized, the cost can be effectively reduced while the disease quantitative evaluation precision is improved, and decision support is provided for road management and maintenance.

The invention discloses a remote sensing image stereo matching method and system based on a Mama model interpretation cost body, and the method comprises the steps: S1, extracting multi-level and multi-scale semantic features from input left and right remote sensing stereo image pairs, and outputting a stereo image pair feature map; s2, aligning the left and right stereo image pair feature maps on the parallax dimension to construct a three-dimensional cost body; s3, introducing a Mama framework based on a selective state space model, interpreting the cost body in combination with a multi-scale feature interaction mechanism, and generating a parallax probability distribution diagram; s4, mapping the probability distribution into a continuous disparity value, and outputting a disparity map; according to the method, adaptive feature screening and global modeling are realized through a dynamic weight selection mechanism of the state space model, the local receptive field limitation of 3D convolution is overcome, and the multi-granularity feature representation capability of the model is improved by promoting interaction and fusion among features of different scales by virtue of a cross-scale information interaction mechanism, so that the robustness of the model is improved. And the overall interpretation accuracy and efficiency are improved.

The invention discloses a cable defect detection method of a binocular intelligent inspection robot, and relates to the field of image processing, and the method comprises the steps: firstly obtaining the state information of a cable, then carrying out the preprocessing and stereo matching of the image information of the cable, and then constructing a target detection and segmentation model; the target detection and segmentation model is adopted to detect and segment cable defects, and model sample data is updated in real time in an incremental learning mode; fusing the cable state information acquired by the infrared camera, the thermal imager and the wireless sensor network by using a multi-sensor data information fusion model; the cable is maintained and managed based on a detection result; according to the invention, the detection precision and accuracy can be improved, cables of different types and specifications can be handled, the adaptability and stability are ensured, and the real-time performance and accuracy of detection can be improved.

The invention relates to the technical field of image processing, and discloses a woolen sweater intelligent production monitoring method and system based on the Internet of Things. The method comprises the following steps: acquiring a wool fiber three-dimensional point cloud through multispectral imaging and a space stereo matching algorithm, and analyzing the twist and crimpness of yarns through a gradient convolutional network to obtain a quality feature set; a high-speed acquisition system is adopted to obtain a knitting needle motion sequence, and a fabric structure map is obtained through a line enhancement algorithm. And constructing knit fault prediction based on the spatial-temporal characteristic network and the memory model, and obtaining a density uniformity index. And inputting the quality feature set and the density index into a fusion network, and performing multi-layer attention processing to obtain a quality evaluation model. And constructing an early warning system by using the depth map network, and generating a process optimization scheme. According to the method, comprehensive perception, dynamic early warning and intelligent optimization of the production process are realized, and the problems of incomplete data acquisition, inflexible parameter control, untimely quality early warning and the like in the prior art are solved.

The invention discloses a badminton tracking method and system based on three-dimensional vision, and relates to computer vision. Badminton motion video data of a left view and a right view are synchronously collected through a binocular camera, a left view frame sequence and a right view frame sequence which are aligned in time are obtained, and the collected video data are preprocessed; obtaining standardized left and right view input frame data; according to input frame data, constructing a 2D key frame detection model based on an improved YOLOv8 network, and extracting the sum of 2D key frame coordinates of left and right views; according to the 2D key frame coordinate sum of the left view and the right view, 3D space positioning is carried out on the badminton through stereo matching and track prediction, 3D track data are obtained, according to the 3D track data, a broken track is repaired through a frame leakage compensation mechanism, and a final 3D track is obtained. In order to solve the problem that in the prior art, key frames are prone to being lost and track breakage is caused during high-speed movement such as smash, the breakage track of the shuttlecock is repaired.

The invention relates to the technical field of three-dimensional reconstruction, and discloses a mark point region priority processing-based three-dimensional scanner rapid reconstruction method, which comprises the following steps of: configuring a plurality of mark points in a region range of a target to-be-measured object, and collecting left and right images of the target to-be-measured object; identifying mark points in the left image and the right image, and extracting two-dimensional image coordinates of each mark point; performing region division on the left and right images based on the center point; performing feature extraction and stereo matching processing on a target image corresponding to the region of interest; calculating a local three-dimensional point cloud of each mark point in the region of interest according to the local disparity map, and analyzing camera poses under a plurality of target view angles according to the two-dimensional image coordinates and the local three-dimensional point clouds; converting the local three-dimensional point cloud to a preset world coordinate system according to the camera pose; and reconstructing a three-dimensional model of the target object to be measured according to the three-dimensional point cloud data under the plurality of target viewing angles. According to the invention, the reconstruction efficiency of the three-dimensional scanner can be improved.

Photometric stereo techniques enable using a single camera to perform an enrollment process for creating a user-specific anatomical model of an eye for gaze tracking. The user-specific anatomical model includes information about a user's center of vision at multiple dilation states of the eye, which can be used to enhance the accuracy of gaze tracking techniques. Accurate gaze tracking techniques enable the use of gaze tracking at close range, for example, gaze tracking within a head-mounted display device.

The invention relates to the technical field of computer vision, and particularly provides a tunnel video stream three-dimensional modeling system based on binocular stereo matching and SLAM (Simultaneous Localization and Mapping), which comprises a multi-modal image acquisition unit, a binocular infrared camera and an RGB (Red, Green and Blue) camera are configured to synchronously acquire an infrared image pair, an RGB image pair and inertial measurement unit data of a tunnel environment, the multi-source data is uploaded to the cloud processing platform in real time through the wireless transmission module; based on a dynamic switching mechanism of environmental perception, performing adaptive fusion pose estimation of a feature point method and a direct method on input multi-modal image data; the stereo matching and dense reconstruction unit is used for calculating a disparity map by adopting an improved self-adaptive window stereo matching algorithm, generating a three-dimensional point cloud in combination with the pose information output by the pose estimation unit, and constructing a global consistency point cloud model of the tunnel scene through a time sequence point cloud registration and splicing algorithm; and a multi-algorithm target detection and semantic fusion unit. The method can meet the requirements of precision and robustness of tunnel reconstruction.

The invention relates to the field of image processing, in particular to a visual image processing method and device, equipment and a program product. The method comprises the following steps: acquiring a binocular image, and generating a first disparity map through a stereo matching network; obtaining a second disparity map through extreme value filtering; generating an edge mask through edge detection; performing edge filtering on the second disparity map according to the edge mask to obtain a third disparity map; and horizontal displacement is obtained according to a parallax value in the third parallax image, parameters are calibrated through a camera, the horizontal displacement is converted into depth information, and a three-dimensional point cloud is generated according to the positions of the pixel points and the depth information. According to the method, a real boundary is extracted through edge filtering, parallax of a boundary area is forcibly corrected by using an edge mask, a transition zone of network prediction is suppressed, burrs or outliers of point clouds after conversion can be effectively reduced, scattered point cloud noise is eliminated, the object contour boundary is clearer, and the perception precision of a visual system is improved.

The invention discloses a light-weight unmanned forklift AI visual anti-collision method based on a domestic embedded platform, and the method comprises the following steps: S1, carrying out the preprocessing of an RGB image through a light-weight pedestrian detection module, carrying out the automatic pruning and compression of a deep convolutional network based on an image recognition algorithm of a light-weight convolutional neural network, and carrying out the recognition of the deep convolutional network; lightweight pedestrian detection features are extracted; s2, using a lightweight pedestrian distance estimation module to extract lightweight pedestrian distance estimation features by model compression through camera calibration, image correction, stereo matching and distance acquisition; s3, performing feature fusion, performing feature analysis in combination with a dynamic adaptive sparse transformation network, and completing multi-pedestrian detection and pedestrian distance estimation by introducing a sparse feature adaptive reconstruction mechanism; and S4, deployment is carried out on a domestic embedded platform, and software function module cutting is carried out. According to the invention, a lightweight deep learning algorithm and adaptive feature fusion are adopted, and multi-pedestrian anti-collision detection on a domestic embedded platform is realized.

The invention relates to the technical field of remote sensing image processing, in particular to a stereo matching method of a high-resolution stereo satellite panchromatic image pair. By comprehensively applying a combined geometric coding convolution and gating iterative optimization mechanism, the capturing capability of the model on a complex geometrical shape is remarkably enhanced; the multi-scale cost volume is aggregated, so that the model can be effectively matched with a ground object on different scales, and meanwhile, shielding and shadow areas are effectively processed, so that the matching accuracy and robustness are improved; the parallax estimation process is further optimized by introducing a gradient consistency constraint loss function, clear distinguishing of the parallax image in an edge region and smooth transition of a weak texture region are ensured, and therefore the overall quality of the parallax image is improved. The objective of the invention is to solve the problem of how to improve the capability of capturing a complex geometrical shape in remote sensing image stereo matching and reduce the dependence on the precision of a data set at the same time.

The invention provides a multi-modal image fusion endoscope operation target positioning device and method, and the method comprises the steps: calibrating internal and external parameters of a binocular camera and a fluorescence camera, and obtaining a pose conversion matrix between the fluorescence camera and the binocular camera; using a U-Net network model to identify and track the position of the fluorescent mark point in the fluorescent image in real time, and obtaining the two-dimensional image coordinate of the fluorescent mark point; processing a left image and a right image of the binocular camera by using a region-based local stereo matching method in binocular stereo imaging to obtain a point cloud image of a target space; and obtaining three-dimensional pose information of the fluorescent mark point by combining the two-dimensional image coordinate of the fluorescent mark point and the point cloud image of the target space. The binocular stereoscopic vision and the fluorescence imaging technology are combined, the three-dimensional pose and depth information of the fluorescence mark point is obtained by registering the information between the fluorescence camera and the binocular camera, more accurate and real-time vision assistance is provided for an operation, and the safety and efficiency of the operation are improved.

The invention provides an intelligent measuring and calculating method for the high-density fish body length of a culture pond in the technical field of fish body length measuring and calculating. The method comprises the steps that S1, a large number of historical fish school images are acquired to train a fish detection model; s2, setting a fish form constraint parameter, calibrating a camera parameter, collecting a culture pond video through a binocular camera, and analyzing the culture pond video to obtain a left eye image and a right eye image; s3, inputting the left eye image and the right eye image into a fish detection model to obtain a fish body bounding box carrying confidence, and filtering the fish body bounding box through the fingerling form constraint parameters and the confidence; s4, performing stereo matching on the fish body bounding boxes based on the left eye image and the right eye image so as to calculate the depth value of the fish body corresponding to each fish body bounding box; and S5, obtaining the fish body length based on the fish body bounding box through the camera parameters and the depth value. The method has the advantages that the accuracy, efficiency and reliability of measuring and calculating the length of the fish body are greatly improved.