714 results about "Stereo image" patented technology

The invention provides a navigation instruction generation method, device and system based on multi-modal environment understanding, and the method comprises the steps: obtaining multi-modal perception information comprising the three-dimensional image data and three-dimensional point cloud data of an environment where a current unmanned aerial vehicle is located, and the task text information of the unmanned aerial vehicle; and performing semantic fusion extraction on the multi-modal perception information and the task text information based on a visual language fusion model to obtain a multi-modal embedded representation, constructing a large language model Prompt based on a plurality of prior navigation templates and the multi-modal embedded representation, and inputting the large language model Prompt into the large language model. According to the method, the navigation instruction text output by the large language model is obtained, then the navigation instruction text is analyzed into the control instruction sequence which can be recognized and executed by the unmanned aerial vehicle, the control instruction sequence is issued to the unmanned aerial vehicle, and the accuracy, continuity and stability of navigation can be maintained in a complex, dynamic and GNSS limited environment.

The invention relates to the technical field of XR display screens, and discloses a high-resolution rendering method and system for an XR display screen, and the method comprises the steps: carrying out the distributed sampling of the optical characteristics of the XR display screen, obtaining a four-dimensional rendering sampling matrix, executing the Latin hypercube sampling, and obtaining a plurality of rendering parameter combinations and corresponding high-resolution rendering image data; performing spatial geometric correction processing on the rendering parameter combination and the high-resolution rendering image data to obtain spatial geometric correction parameters; performing local enhancement processing on the fixation point coordinate sequence and the space geometric correction parameter to obtain local resolution enhancement data; and calculating rendering channel parameters of a plurality of display channels according to the spatial geometric correction parameters and the local resolution enhancement data, and executing multi-channel synchronous control to obtain target stereo image display parameters, thereby improving the rendering quality and the calculation efficiency of XR display, and reducing the resource consumption of the system at the same time.

The invention provides a control system and method for an oral surgery robot, and relates to the field of intelligent control, and the method comprises the steps: firstly obtaining thin-layer CT scanning data covering neck and throat regions, constructing a throat three-dimensional model, then collecting a surgery view three-dimensional image through a 3D endoscope, and carrying out the 3D image collection; performing semantic-level dynamic registration based on visual features on the three-dimensional model of the throat, determining pose information of the oral surgical robot, outputting a next target pose point of the oral surgical robot in combination with the pose information and a pre-operative planned surgical resection path, and finally determining the target pose point of the oral surgical robot on the basis of the target pose point. And generating a joint control instruction of the oral surgical robot. Therefore, the navigation preparation time in the operation is shortened, the high consistency of the motion track of the mechanical arm and the focus resection path is ensured, and the track deviation risk caused by registration deviation of a traditional system can be fundamentally solved.

A non-transitory machine-readable media may store instructions that, when run by one or more processors, cause the one or more processors to generate stereo endoscopic image data of a medical field and define a medical field volume from the stereo endoscopic image data. The processors may also project a 3D representation of the medical field volume to an extended-reality display device in a remote volume. The 3D representation may include a stereoscopic image and a 3D scene generated from the stereo endoscopic image data. The processors may also generate visual guidance in the remote volume to augment the stereo endoscopic image, map the visual guidance from the remote volume to the medical field volume to generate an augmented image of the medical field volume, and project the augmented image of the medical field volume to a display device viewed by an operator of instruments in the medical field.

The invention provides a robust stereo matching method fusing monocular semantic prior and multi-dimensional expert aggregation, relates to the technical field of computer vision and stereo matching, establishes a robust stereo matching framework fusing monocular semantic prior and multi-dimensional expert aggregation, and comprises a monocular branch and a binocular branch, inputting a stereo image into a robust stereo matching framework based on fusion of monocular semantic prior and multi-dimensional expert aggregation for matching calculation, wherein the matching calculation comprises the following steps: extracting semantic prior features of the input image by using a monocular branch; the method comprises the following steps: extracting a geometric enhancement feature map of a stereo image by using binocular branches, generating a matching cost body based on the geometric enhancement feature map, and carrying out refined iterative updating on the matching cost body to obtain a matched disparity map. According to the method, monocular and binocular depth estimation is cooperatively supported in a unified network architecture, and a parallel monocular depth prior path is utilized to actively guide and strengthen a core binocular matching process.

The invention relates to an urban live-action three-dimensional construction method based on a high-resolution satellite image. The method comprises the following steps: carrying out stereo image pair screening and stereo image pair preprocessing on a high-resolution multi-view satellite image; carrying out adjustment optimization based on a rational function model on the satellite stereo image pair; generating an epipolar ray image based on the optimized rational function model; performing dense matching on the epipolar line image by combining building edge straight line segment constraint and an MGM algorithm to generate a disparity map; generating a three-dimensional point cloud and a three-dimensional model formed by a plurality of stereo image pairs through forward intersection based on the disparity map and the optimization parameters; generating a digital elevation model, a digital surface model and a three-dimensional Mesh model based on the three-dimensional point cloud; and carrying out precision evaluation and integrity evaluation by adopting pixel proportions of plane precision, elevation precision and height difference. According to the method, the urban live-action three-dimensional model and the corresponding digital product can be constructed quickly, efficiently and accurately by using the high-resolution satellite image.

An advanced driver assist system (ADAS) includes a processing circuit and a memory storing instructions executable by the processing circuit. The processing circuit executes the instructions to cause the ADAS to: obtain, from a vehicle, a video sequence including a plurality of frames captured while driving the vehicle, where each of the frames corresponds to a stereo image including a first viewpoint image and a second viewpoint image; determine depth information in the stereo image based on reflected signals received while driving the vehicle; fuse the stereo image and the depth information to generated fused information, and detect at least one object included in the stereo image based on the fused information.

The invention discloses an intelligent blind guiding glasses system based on a multi-mode large language model. The intelligent blind guiding glasses system comprises an image acquisition module, a voice acquisition module, a large language model module, a bone conduction sound transmission module, a navigation path planning module, a central processor module and a wireless communication module, the image acquisition module is used for acquiring a three-dimensional image of a front scene and providing three-dimensional space information; the voice acquisition module is used for acquiring a voice instruction of a user; the big language model module is used for generating character answers and converting the character answers into voice according to the voice instruction of the user by utilizing a big language model; the bone conduction sound transmission module is used for receiving and playing the voice generated by the large language model module; the navigation path planning module is used for planning a navigation path according to the user position and the destination; the functions of navigation positioning, line guiding, voice prompting and the like are provided for the blind, the glasses are adopted as a carrier, carrying is convenient, voice awakening and other instructions are adopted to replace keys, operation is easy, and practicability is high.

Embodiments of the present disclosure relate to surface estimation using stereo imaging and surface disparities. For example, a surface disparity field representing a surface in the environment (e.g., the ground) may be estimated from stereo image data and used for various downstream tasks. For example, the difference between a stereo disparity field and a ground disparity field may be used to detect objects, a representation of a navigable space may be generated by radially casting 2D rays in the ground disparity field, the ground disparity field may be used to compensate ego-motion for high dynamic attitude changes, and / or the ground disparity field may be lifted to 3D and used to fit a surface profile to points sampled from the lifted point cloud.

The invention discloses a remote sensing image stereo matching method and system based on a Mama model interpretation cost body, and the method comprises the steps: S1, extracting multi-level and multi-scale semantic features from input left and right remote sensing stereo image pairs, and outputting a stereo image pair feature map; s2, aligning the left and right stereo image pair feature maps on the parallax dimension to construct a three-dimensional cost body; s3, introducing a Mama framework based on a selective state space model, interpreting the cost body in combination with a multi-scale feature interaction mechanism, and generating a parallax probability distribution diagram; s4, mapping the probability distribution into a continuous disparity value, and outputting a disparity map; according to the method, adaptive feature screening and global modeling are realized through a dynamic weight selection mechanism of the state space model, the local receptive field limitation of 3D convolution is overcome, and the multi-granularity feature representation capability of the model is improved by promoting interaction and fusion among features of different scales by virtue of a cross-scale information interaction mechanism, so that the robustness of the model is improved. And the overall interpretation accuracy and efficiency are improved.

The invention discloses a machine vision-oriented three-dimensional image coding method, which comprises the following steps of: performing feature extraction on input left and right views by using a three-dimensional image feature extraction module to obtain three-dimensional image multi-scale features containing 3D scene information; a stereo image multi-scale feature coding module is used for coding the stereo image multi-scale features to obtain compact left and right viewpoint potential representations, and stereo entropy coding is carried out on the left and right viewpoint potential representations to obtain code streams for transmission; performing stereo entropy decoding on the transmitted code stream to obtain decoded left and right viewpoint potential representations, and decoding the left and right viewpoint potential representations at a decoding end by using a stereo image multi-scale feature decoding module to obtain reconstructed stereo image multi-scale features; and performing visual analysis on the reconstructed stereo image multi-scale features by using a 3D visual analysis module to obtain a 3D target detection result. According to the invention, the 3D visual task performance and the stereo image compression efficiency are effectively improved.

Various implementations disclosed herein include devices, systems, and methods that convert a mono image to a stereo image pair. For example, a process may obtain an input image including appearance values at pixel positions. The input image corresponds to an appearance of a scene from a first viewpoint. The process may further determine a depth image comprising depth values at original pixel positions that are mapped to corresponding pixel positions of the input image. The process may further generate a first output image corresponding to a second viewpoint of the scene different than the first viewpoint and a second output image corresponding to a third viewpoint of the scene different than the second viewpoint. The process may further enable comfort-based 3D style presets for adjusting comfort parameters for viewing a stereo image pair.

Provided is an electronic device including a camera module configured to sense an object based on a plurality of photoelectric conversion elements and generate stereo image data having a disparity based on at least two of the plurality of photoelectric conversion elements sharing a micro lens, a memory configured to store at least one instruction and the stereo image data, and a processing circuit configured to execute the at least one instruction to detect an object of interest based on the stereo image data, and generate a crop signal for instructing the camera module to sense a region of interest including the object of interest; and generate depth information based on the stereo image data.

A virtual image display device and a stereoscopic image establishing method are provided. The virtual image display device includes a head-mounted display and an accessory. The head-mounted display performs an image capturing operation on a first region of an object to obtain first image information. The accessory has a first image capturing device for performing an image capturing operation on a second region of the object to obtain second image information. The head-mounted display establishes stereoscopic image information of the object based on the first image information and the second image information.

A computer that includes a processor and a memory, the memory including instructions executable by the processor to select first side stereo images of first and second pairs of stereo images acquired at first and second time steps, respectively, and mask first and second first side stereo images, determine point features in masked first and second first side stereo images and determine line features in masked first and second first side stereo images. Matching point features in the masked first and second first side stereo images can be determined using a first attention / graph neural network (attn / GNN) based on keypoints determined based on the line features. Matching line features in the masked first and second first side stereo images can be determined using a second attn / GNN based on keypoints determined based on the line features. Three-dimensional (3D) locations in a scene can be determined by determining stereo disparity based on the matched point features included in the first first side stereo image and point features determined in a first second side stereo image of first and second stereo pairs of images and a 3D stereo camera pose can be determined by determining a perspective-n-point and line algorithm on the 3D locations.

The invention discloses a low-canopy-density single-tree-scale young forest remote sensing extraction scheme fusing two-dimensional and three-dimensional features, and the scheme comprises the steps: generating a multi-view epipolar line image based on a two-linear-array three-dimensional image of GF-7 and RPC parameters; generating a pyramid image for the multi-view epipolar line image, and determining a disparity map of the two linear array stereoscopic images based on the pyramid image; calculating a point cloud based on the disparity map and carrying out grid processing on the point cloud to generate a DSM; laser height measurement data of a satellite is used for optimizing the DSM, the optimized DSM is obtained, an isolated forest algorithm is adopted for carrying out image analysis on the optimized DSM, and pixels containing young forests are extracted; fusing the panchromatic rearview image and the multispectral image of the satellite remote sensing image to generate a panchromatic-multispectral fused image; performing shadow extraction on the panchromatic-multispectral fusion image by adopting a shadow detection algorithm to obtain a target shadow region; the pixel containing the young forest is compared with the target shadow area, the target young forest extraction result is determined, and the accuracy of the young forest extraction result can be improved.

The invention discloses a panoramic image determination method and device, equipment, a medium and a product. The method comprises the following steps: acquiring a current scene image, shot by at least one camera device, of an environment to which a vehicle belongs, and kinematics data and attitude data of the vehicle from a current shooting moment to a previous shooting moment; determining a scene image transformation matrix based on the kinematics data and the attitude data; acquiring a historical scene image shot by the camera device at a previous shooting moment, and performing stereo matching on the current scene image and the historical scene image based on the scene image transformation matrix and shooting parameters of the camera device to obtain a scene stereo image corresponding to the camera device; and determining and displaying a panoramic image based on the scene stereo image of each camera device. The authenticity, accuracy and reliability of panoramic image determination are improved, and the driving safety is improved.

The invention discloses a hub defect detection method based on a 2.5 D photometric stereo image. The method comprises the following steps: sending an acquired preprocessed hub image into a pre-trained deep learning model for identifying and positioning coating hub defects so as to output a detection result; wherein the preprocessing of the hub image comprises the following steps: acquiring images of a coated hub under various exposures through a camera, and fusing the images under the various exposures by adopting an exposure fusion algorithm to obtain the preprocessed hub image. The 2.5 D photometric stereo image technology is utilized, and supervised learning and unsupervised learning methods are combined, so that the detection of the defects of the high-reflective coating hubs with different colors is realized.

The invention provides a welding spot quality detection method and system, a binocular camera is combined with an image fusion algorithm to form a three-dimensional image of a welding spot, welding spot feature information is obtained through calculation and is automatically compared with technical standard requirements, the quality of the welding spot is judged, and cause feedback of disqualification is given. The method solves the problem of large manual misjudgment caused by the fact that the welding spot cannot be shot by 360 degrees in all directions in the re-inspection operation after AOI misjudgment, the understanding of the geometric shape of the welding spot can be improved by using an image fusion algorithm, and potential defects can be highlighted. And meanwhile, the welding spot characteristics are automatically compared, so that the detection speed, the detection efficiency and the detection accuracy are improved, and subjective errors possibly caused by manual detection are reduced. In addition, due to the reason feedback function, an inspector and the quality control process are greatly facilitated, the inspector is helped to rapidly position problems and take correction measures, and automation and accuracy of welding spot detection are improved.

The invention discloses a vision and laser collaborative assembly positioning method and system, and belongs to the technical field of industrial automation control, and the method comprises the steps: obtaining stereo image data and positioning light spot data, generating multi-source input data, and carrying out the analysis and matching, calculating initial pose estimation, and carrying out the positioning of the three-dimensional image data and the positioning light spot data. Generating a visual positioning result, comparing the visual positioning result with theoretical position data, correcting deviation, generating cooperative positioning data, combining a standard operation process file, generating a visual guiding scheme containing a display instruction and a laser control instruction, synchronously executing the display instruction and the laser control instruction, realizing assembly guiding, and performing assembly. And meanwhile, the assembly process is monitored in real time to generate operation progress data, and the detailed degree of the visual guide scheme is adjusted in combination with historical operation data. According to the method, dynamic compensation and cooperative positioning are carried out by acquiring the three-dimensional image and the laser spot data, and a virtual-real combined guiding scheme is adjusted in real time, so that high-stability self-adaptive assembly process control can be realized.

The invention discloses a power distribution area load state monitoring method and system based on stereo portrait analysis, relates to the technical field of urban power grids, and effectively improves the positioning precision and response speed of a load fluctuation area by introducing a three-dimensional image reconstruction and dynamic partitioning mechanism; a graph neural network and space-time long-short-term memory network modeling method is combined, so that the trend perception capability of the load state evolution process is enhanced; further, multi-source fusion recognition of abnormal behaviors is realized through association matching of the image change track and the load feature sequence, and the monitoring accuracy and sensitivity are effectively improved; based on risk index assessment and a map visualization expression mode, an abnormal area positioning result is visual and transparent, and intelligent early warning and decision assistance are supported; the method has the advantages of high data fusion capability, high risk identification precision, high visual interactivity and the like, and the intelligent and practical level of load state monitoring of the power distribution area is remarkably improved.

A method for adjusting a depth of a stereoscopic image includes the following. A first disparity correction amount is obtained according to a distance between centers of two lenses of a stereoscopic camera and an interpupillary distance (IPD) of a first observer. A second disparity correction amount is obtained according to a difference between image distances of a stereoscopic display and the stereoscopic camera at zero disparity. A third disparity correction amount is obtained according to the IPD of the first observer and an IPD of a second observer. A stereoscopic image of an object photographed by the stereoscopic camera which has been corrected by the first disparity correction amount, the second disparity correction amount, and the third disparity correction amount is transmitted to the stereoscopic display. A device for adjusting a depth of a stereoscopic image is also provided.

The present disclosure relates to an image modification process used to generate a chosen type of stereoscopic video of the common field of view shared by two cameras without requiring any pre-set and arrangement normally needed for stereoscopic display. The process can use left and right video streams of images originating from a wide range of camera types sharing some portion of their respective fields of view that are not limited to a specific relative orientation (convergent, parallel or divergent) and can be used to adapt in real time the stereoscopic video to be displayed to a viewer where his dynamic inputs are constantly updated. The proposed method can be used to optimize, correct and or allow the use of various apparatuses (webcams, cameras, scanners, etc.) that may come out of the production lines with imperfects to the capture cameras.

The invention provides a measurement control point-free island stereo image pair extraction high-precision DSM method, which comprises the following steps of: firstly, carrying out semi-global dense matching by adopting a semi-global dense matching algorithm to obtain a disparity map; secondly, performing coordinate three-dimensional positioning of the rational function model, and processing longitude, latitude and height coordinates to obtain a three-dimensional point cloud without deformation; and finally, realizing point cloud reconstruction DSM, and generating a DSM three-dimensional model. Experimental results show that the method for reconstructing the DSM by the high-resolution satellite stereo image pair is well applied to World-View3 satellite images, and the DSM three-dimensional model of the island reef can be constructed.

A high depth of field image generating apparatus according to the present disclosure includes a region segmentation unit which segments a region for a stereo image to generate region data, a depth estimating unit which estimates depths for the stereo image to generate depth data, and a high depth of field image generating unit which generates a high depth of field image from the stereo image, the region data, and the depth data.

The invention provides a dynamic polarization three-dimensional imaging system, which relates to the technical field of anti-counterfeiting encryption and comprises a light source, a polarization film and a color three-dimensional imaging module which are sequentially arranged along a light path, the light source is used for providing visible band incident light; the polarizing film is configured to transmit TM polarized light when rotating 0 degree and transmit TE polarized light when rotating 90 degrees; the color three-dimensional imaging module comprises a micro lens array, a transparent spacing layer and a micro image-text array which are sequentially arranged along a light path; the micro image-text array comprises a metal coating layer, a grating layer and a glass substrate which are stacked from top to bottom, wherein the grating layer forms a Fabry-Perot-like microcavity resonance structure; the polarization state of the incident light is switched by rotating the polarization film, so that the stereo image of the micro image-text array is dynamically switched among an appearing state, a hiding state or a color changing state. According to the invention, wide-color-gamut and dynamically-adjustable color stereo imaging is realized, and the complexity and the identifiability of anti-counterfeiting encryption are improved.

A perception system is disclosed. The perception system includes at least one memory configured to store machine executable instructions, and at least one processor configured to execute the stored executable instructions to: (i) extract camera features from stereo images; (ii) extract LiDAR features from a LiDAR point cloud; (iii) transform the camera features in a bird's-eye-view (BEV) space; (iv) transform the LiDAR features in the BEV space; and (v) fuse the transformed camera features and LiDAR features in the BEV space using a learned fusion with attention technique to generate the fused camera features and LiDAR features in the BEV space.

The invention relates to the technical field of image data processing, in particular to a suspension spring cold roll forming precision online detection system based on image recognition, which comprises an image acquisition and preprocessing module, a spring morphology characterization module, a spring image reconstruction module, a three-dimensional image alignment module and a dynamic monitoring module. The image acquisition and preprocessing module dynamically acquires a multi-angle image of the spring and determines a target area; the spring morphology characterization module performs morphology analysis, establishes a first morphology graph and extracts two-dimensional features; the spring image reconstruction module adopts Faster R-CNN to generate a two-dimensional mask, reconstructs a three-dimensional point cloud through a multi-angle image and extracts key nodes; and the three-dimensional image alignment module aligns the three-dimensional data with the standard model, and calculates the precision deviation to carry out qualification judgment. And the dynamic monitoring module can perform periodic execution, and triggers a stop signal to realize closed-loop control when the precision reaches the standard. In addition, the system adopts a dynamic deviation threshold, and the detection efficiency is improved according to layered processing of spring types.

The invention discloses a near-ground distance measurement method based on virtual large baseline four-eye vision, a medium and equipment, and the method comprises the steps: synchronously obtaining four images when a lifting appliance enters a near-ground operation range; based on the calibration relation between the cameras, the high-altitude images are respectively projected and transformed to the visual angles of the corresponding lifting appliance cameras and fused, two virtual camera images are generated, and therefore a virtual large base line far exceeding the physical distance is constructed; performing three-dimensional correction and cutting on the virtual image pair to obtain a row-aligned three-dimensional image pair; performing feature matching and triangulation on the virtual image pair to generate a sparse reference depth map; and inputting the stereo image pair and the sparse depth map into a pre-trained depth estimation neural network model together, outputting a dense depth map, and converting the dense depth map into a coordinate system taking the lifting appliance as an original point to obtain a vertical distance. According to the invention, the ultra-large baseline is virtually synthesized by using the existing camera, the remote distance measurement precision is improved, the cost is low, and the reliability is high.

The techniques described herein relate to systems, apparatus, articles of manufacture, and methods for gaze angle triggered fundus imaging. An example method includes detecting a pupil of a subject in a stereo image, controlling at least one actuator to align an imaging path of a fundus camera with the pupil after detecting the pupil in the stereo image, and capturing a fundus image of a retina of the subject at a gaze angle after determining that the pupil is oriented towards a target direction based on the gaze angle associated with the pupil in the image.