25 results about "Depth plane" patented technology

An orientation assistance system comprises means for acquiring a real or virtual visual environment, non-visual human-machine interface means and means for processing the digital representation of the visual environment in order to provide an electrical signal for controlling a haptic interface. The means for processing the digital representation periodically extracts at least one pulsed digital activation pattern for a subset of spikes of the haptic region. The haptic interface consists of a lumbar belt having an active surface of N×M actuators, where N and M are integers greater than or equal to 10. The processing means provides, for each acquisition of the visual environment, a sequence of P activation frames for the actuators, where P is an integer between 2 and 15, preferably between 5 and 10, each of the frames corresponding to the representation of the environment in an incremental depth plane.

The application discloses a kind of fruit tree image acquisition method and system based on two-dimensional skeleton and viewpoint planning, it is related to wisdom agricultural equipment technical field, method includes: obtaining crown image obtains branch mask and depth information;To mask thinning pruning obtains initial skeleton, identifies feature point and calculates growth direction after artifact removal;Based on feature point and direction vector, topological reconstruction is carried out to broken skeleton, and complete two-dimensional skeleton graph is obtained;Based on two-dimensional skeleton graph, viewpoint planning is carried out in reference depth plane, generates viewpoint sequence and is mapped into three-dimensional pose, realizes autonomous image acquisition.The application solves the problem of information loss caused by occlusion and three-dimensional reconstruction calculation lag, and realizes real-time, safe and high-resolution image acquisition.

The present specification relates to a technique by which a hologram generation device generates a hologram with reduced speckle noise by optimizing a carrier according to a specific distance. A method by which a hologram generation device generates a hologram, according to an embodiment of the present specification, comprises the steps of: setting a carrier by initializing an arbitrary complex field having a random phase and a random amplitude; removing a specific band component from the frequency band of the set carrier so as to transform the set carrier; propagating the transformed carrier for each specific distance, which is a depth plane in which a target object is present; setting a loss function for each specific distance on the basis of a target uniform amplitude at the specific distance and the amplitude of the carrier propagated to the specific distance; and updating the propagated carrier for each specific distance on the basis of the loss functions set for each specific distance.

This application relates to a 3D reconstruction method, apparatus, device, readable storage medium, and program product. The method includes: acquiring multi-view images of a target object and camera parameters during the acquisition of the multi-view images; determining the spatial range of the depth planes and the number of depth planes of the target object; determining the pixel movement range and unit movement amount of pixels in the multi-view images based on the spatial range and number of depth planes; determining the pixel movement positions of pixels in the multi-view images based on the pixel movement range and unit movement amount, with adjacent pixel movement positions spaced apart by a unit movement amount, and the number of pixel movement positions matching the number of depth planes; determining each virtual depth plane based on each pixel movement position; and performing 3D reconstruction of the target object based on the multi-view images, camera parameters, and each virtual depth plane to obtain the 3D reconstruction result of the target object. This method can improve the integrity of the 3D reconstruction.

Systems and methods are disclosed for operating a head-mounted display system based on user perceptibility. The display system may be an augmented reality display system configured to provide virtual content on a plurality of depth planes by presenting the content with different amounts of wavefront divergence. Some embodiments include obtaining an image captured by an imaging device of the display system. Whether a threshold measure or more of motion blur is determined to be exhibited in one or more regions of the image. Based on a determination that the threshold measure or more of motion blur is exhibited in one or more regions of the image, one or more operating parameters of the wearable display are adjusted. Example operating parameter adjustments comprise adjusting the depth plane on which content is presented (e.g., by switching from a first depth plane to a second depth plane), adjusting a rendering quality, and adjusting power characteristics of the system.

The invention discloses a structure multi-target deformation monitoring method based on visual segmentation and deblurring enhancement, and relates to the technical field of engineering structure health monitoring. The method comprises the following steps: acquiring continuous time sequence multi-target images of different depth-of-field planes of a to-be-monitored structural plane; inputting an interactive prompt point into a pre-training segmentation model in the first frame to generate a multi-target mask, and removing a noise region based on connected domain analysis; cutting by adopting a self-adaptive multi-scale cutting strategy based on boundary judgment to obtain sub-graphs; performing image enhancement on the sub-image by adopting a pre-trained deblurring neural network; extracting a central point and an external rectangle coordinate of a target mask area in the sub-graph and uniformly mapping the central point and the external rectangle coordinate back to a full-graph coordinate system; and dynamically updating a cutting window of a subsequent frame based on the central point and boundary information of the first frame, and converting pixel-level displacement into physical unit or three-dimensional coordinate change by comparing the central point positions of the first frame and the subsequent frame and combining camera calibration parameters, thereby realizing high-precision continuous monitoring.

Systems and methods for depth plane selection in display system such as augmented reality display systems, including mixed reality display systems, are disclosed. A display(s) may present virtual image content via image light to an eye(s) of a user. The display(s) may output the image light to the eye(s) of the user, the image light to have different amounts of wavefront divergence corresponding to different depth planes at different distances away from the user. A camera(s) may capture images of the eye(s). An indication may be generated based on obtained images of the eye(s), indicating whether the user is identified. The display(s) may be controlled to output the image light to the eye(s) of the user, the image light to have the different amounts of wavefront divergence based at least in part on the generated indication indicating whether the user is identified.

A virtual reality (VR) and / or augmented reality (AR) display system is configured to control a display using control information that is embedded in or otherwise included with imagery data to be presented through the display. The control information can indicate depth plane(s) and / or color plane(s) to be used to present the imagery data, depth plane(s) and / or color plane(s) to be activated or inactivated, shift(s) of at least a portion of the imagery data (e.g., one or more pixels) laterally within a depth plane and / or longitudinally between depth planes, other adjustment(s) to the virtual imagery, and / or other suitable controls.

This disclosure provides a method, apparatus, electronic device, and medium for reconstructing the dose distribution of a particle irradiation target area. The method includes: monitoring the irradiation information of particles on a monitoring plane along the incident path, including beam spot position, beam spot size, and dose corresponding to each position, wherein the particles include particles of various energies; calculating the dose of particles of different energies at different depth planes in the target area based on the particle dose ratio of particles of different energies at different depth planes in the target area to the monitoring plane; calculating the particle dose of each energy particle at each position in the target area at different depth planes based on the particle distribution pattern of each energy particle on the monitoring plane; constructing a planar dose model of the target area at different depth planes based on the particle dose of each energy particle at each position in the target area at different depth planes, and reconstructing a three-dimensional dose model by superimposing the models; and fusing the three-dimensional dose model with a three-dimensional image of the target area to obtain a three-dimensional particle dose distribution model of the target area.

This invention proposes a large depth-of-field holographic 3D display method based on a digital cone lens. The method includes the following steps: First, based on depth information, the 3D object to be recorded is segmented into different depth planes, and the wavefront information of each depth plane is extracted. Then, an improved 3D Fourier algorithm is used to obtain the phase distribution of the 3D object on the holographic plane, denoted as Φ. I The second step is to encode the optical cone lens into a digital cone lens using phase encoding, and denote the phase distribution of the digital cone lens as Φ. II The third step involves co-encoding the digital cone lens with the hologram of the 3D object, and then coherently superimposing the phase distribution Φ. I and Φ II The final large depth-of-field hologram is obtained. In the fourth step, the final large depth-of-field hologram is loaded onto the spatial light modulator, and the spatial light modulator is illuminated with a light source. Based on the 3D Fourier algorithm and the depth-of-focus characteristics of the cone lens, the large depth-of-field hologram of the 3D object is reconstructed.

The invention relates to holographic display, in particular to a phase-type hologram generation method and a waveguide-based holographic display system, and the method comprises the steps: dividing a three-dimensional image into a plurality of layered depth images located on different depth planes according to depth information, and extracting the amplitude of each layered depth image; generating a random initial complex amplitude on a hologram plane; forward-propagating the random initial complex amplitude to reconstruction planes at different distances through angular spectrum diffraction; extracting the phase of the complex amplitude of each reconstructed image, and combining the phase with the amplitude of the corresponding layered depth image to form a new complex amplitude; the new complex amplitude of each reconstructed image is reversely propagated to a hologram plane through angular spectrum diffraction; superposing all complex amplitudes of the hologram, performing phase extraction, and combining the extracted phase with the plane wave amplitude to form a new complex amplitude; according to the technical scheme provided by the invention, the defects that in the prior art, three-dimensional immersive experience is difficult to provide for a user, and meanwhile, light weight and portability of equipment are difficult to realize can be effectively overcome.

A method for stereoscopic telestration in a 3D display system, and system therefore includes the steps of, and hardware for, aligning a telestration base plane to a reference object visible in a stereoscopic video stream; fusing a telestration overlay at the telestration base plane to ensure accurate depth alignment; and repositioning the telestration overlay to a moved telestration plane in front of or behind the telestration base plane by adjusting left and right eye overlays symmetrically. The repositioning may be performed without losing positional accuracy relative to the stereoscopic image stream.

This invention discloses a method for monitoring multi-target deformation of structures based on visual segmentation and deblurring enhancement, relating to the field of engineering structural health monitoring technology. The method includes: acquiring continuous temporal multi-target images of different depth planes of the structure to be monitored; generating a multi-target mask in the first frame by inputting interactive prompts into a pre-trained segmentation model, and removing noise regions based on connected component analysis; cropping using an adaptive multi-scale cropping strategy based on boundary determination to obtain sub-images; enhancing the sub-images using a pre-trained deblurring neural network; extracting the center point and circumscribed rectangle coordinates of the target mask regions within the sub-images and mapping them back to the full image coordinate system; dynamically updating the cropping window of subsequent frames based on the center point and boundary information of the first frame; and converting pixel-level displacement into physical units or three-dimensional coordinate changes by comparing the center point positions of the first and subsequent frames and combining camera calibration parameters, thereby achieving high-precision continuous monitoring.

The invention relates to a three-dimensional reconstruction method, device and equipment, a readable storage medium and a program product. The method comprises the following steps: acquiring a multi-view image for a target object and camera parameters when the multi-view image is acquired; determining a depth plane space range and a depth plane number of the target object; determining a pixel movement range and a unit movement amount of pixels in the multi-view image according to the depth plane space range and the depth plane number; based on the pixel movement range and the unit movement amount, pixel movement positions of pixels in the multi-view image are determined, the adjacent pixel movement positions are spaced by the unit movement amount, and the number of the pixel movement positions is matched with the number of the depth planes; respectively determining each virtual depth plane according to the moving position of each pixel; and based on the multi-view image, the camera parameters and the virtual depth planes, performing three-dimensional reconstruction on the target object to obtain a three-dimensional reconstruction result of the target object. By adopting the method, the completeness of three-dimensional reconstruction can be improved.

The present invention relates to a method and system for processing a digital representation of a visual environment to control a haptic interface including a waistband having active surfaces of N×M actuators, where N and M are integers greater than or equal to 10, comprising an actuator activation protocol that computes a sequence of P activation frames for these actuators for each acquisition of the visual environment, where P is an integer between 2 and 1000, preferably between 5 and 50, each of these frames corresponding to a representation of the environment in an incremental depth plane, characterized in that the method further comprises a processing step of applying a context correction function that modifies the haptic pixel activation protocol.

Methods and systems are disclosed for presenting virtual objects on a limited number of depth planes using, e.g., an augmented reality display system. A farthest one of the depth planes is within a mismatch tolerance of optical infinity. The display system may switch the depth plane on which content is actively displayed so that the content is displayed on the depth plane on which a user is fixating. The impact of errors in fixation tracking is addressed using partially overlapping depth planes. A fixation depth is determined and the display system determines whether to adjust selection of a selected depth plane at which a virtual object is presented. The determination may be based on whether the fixation depth falls within a depth overlap region of adjacent depth planes. The display system may switch the active depth plane depending upon whether the fixation depth falls outside the overlap region.

A display system includes a head-mounted display configured to project light, having different amounts of wavefront divergence, to an eye of a user to display virtual image content appearing to be disposed at different depth planes. The wavefront divergence may be changed in discrete steps, with the change in steps being triggered based upon whether the user is fixating on a particular depth plane. The display system may be calibrated for switching depth planes for a main user. Upon determining that a guest user is utilizing the system, rather than undergoing a full calibration, the display system may be configured to switch depth planes based on a rough determination of the virtual content that the user is looking at. The virtual content may have an associated depth plane and the display system may be configured to switch to the depth plane of that virtual content.

A method for representing a three-dimensional scene stored as a three-dimensional dataset includes determining a set of P depth planes along a viewing direction, the method includes generating, for each depth plane from the three-dimensional dataset, a proxy three-dimensional dataset including the P proxy images by (i) constructing a three-dimensional dataset, and (ii) generating a proxy image of the P proxy images from at least one cross-sectional image of a plurality of cross-sectional images, each representing a respective cross-section of the three-dimensional scene at a respective scene depth.

Methods and systems for depth-based foveated rendering in a display system are disclosed. The display system may be an augmented reality display system configured to provide virtual content on a plurality of depth planes using different wavefront divergence. Some embodiments include monitoring eye orientations of a user of the display system. A fixation point can be determined based on the eye orientations, the fixation point representing a three-dimensional location with respect to a field of view. Location information of virtual object(s) to present is obtained, with the location information including three-dimensional position(s) of the virtual object(s). A resolution of the virtual object(s) can be adjusted based on a proximity of the location(s) of the virtual object(s) to the fixation point. The virtual object(s) are presented by the display system according to the adjusted resolution(s).

The invention relates to a light field calculation imaging system based on rotating prisms, which comprises a camera, at least one rotating prism and a controller, and is characterized in that the controller is respectively connected with the rotating prism and the camera, the rotating prism is located between the camera and a target to be imaged, and the controller is configured to execute the following steps: S1, constructing a homography relationship between an actual light field and a theoretical light field; s2, obtaining a discretized angle sequence of the rotating prism according to the target resolution, and obtaining a corresponding sub-angle image sequence of the to-be-imaged target collected by the camera based on the angle sequence; s3, a plurality of refocusing depth planes are determined, sub-angle image sequences reconstructed on the depth planes are obtained, and refocusing images corresponding to the refocusing depth planes are obtained through calculation; and S4, for each pixel point, calculating a cost function value of the pixel point corresponding to each refocusing image, and selecting the depth value of the refocusing depth plane corresponding to the refocusing image with the optimal cost function value as the depth value of the refocusing depth plane. Compared with the prior art, high-angle-resolution light field sampling can be carried out, high-precision mapping of light field data from sampling to reconstruction is achieved, and meanwhile the compactness and flexibility of the system structure are guaranteed.

This invention discloses a method for resolving gaze depth ambiguity in near-eye AR devices, an interactive device, and a storage medium. The method includes: constructing a sparse scene plane table; continuously extracting temporal features of eye movement patterns based on pupil image sequences acquired by a near-eye camera; identifying gesture action types and hand depth information; fusing eye movement pattern temporal features and gesture state information to infer the user's current task stage and corresponding gaze depth range; intersecting the current gaze direction vector with each plane in the sparse scene plane table; selecting the candidate plane with the highest confidence as the depth plane of the current gaze target, and outputting the estimated two-dimensional image coordinates and three-dimensional world coordinates of the gaze intersection point. This invention replaces the binocular convergence scheme with a monocular near-eye camera, significantly reducing hardware costs and power consumption while improving the gaze depth disambiguation accuracy in multi-target depth overlap scenes through task state semantic constraints.

A method for stereoscopic telestration in a 3D display system, and system therefore includes the steps of, and hardware for, aligning a telestration base plane to a reference object visible in a stereoscopic video stream; fusing a telestration overlay at the telestration base plane to ensure accurate depth alignment; and repositioning the telestration overlay to a moved telestration plane in front of or behind the telestration base plane by adjusting left and right eye overlays symmetrically. The repositioning may be performed without losing positional accuracy relative to the stereoscopic image stream.

The invention discloses a video interpolation frame generation method and device, equipment and a medium. The method comprises the following steps: acquiring adjacent video frame images which are respectively a front frame image and a rear frame image; in combination with the camera parameter matrix, processing the post-frame image re-projection to obtain a plane scanning body image; obtaining a mixed weight and a background image of a multi-plane image according to the previous frame image and the plane scanning body image, constructing the multi-plane image comprising a plurality of depth layers, and then obtaining a plurality of depth plane graphs; transforming each depth plane graph to obtain an intermediate reference frame image; extracting image multi-scale features based on the front frame image, the rear frame image and the middle reference frame image, and then refining the optical flow layer by layer to obtain a fusion image; and aligning the fused image, extracting context features of the image, and generating a video interpolation frame. According to the method, space structure information and motion modeling are effectively combined in the interpolation process to generate the interpolation video frame with relatively high precision.

The invention discloses a multi-view image super-resolution method and system based on unified three-dimensional representation, and belongs to the field of image processing and computer vision. Comprising the following steps: acquiring a plurality of low-resolution images of different visual angles in the same scene; estimating camera parameters corresponding to each view angle and extracting visual feature representation of each view angle; respectively projecting the feature representation of each view angle to a series of discrete depth planes which are arranged in parallel and have different fixed depth values, and fusing the features of a plurality of view angles projected to the same position to generate a high-resolution multi-plane image fusing the complementary texture and geometric information of all view angles, and taking the high-resolution multi-plane image as unified three-dimensional space representation; and rendering a high-resolution reference image corresponding to the target visual angle from the high-resolution multi-plane image, and performing feature fusion and super-resolution reconstruction on the low-resolution image and the high-resolution reference image corresponding to the target visual angle to generate a super-resolution image of the target visual angle.

The application provides a passive positioning and navigation method and system in a complex cabin, the method comprising: processing inertial measurement data, visible light images and depth images, fusing depth information of visual feature points to generate initial pose estimation through visual-inertial tight coupling optimization; when cabin text identifiers or visual labels are detected in the visible light images, text features or label corner pixel coordinates are extracted, and inverse depth plane constraints and absolute pose constraints are respectively constructed; cabin text identifiers and visual labels in the complex cabin are used as passive and high-precision calibration sources to provide effective geometry and absolute pose constraints for pose calibration, and the defects of the existing pure visual-inertial positioning in lacking effective calibration mechanism are made up; the initial pose estimation is calibrated based on the inverse depth plane constraints and the absolute pose constraints to obtain target pose estimation; and navigation guide information is generated based on the target pose estimation to realize visual display, and high-precision and high-robustness positioning and navigation in the complex cabin environment.