40 results about "Perspective camera" patented technology

A method includes determining at an extended reality (XR) device, a first set of vertex adjustment values of a distortion mesh and receiving image frame data of a scene captured at a first time and at a first head pose using a see-through camera of the XR device. The method further includes applying the first set of vertex adjustment values of the distortion mesh to the image frame data to obtain intermediate image data, and predicting a second head pose at a second time subsequent to the first time. The method also includes generating, based on the predicted second head pose, a second set of vertex adjustment values of the distortion mesh, applying the second set of vertex adjustment values of the distortion mesh to the intermediate image data to generate a rendered virtual frame and displaying the rendered virtual frame by the XR device at the second time.

A computer-implemented method includes converting by a pose tokenizer, based on a learned codebook, pose parameters of a body into a sequence of discrete pose tokens; randomly masking a portion of the sequence of discrete pose tokens; predicting the randomly masked sequence of discrete pose tokens based on multi-scale features extracted from a monocular image by an image conditioned masked transformer; optimizing the sequence of discrete pose tokens by aligning a re-projected three-dimensional (3D) pose with an estimated two-dimensional (2D) pose; directly regressing, from the multi-scale features, a shape parameter of the body and a weak perspective camera parameter; and generating a 3D mesh reconstruction of the body based on the shape parameter and the weak perspective camera parameter.

The utility model relates to an MR head display system with a medium-high definition perspective camera. The MR head display system comprises an RGB binocular image sensor, a flexible board, a TOF camera, a signal processing board and a 3D image video stream display module. The 3D image video stream display module comprises a binocular OLED (Organic Light Emitting Diode) display screen and two RGB (Red, Green and Blue) camera mainboards; the display screen is electrically connected with the RGB camera mainboards; the two RGB camera mainboards are connected with the signal processing board through the flexible board and transmit shot video information of a real scene to the signal processing board; the signal processing board is also connected with the TOF camera and the PC and is used for receiving distance information transmitted by the TOF camera and VR virtual reality images stored in the PC; the signal processing board transmits the generated 3D image video stream to the two RGB camera mainboards through the flexible board; and the two RGB camera mainboards transmit the 3D image video stream to the binocular display screen for display through the video interface.

A method includes determining that an inter-pupil distance (IPD) between display lenses of a video perspective (VST) augmented reality (XR) device has been adjusted relative to a default IPD. The method further includes obtaining an image captured using a perspective camera of the VST XR device. The perspective camera is configured to capture an image of a three-dimensional (3D) scene. The method further includes transforming the image according to a change in the IPD relative to the default IPD to match a viewpoint of a corresponding one of the display lenses to generate a transformed image. The method further includes correcting distortion in the transformed image based on one or more lens distortion coefficients corresponding to the change in the IPD to generate a corrected image. Further, the method includes initiating presentation of the corrected image on a display panel of the VST XR device.

The disclosed systems and techniques facilitate efficient detection and classification of traffic signs in driving environments. The disclosed techniques include, obtaining, using a sensing system of a vehicle a first set of perspective camera images of an environment and a second set of radar images of the environment. The techniques further include generating, using a first neural network, one or more camera features characterizing the first set of images, generating, using a second neural network, one or more radar features characterizing the second set of images, and processing the one or more camera features and the one or more radar features to obtain an identification of one or more traffic signs in the environment.

A method includes determining at an extended reality (XR) device, a first set of vertex adjustment values of a distortion mesh and receiving image frame data of a scene captured at a first time and at a first head pose using a see-through camera of the XR device. The method further includes applying the first set of vertex adjustment values of the distortion mesh to the image frame data to obtain intermediate image data, and predicting a second head pose at a second time subsequent to the first time. The method also includes generating, based on the predicted second head pose, a second set of vertex adjustment values of the distortion mesh, applying the second set of vertex adjustment values of the distortion mesh to the intermediate image data to generate a rendered virtual frame and displaying the rendered virtual frame by the XR device at the second time.

The disclosed systems and techniques facilitate efficient detection and classification of traffic signs in driving environments. The disclosed techniques include, obtaining, using a sensing system of a vehicle a first set of perspective camera images of an environment and a second set of radar images of the environment. The techniques further include generating, using a first neural network, one or more camera features characterizing the first set of images, generating, using a second neural network, one or more radar features characterizing the second set of images, and processing the one or more camera features and the one or more radar features to obtain an identification of one or more traffic signs in the environment.

A method may include obtaining an image captured by a perspective camera of a VST AR device. The method may include identifying a first distortion generated by at least one see-through camera lens, and identifying a second distortion generated by at least one display lens of the VST AR device. The method may include determining a combined distortion based on the first distortion and the second distortion. The method may include pre-warping the image to counteract the combined distortion such that the image is undistorted when a user views the pre-warped image through at least one display lens of the VST AR device. Further, the method may include presenting the pre-warped image to a user on at least one display of the VST AR device.

The present application relates to a kind of video perspective camera automatic focusing method and system for XR head-mounted display, belong to image processing technical field, solve the existing video perspective camera focusing speed slow, and the depth information of monocular image and focusing parameter cannot be effectively mapped problem.Method includes: the current image and its corresponding depth map of the video perspective camera of XR head-mounted display are collected;According to the fixation point coordinates of user in current image, the local region with fixation point coordinates as center is extracted from depth map as fixation area depth map;Fixation area depth map and fixation point coordinates are input into trained focusing parameter prediction model, and the target focusing parameter of video perspective camera is obtained;According to target focusing parameter, video perspective camera is adjusted, and single focusing is completed.Accurate mapping from image to focusing parameter is realized, and focusing speed is improved.

The disclosed systems and techniques facilitate efficient detection and classification of traffic signs in a driving environment. The disclosed techniques include obtaining a first set of perspective camera images of an environment and a second set of radar images of the environment using a sensing system of a vehicle. The technique further includes generating one or more camera features characterizing the first set of images using a first neural network, generating one or more radar features characterizing the second set of images using a second neural network, and processing the one or more camera features and the one or more radar features to obtain an identification of the one or more traffic signs in the environment.

The invention relates to the technical field of video fusion, and discloses a video fusion method and system based on a shadow map and a program product, and the method comprises the following steps: creating a virtual perspective camera in a three-dimensional scene, and generating the shadow map according to a visual cone of the virtual perspective camera; pixels in the three-dimensional scene are converted from a visual space coordinate system of a physical world camera to a virtual perspective camera coordinate system, NDC coordinates of the pixels are obtained, the NDC coordinates are aligned with texture coordinates of the shadow map, and sampling texture coordinates are obtained; judging whether the pixels in the visual range of the virtual perspective camera are in the visual area of the shadow map or not, and obtaining the color of the current pixel of the three-dimensional scene; and obtaining the color of the current pixel of the fused three-dimensional scene. According to the invention, the problems of shielding area processing errors and the like in the prior art are solved.

The invention discloses a dynamic scene online three-dimensional reconstruction system and method for humanoid robot teleoperation. The method comprises the following steps: receiving an RGB-D video data stream collected by a first visual angle camera of a humanoid robot; performing dynamic and static separation processing on the RGB-D video data stream to obtain a mask graph comprising a static background and a dynamic object; constructing a static Gaussian map dictionary and a dynamic Gaussian map dictionary based on the mask graph; carrying out real-time asynchronous updating on the static Gaussian map dictionary and the dynamic Gaussian map dictionary by adopting a dual key frame selection strategy; and performing fusion rendering on the static Gaussian map dictionary and the dynamic Gaussian map dictionary to obtain a three-dimensional scene view for teleoperation, and completing online three-dimensional reconstruction of a dynamic scene. The method can be used for dynamic scenes where the humanoid robot performs teleoperation, such as training data acquisition in life scenes, electric power inspection, dangerous scene inspection and the like, and a real-time and immersive three-dimensional environment view is provided for a teleoperator.

Imaging systems and methods compensate for wigwag movement of a C-arm fluoroscope to refine camera pose estimates. The methods involve computing a primary movement axis from samples of markers in fluoroscopic images of a fluoroscopic sweep of a structure of markers and processing the primary movement axis to obtain a secondary movement axis. The methods further involve aligning two-dimensional samples of each marker with the primary and secondary movement axes to obtain an aligned signal and determining a difference signal for a secondary component of the aligned signal. The difference signal is then converted to a rotation axis translation signal. The method further involves estimating a 3D position of the rotation axis. The estimated pose of the C-arm fluoroscope is then refined to compensate for the wigwag movement using the rotation axis translation signal and the estimated 3D position of the rotation axis.

The invention discloses a production line display method and device based on a three-dimensional rendering technology and electronic equipment, and relates to the technical field of intelligent manufacturing, and the method comprises the steps: creating L target models corresponding to a production line, the L target models being three-dimensional models corresponding to production equipment and equipment connection assemblies in the production line, the data volume of each target model is smaller than a preset data volume threshold; creating a scene container, a perspective camera and a renderer corresponding to the production line, and loading the L target models to the scene container through the renderer to obtain an initial display scene of the production line; based on the layout information of the production line, the equipment operation data and the light information, updating the initial display scene to obtain a target display scene of the production line; and dynamically displaying the target display scene of the production line through the perspective camera. The technical problems of low visualization degree and low production line management efficiency in simulation display of the industrial production line based on the prior art are solved.

The application provides a sparse-to-dense visual positioning method and system based on feature Gaussian splash, and the method comprises the following steps: initializing a color-decoupled feature Gaussian field based on a training image set, optimizing the color-decoupled feature Gaussian field based on a query feature map set in combination with feature rendering and feature alignment loss cyclic optimization, and outputting a compact feature Gaussian scene model; screening a Gaussian landmark set in the compact feature Gaussian scene model by using a matching-oriented sampling strategy; training a scene-specific detector; extracting sparse local features of a query landmark heat map corresponding to a query image and performing sparse feature matching with the Gaussian landmark set to obtain an initial pose of a query perspective camera; based on 3D Gaussian splash, rendering a dense feature map and a depth map of the query perspective in the compact feature Gaussian scene model by using the initial pose of the query perspective camera, performing cluster-based proxy matching-based sparse-to-dense accelerated pose optimization, and obtaining accurate positioning of the query perspective camera.

The invention relates to the technical field of medical image processing, in particular to a neonatal feeding behavior image recognition and oral movement evaluation system which comprises a data acquisition module, a feature extraction module, a differential geometric modeling module, a trajectory analysis module, a collaborative movement analysis module, an anomaly detection module, a result display module and an application service module. According to the system, a high-speed camera, a three-dimensional face tracking camera and an optical perspective camera work cooperatively, multi-view image data of newborn face and oral cavity movement are synchronously collected through a data collection analyzer, and a differential geometry theory is innovatively applied to oral cavity movement analysis. Facial feature point groups are regarded as Riemannian manifolds embedded in a three-dimensional Euclidean space, oral movement characteristics are accurately described through curvature feature calculation, shape operator construction and differential invariant extraction, a coupling model of lip manifolds and mandibular manifolds is systematically constructed, synchronism measurement and topological features between the manifolds are analyzed, and the oral movement characteristics are accurately described. And accurate detection of feeding abnormity is realized.

The invention provides a heterogeneous binocular stereoscopic vision three-dimensional measurement method with a telecentric lens and a perspective lens, which comprises the following steps of: firstly, constructing a heterogeneous binocular vision three-dimensional measurement system, then calibrating the heterogeneous binocular vision three-dimensional measurement system, and synchronously shooting calibration objects in different poses by using two cameras in the system, closed-form solutions of internal and external parameters of the telecentric camera and the perspective camera are respectively solved, lens distortion and consistency of relative poses among all image pairs are considered, a re-projection error optimization equation is established, and finally a calibration result is used for three-dimensional reconstruction. According to the stereoscopic vision system with the non-paired telecentric and perspective lenses, the potential of the telecentric lens in the vision system is released in 3D measurement application, and an accurate and reliable calibration result is provided.

The invention discloses a self-adaptive panoramic picture interpolation method based on Lanczos2, and belongs to the field of real image processing. The conventional image interpolation is based on a rectangular photo of a conventional perspective camera, and the information density of each part in the photo is consistent, so that each part of the photo needs to be considered equivalently. However, for a 2: 1 panorama, the information density degrees of different latitudes are different, so that direct interpolation is not the best method. Therefore, the invention provides a self-adaptive panoramic picture interpolation algorithm based on pixel latitude and edge conditions. According to the method, the sampling pixel weight is determined according to the latitude difference between pixel blocks and the gray level change condition of the current pixel part. According to surrounding pixel features, each pixel in the high-resolution graph is interpolated, the higher the latitude is, the smaller the information density is, the influence is lower, and the lower the latitude is, otherwise, the lower the information density is. Therefore, the high-influence pixels are highlighted, the feature expression of the high-influence pixels is enhanced, the image is smoother, and meanwhile, when the map is applied to the sphere, more information can be shown.

The invention discloses a visual training animation production method and device, equipment and a medium, and relates to the technical field of visual training, and the visual training animation production method comprises the following steps: obtaining a plurality of display elements, and mapping the display elements to a virtual three-dimensional space provided with a perspective camera; the display elements are replaceable elements; the display elements are to-be-displayed elements stored in an associated medium of the display device and / or elements displayed in a reference area of the display device; determining a moving direction vector corresponding to the display element, and moving the display element in the virtual three-dimensional space according to a preset element moving sequence on the basis of a preset moving rule and the moving direction vector corresponding to the display element, the display elements are projected and rendered to the reference area by using the perspective camera every preset frame rendering time interval so as to complete animation production; the preset movement rule is that the display element moves between the farthest plane and the nearest plane. The convenience and universality of visual training can be improved.

A method includes obtaining, using multiple imaging sensors of an electronic device, a left image frame and a right image frame forming a stereo pair of image frames. The method also includes identifying, using at least one processing device of the electronic device, extrinsic parameters associated with relative positions and orientations of the imaging sensors. The method further includes performing, using the at least one processing device, an online stereo rectification of the stereo pair of image frames based on the extrinsic parameters such that epipolar lines of the left and right image frames are horizontally aligned to generate a rectified stereo pair of image frames. In addition, the method includes rendering, using the at least one processing device, one or more images for display based on the rectified stereo pair of image frames.

Estimating temporally consistent 3D human body shape, pose, and motion from a monocular video is a challenging task due to occlusions, poor lightning conditions, complex articulated body poses, depth ambiguity, and limited availability of annotated data. Embodiments of present disclosure provide a method for temporally consistent motion estimation from monocular video. A monocular video of person(s) is captured by a weak perspective camera and spatial features of body of the persons are extracted from each frame of the video. Then, initial estimates of body shape, body pose, and features of the weak perspective camera are obtained. The spatial features and initial estimates are then aggregated to obtain spatio-temporal features by a combination of self-similarity matrices between the spatial features, pose and the camera and self-attention maps of the camera features and the spatial features. The spatio-temporal aggregated features are then used to predict shape and pose parameters of the person(s).

The application discloses a regression-based parameterized non-model 3D human body posture and shape prediction method and device. The method comprises the following steps: processing the scaled human body image by using a non-model method, acquiring 3D vertices, image features and weak perspective camera parameters; projecting the 3D vertices to a 2D plane to acquire 2D vertices; inputting the 2D vertex coordinates and the image features into a body part perception sampling module to acquire part perception image features and initial T posture 3D vertex coordinates; connecting the part perception image features and the initial T posture 3D vertex coordinates by using a ConCat operation and inputting the connected part perception image features and the initial T posture 3D vertex coordinates into a body part decoding module to decode and regress to calculate absolute rotation and translation information of each part of the body; calculating relative rotation of each joint relative to a parent joint according to the absolute rotation information of each part of the body to obtain posture parameters; inputting the posture parameters and the 3D vertex coordinates into a shape regression module, converting the 3D vertex coordinates into standard T posture 3D vertex coordinates and inputting the standard T posture 3D vertex coordinates into a series of full connection layers to regress and calculate shape parameters.

The present application discloses a method for rendering a panoramic video. The method includes obtaining a current frame of image from a video source, and generating texture map data, where the generating texture map data comprises determining a viewpoint region based on a field of view of a perspective camera, and rendering image pixels outside the viewpoint region at a lower resolution than rendering image pixels within the viewpoint region; mapping the current frame of image to a three-dimensional image based on a spherical rendering model the texture map data; and projecting the three-dimensional image onto a two-dimensional screen.

A method includes determining that an inter-pupillary distance (IPD) between display lenses of a video see-through (VST) extended reality (XR) device has been adjusted with respect to a default IPD. The method also includes obtaining an image captured using a see-through camera of the VST XR device. The see-through camera is configured to capture images of a three-dimensional (3D) scene. The method further includes transforming the image to match a viewpoint of a corresponding one of the display lenses according to a change in IPD with respect to the default IPD in order to generate a transformed image. The method also includes correcting distortions in the transformed image based on one or more lens distortion coefficients corresponding to the change in IPD in order to generate a corrected image. In addition, the method includes initiating presentation of the corrected image on a display panel of the VST XR device.

A method includes identifying a passthrough transformation associated with a VST XR device. The VST XR device includes see-through cameras that are (i) offset from forward axes extending from expected locations of a user's eyes and / or (ii) rotated such that optical axes of the see-through cameras are angled relative to the forward axes. The method also includes obtaining images captured using the see-through cameras, applying the passthrough transformation to the images to generate transformed images, and displaying the transformed images on one or more display panels of the VST XR device. The passthrough transformation is based on (i) a first transformation between see-through camera viewpoints and viewpoint-matched virtual camera viewpoints and (ii) a second transformation that aligns principal points of the see-through cameras and principal points of the one or more display panels.

A method includes obtaining, using multiple imaging sensors of an electronic device, a left image frame and a right image frame forming a stereo pair of image frames. The method also includes identifying, using at least one processing device of the electronic device, extrinsic parameters associated with relative positions and orientations of the imaging sensors. The method further includes performing, using the at least one processing device, an online stereo rectification of the stereo pair of image frames based on the extrinsic parameters such that epipolar lines of the left and right image frames are horizontally aligned to generate a rectified stereo pair of image frames. In addition, the method includes rendering, using the at least one processing device, one or more images for display based on the rectified stereo pair of image frames.

A method includes: determining a first set of vertex adjustment values ​​for a distorted mesh at an extended reality (XR) device, and receiving image frame data of a scene captured at a first time in a first head pose using a perspective camera of the XR device. The method further includes: applying the first set of vertex adjustment values ​​for the distorted mesh to the image frame data to obtain intermediate image data, and predicting a second head pose at a second time after the first time. The method further includes: generating a second set of vertex adjustment values ​​for the distorted mesh based on the predicted second head pose, applying the second set of vertex adjustment values ​​for the distorted mesh to the intermediate image data to generate a rendered virtual frame, and displaying the rendered virtual frame by the XR device at a second time.

The present disclosure relates to an image display method and apparatus, and an electronic device, a mirror image display method and apparatus in an extended reality space, an electronic device, and a medium. The image display method includes: determining a first orientation of a mirror camera; determining a first position of the mirror camera based on a position of a first-person perspective camera, a preset distance, and the first orientation; and obtaining a first image based on the first position and the first orientation, and displaying the first image on a virtual mirror.

The present invention discloses a multi-perspective 3D target detection method and system, which belongs to the field of computer vision technology. The method is optimized based on the original three-dimensional target detection model PETR, introduces 3DRoPE to replace the original 3D position encoding method, and sets learnable parameters for 3D position information to enhance adaptability. In addition, by fusing the pose geometry information of multi-perspective cameras into the position embedding of each perspective image, the model's ability to handle complex spatial relationships is further improved. After training with the NuScenes dataset, this model can receive real-time image input from multiple cameras and output accurate 3D target detection results, significantly improving the perception accuracy and safety in autonomous driving environments.

A method includes generating a virtual view image and a virtual depth map based on an image captured using a see-through camera and a corresponding depth map. The virtual view image and the virtual depth map include holes for which image data or depth data cannot be determined. The method also includes searching one or more previous images to locate a region in at least one previous image that includes missing pixels in the holes. The method further includes at least partially filling the holes in the virtual view image and the virtual depth map with image data and depth data associated with the located region to generate a filled virtual view image and a filled virtual depth map. In addition, the method includes generating a virtual view to present on a display panel of a VST AR device using the filled virtual view image and the filled virtual depth map.