811 results about "Virtual camera" patented technology

The invention provides a cross-modal adaptive matching online external parameter calibration method and system for a laser radar and a camera, and belongs to the technical field of online external parameter calibration of the laser radar and the camera, initial external parameters are obtained by using a static calibration result, a laser radar point cloud is projected to a visual angle of a virtual camera, an LIP image aligned with a camera image is generated, and a calibration result is obtained. Performing cross-modal segmentation on the LIP image and the RGB image, extracting a semantic mask, generating a cross-modal feature point pair through a strategy, and performing preliminary external parameter calculation in combination with a PnP algorithm; global optimization is carried out through a multi-modal loss function, and an inter-frame smoothing and increment optimization strategy is introduced to ensure accurate estimation of external parameters. Finally, the optimized external parameter matrix is applied to projection and matching of subsequent frames, and can adapt to environment change and data dynamic update in real time. The method has high precision and real-time performance, overcomes the defects that off-line calibration depends on a specific calibration target and cannot cope with dynamic environment changes, and adapts to application requirements in high-speed motion and complex environments.

The invention discloses a tunnel fracture recognition method based on three-dimensional live-action reconstruction and ordered re-acquisition of a virtual camera. The method specifically comprises the following steps: acquiring a multi-view rock fracture image; acquiring three-dimensional point cloud data of a tunnel wall surface according to the acquired rock mass fracture image, reconstructing a geometrical shape of the tunnel, mapping acquired texture information to a three-dimensional grid vertex through an image processing algorithm, and generating a three-dimensional tunnel model; setting parameters of the virtual camera, orderly collecting to form a virtual image, and enabling the position of the virtual camera to correspond to the real position of the model; and extracting information of the crack of the virtual image, and projecting the coordinates of the two-dimensional image back to the three-dimensional space through the projection matrix to obtain the space coordinates of the crack. A virtual image is generated by combining a texture mapping technology of a three-dimensional model, so that the limitation of a visual angle, illumination and space in actual image acquisition is overcome, and high-precision identification of tunnel cracks is realized.

A computer system implements a 3D virtual environment (312) configured to be accessed by a plurality of client devices each having corresponding a user graphical representation (322) within the 3D virtual environment, which includes positions for the user graphical representations (322) arranged in a geometry (330) and a virtual camera (314) positioned within the 3D virtual environment. The virtual camera (314) moves on a predetermined path (332) that maintains a distance between the virtual camera (314) and the positions (322) arranged in the geometry for the user graphical representations. The computer system captures a video stream from the perspective of the virtual camera on the predetermined path. The video stream includes video of the user graphical representations in the positions arranged in the geometry. The predetermined path maintains a constant viewing orientation angle between the virtual camera and the positions arranged in the geometry, which may include predefined seating positions at a virtual conference table.

The invention relates to the technical field of computer vision and virtual reality, and discloses a three-dimensional video fusion method based on camera self-calibration and projection texture mapping, and the method comprises the following steps: S1, obtaining video data, and collecting at least one frame of two-dimensional image in a to-be-fused video stream; optionally, the two-dimensional image is preprocessed; and S2, calibrating internal reference of the camera, detecting linear features in the two-dimensional image by using an image processing algorithm, estimating the position of a vanishing point by using a least square method or other optimization algorithms based on the detected linear segment, and calculating an internal reference matrix of the virtual camera according to the optimized vanishing point position. The perspective relation between the video image and the surface of the three-dimensional model is determined through the vanishing point detection technology, accurate fusion of the video image and the three-dimensional model is achieved, the sense of reality of a virtual scene is improved, and the tedious camera calibration process and the complex three-dimensional reconstruction process based on a calibration plate are avoided.

A video having a wide field of view (e.g., spherical field of view) may be presented within a graphical user interface. The graphical user interface may include multiple punchouts of the wide field of view, with the number of punchouts set based on user interaction with a punchout-number element of the graphical user interface. Individual punchouts may be used as output of a virtual camera to provide views of differential spatial parts of the video. Presentation of different spatial parts of the video within different punchouts may be automatically synchronized based on the different spatial parts originating from a single video.

The invention provides a method and a system for generating deviation guide information in an approaching landing section of an aircraft and a readable storage medium, and the method comprises the steps: S1, real image collection and processing: capturing an outboard environment image through an airborne visual sensor, and extracting key feature points of a target landing airport runway; s2, virtual image generation: based on the airport three-dimensional high-precision map, generating an approach glide image under a visual angle of a virtual camera; s3, image feature registration: carrying out spatial alignment on the key feature points of the runway in the real image and the key feature points of the runway in the virtual image; the pixel deviation of the runway center point in the real image and the virtual image is calculated, glide guidance is provided for a pilot or is directly input into an automatic flight system.The method can be used for the aircraft in the approaching landing section, the enhanced outboard visual display is provided for the pilot in the cockpit based on the equivalent ILS deviation guidance method, and the visual guidance effect is improved. And providing approaching glide deviation guide information.

The invention discloses a point cloud 2D-3D fusion-based building indoor scene deep learning reconstruction method and system, and the method comprises the steps: directly generating a two-dimensional image from a point cloud, combining the two-dimensional image with three-dimensional point cloud data, and achieving the more accurate semantic classification of the point cloud through a deep learning model. The method comprises the following steps: firstly, generating two-dimensional images of multiple angles by using a virtual camera, and carrying out semantic segmentation on the images through a YOLO model; thirdly, the classification result of the YOLO model is combined with the point-by-point classification result of the point cloud through the PointNet series model, and the final semantic label of each point is updated based on the confidence degree; according to the method, the point cloud data with complex or sparse shapes can be effectively processed, the point cloud components with similar shapes are effectively distinguished, and the reconstruction precision and semantic segmentation effect of the point cloud data are improved. Meanwhile, the method can be applied to a building information model BIM, and support is provided for operation, maintenance and intelligent construction management of a building.

A method includes identifying a passthrough transformation associated with a VST XR device. The VST XR device includes see-through cameras that are (i) offset from forward axes extending from expected locations of a user's eyes and / or (ii) rotated such that optical axes of the see-through cameras are angled relative to the forward axes. The method also includes obtaining images captured using the see-through cameras, applying the passthrough transformation to the images to generate transformed images, and displaying the transformed images on one or more display panels of the VST XR device. The passthrough transformation is based on (i) a first transformation between see-through camera viewpoints and viewpoint-matched virtual camera viewpoints and (ii) a second transformation that aligns principal points of the see-through cameras and principal points of the one or more display panels.

A shadow generation method for a three-dimensional model, the method comprising: acquiring light source information of a virtual light source and an observation parameter of a virtual camera, wherein the light source information is used for forming an illumination effect of the virtual light source in a virtual scene, and the observation parameter is used for forming a field-of-view range of the virtual camera in the virtual scene for a three-dimensional model; on the basis of the light source information and the observation parameter, generating a multi-angle contour map corresponding to the three-dimensional model, wherein the multi-angle contour map is used for representing an initial shadow; on the basis of the light source information and the observation parameter, projecting the multi-angle contour map onto a reference plane, so as to obtain a hard shadow image corresponding to the three-dimensional model; on the basis of the light source information and material attributes of the three-dimensional model, obtaining a shadow feature of the three-dimensional model; and inputting the hard shadow image and the shadow feature into a trained shadow prediction model based on a convolutional neural network, and on the basis of the hard shadow image and the shadow feature and by means of the shadow prediction model, outputting a soft shadow image corresponding to the three-dimensional model.

The invention relates to a three-dimensional reconstruction method, system and device of a supporting structure and a storage medium. The method comprises the following steps: acquiring two-dimensional medical images of a support structure implanted in a lumen of a living body at different visual angles; performing image segmentation on a target support in the two-dimensional medical image under each view angle to generate a two-dimensional target image corresponding to each view angle; performing parametric deformation on the original three-dimensional grid of the target support through a deformation model based on a graph neural network to generate a deformed three-dimensional grid; projecting the deformed three-dimensional grid according to a virtual camera pose parameter corresponding to the two-dimensional medical image under each view angle, and rendering to generate a two-dimensional rendered image corresponding to each view angle; and iteratively optimizing the three-dimensional grid according to an error between the two-dimensional rendering image corresponding to each view angle and the two-dimensional target image, and stopping iteration when a preset termination condition is met so as to output an optimal three-dimensional grid. According to the invention, the precision of three-dimensional reconstruction of the support structure can be improved.

The invention discloses a multi-camera synchronous electronic image stabilization method and device based on virtual camera imaging, and the method comprises the steps: obtaining and splicing images captured by all cameras in a multi-camera cooperative imaging system, and constructing a virtual camera image; calculating the translation offset of the virtual camera according to the perspective transformation matrix between the adjacent cameras, and calculating the internal reference of the virtual camera; converting the pixel coordinate system into a to-be-stabilized virtual camera coordinate system according to the internal reference of the virtual camera; converting the compensation amount to a virtual camera coordinate system to be stabilized; calculating an image stabilization rotation matrix for converting the coordinate system of the virtual camera to be stabilized into the coordinate system of the stable virtual camera; and converting the to-be-stabilized image from the stable virtual camera coordinate system to the pixel coordinate system according to the image stabilization rotation matrix and the internal reference of the virtual camera to obtain a stabilized image. The method can be applied to multi-camera synchronous electronic image stabilization in a multi-camera collaborative imaging system, and high-quality anti-shake video real-time output is realized.

There is provided an apparatus comprising a split rendering client that is configured for obtaining a predicted pose of a virtual camera at a first time and a time information indicative of a time at which a video frame is expected to be displayed on a display, providing rendering metadata to a split rendering server. The rendering metadata comprises the predicted pose and the time information. The split rendering client is further configured for receiving, from the split rendering server, a plurality of rendered and encoded frames and a render pose associated with each respective rendered and encoded frame of the plurality of rendered and encoded frames, obtaining a second predicted pose at a second time different than the first time, decoding at least two rendered and encoded frames of the plurality of the rendered and encoded frames to generate at least two decoded frames, based on the render pose associated with each respective decoded frame and the second predicted pose, selecting a primary decoded frame and one or more secondary decoded frames from the at least two decoded frames, composing a video frame to be displayed based on the primary decoded frame and at least one or more secondary decoded frames and causing the video frame that is composed to be displayed.

The invention relates to the technical field of artificial intelligence, particularly provides a three-dimensional reconstruction method of a vehicle driving scene, electronic equipment and a medium, and aims to solve the problem of improving aerial view rendering quality in three-dimensional reconstruction of the vehicle driving scene. The method provided by the invention comprises the following steps: rendering by adopting a three-dimensional Gaussian sputtering model of a vehicle driving scene to obtain a rendered image under multiple view angles, and obtaining a reconstructed driving scene according to the rendered image; wherein a virtual camera with a bird's-eye view angle and a second camera parameter of the virtual camera at a moment t are set, and the moment t is any one of a plurality of moments; according to the vehicle-mounted camera data, generating a bird's-eye view under the visual angle of the virtual camera at the t moment, and according to the bird's-eye view and the second camera parameter, obtaining virtual camera data at the t moment; and training according to all the vehicle-mounted camera data and the virtual camera data of the virtual camera at the multiple moments to obtain a three-dimensional Gaussian sputtering model. Through the method, the rendering quality of the aerial view in three-dimensional reconstruction of the vehicle driving scene can be effectively improved.

Embodiments of the present invention provide a video generation method and apparatus, a medium, and a device. The method comprises: receiving a video description text, and generating a target text on the basis of the video description text, wherein the target text comprises at least one sub-text, each sub-text corresponds to one virtual scene, and the sub-text comprises a virtual character in the virtual scene and an interactive text corresponding to the virtual character; on the basis of the interactive text in the sub-text, generating a control instruction corresponding to the virtual character in the sub-text, and on the basis of the control instruction corresponding to each sub-text, generating a control instruction sequence corresponding to the target text; and controlling the virtual character to execute a control instruction in the control instruction sequence, and by means of a virtual camera, recording frames obtained by executing the control instruction by the virtual character to obtain a target video corresponding to the video description text.

Computing an output image of a dynamic scene. A value of E is selected which is a parameter describing desired dynamic content of the scene in the output image. Using selected intrinsic camera parameters and a selected viewpoint, for individual pixels of the output image to be generated, the method computes a ray that goes from a virtual camera through the pixel into the dynamic scene. For individual ones of the rays, sample at least one point along the ray. For individual ones of the sampled points, a viewing direction being a direction of the corresponding ray, and E, query a machine learning model to produce colour and opacity values at the sampled point with the dynamic content of the scene as specified by E. For individual ones of the rays, apply a volume rendering method to the colour and opacity values computed along that ray, to produce a pixel value of the output image.

The invention provides a point cloud rigid body registration method and device, electronic equipment and a readable storage medium, and the method comprises the steps: photographing a plurality of point cloud images for each to-be-registered point cloud data set through a virtual camera at different poses, and inputting the point cloud images into a preset three-dimensional visual processing tool for pose estimation; based on a virtual camera pose of a virtual camera during shooting of the point cloud image corresponding to each to-be-registered point cloud data set and shooting pose estimation of the point cloud image corresponding to each to-be-registered point cloud data set, a conversion relation between a coordinate space of each to-be-registered point cloud data set and a target coordinate space is obtained; and the point cloud data in each point cloud data set to be registered can be easily converted into the same coordinate space based on the conversion relation, so that the registration among the point cloud data sets to be registered is realized. The scheme of the invention has the characteristics of small calculation amount, high registration efficiency and high registration precision.

A virtual camera augmented reality (AR) system detects selection of a virtual camera added as an extension in an internet browser, generates a new canvas element for the virtual camera, and accesses video frames captured by a hardware camera coupled with a computing device. The virtual camera AR system detects selection of an AR option and applies the AR option to each video frame of at least a subset of the accessed video frames, at a predefined rate. The virtual camera AR system provides each video frame of at least the subset of video frames comprising the applied AR option to the new canvas element and causes display, of each video frame of at least the subset of video frames comprising the applied AR option in the new canvas element, on a user interface of the computing device.

An automatic visual data labeling framework for heterogeneous data types is described. An example method can include obtaining scan data and video data that each depict a scene including an object. The scan data can be generated by a scanner. The video data can be generated by a camera. The method can also include generating a virtual representation of the scene in a virtual environment based on the scan data. The virtual representation can include a virtual representation subset corresponding to the object. The virtual environment can be associated with a virtual camera. The method can also include applying label data to the virtual representation subset to create a labeled virtual representation subset corresponding to the object. The method can also include applying the labeled virtual representation subset to the object depicted in the video data based on a correlation of the scanner, the camera, and the virtual camera.

The invention discloses a stereoscopic vision optimization method and system for a naked-eye 3D large screen, and particularly relates to the technical field of naked-eye 3D vision optimizing.The method comprises the steps that environment illumination and audience positions are sensed in real time through multi-sensor fusion, and an environment light field model is established; glare crosstalk noise is predicted based on physical simulation, and self-adaptive suppression and compensation are carried out in combination with human eye visual sensitivity and image content features; a virtual camera is dynamically generated according to the real-time positions of the eyes of the audience, and a lightweight neural radiation field renderer is used for real-time re-rendering, so that motion parallax is realized; and finally, intelligently fusing the glare compensation layer and the perspective correction layer, coding and outputting to a screen. The system correspondingly comprises an environment perception module, a glare compensation module, a perspective rendering module and a fusion coding module. The naked-eye 3D large-screen display method effectively inhibits ambient light interference, improves the quality and immersion of a stereoscopic picture under different visual angles, and is suitable for naked-eye 3D large-screen display under outdoor and complex illumination environments.

Technologies for transitioning between two-dimensional (2D) and three-dimensional (3D) display views for video conferencing are described herein. Video conferencing applications can have multiple display views for a user participating in a video conference. In certain situations a user may want to transition from a 2D display view of the video conference to a more immersive 3D display view. These transitions can be visually jarring and create an uncomfortable user experience. The transition from a 2D display view to a 3D display view can be improved by executing the transition to a 3D display view by manipulating visual properties of a virtual camera that is employed to generate the display views.

The invention discloses a camera point location determination method, device and equipment based on a three-dimensional map. The method comprises the following steps: determining an actual image collected by an actual camera deployed in an actual scene, and determining a plurality of first virtual images collected by a plurality of virtual cameras corresponding to the actual camera; performing feature extraction on the actual image and each first virtual image to obtain key point information of each image; determining an image matching result between the actual image and each first virtual image according to a matching result of the key point information of the actual image and each first virtual image; and determining a second virtual image from the first virtual images according to an image matching result, and determining a point location determination result of the actual camera according to the second virtual image. According to the method, the accuracy and the consistency of point location determination are remarkably improved, so that the possibility of manual marking errors is reduced; and moreover, the point position marking speed is increased, and real-time marking is realized.

To suppress jitter caused by a CG model that shows different motion from a camera image, and reduce discomfort of a user who views a composite image generated by synthesizing the CG model with the camera image.SOLUTION: An information processing apparatus includes: a determination means which determines position and attitude of a virtual camera; detection means which detects whether at least one of the position and attitude of the virtual camera determined by the determination means is stable or not; and acquisition means which acquires information on the stable position and attitude of the virtual camera, as reference position and attitude information. The determination means updates at least one of detected unstable position and attitude of the virtual camera, based on an attitude difference between the attitude of the virtual camera and the reference attitude.SELECTED DRAWING: Figure 8

To enhance user experience.SOLUTION: A detection unit detects the orientation of a user P1 on the basis of signals output from a sensor. A display processing unit changes the orientation of a virtual camera 61 arranged in a virtual space so as to follow the orientation of the user P1 detected by the detection unit, and causes a display to display the image of the virtual space obtained by the virtual camera 61. An audio processing unit causes a plurality of speakers 30 to output audio in the virtual space, including a predetermined specific sound A5, in such a manner that the specific sound A5 is output from a specific direction toward the user P1 in accordance with the orientation of the user P1 detected by the detection unit.SELECTED DRAWING: Figure 7

The invention discloses a virtual scene editing processing method and device, electronic equipment and a storage medium, a game editing interface can be displayed, and the game editing interface comprises a to-be-edited virtual scene shot by a virtual camera and a plurality of virtual objects in the virtual scene; in response to a selection operation on the target virtual object, switching a rotation center of the virtual camera to a first reference point of the target virtual object; determining a target distance between the virtual camera and the target virtual object; and in response to the first view angle adjustment event, controlling rotation of the virtual camera based on the first reference point and the target distance to adjust the view angle, so that a virtual scene shot by the virtual camera under the adjusted view angle contains the target virtual object. On the basis, the creator can edit the virtual scene only by executing the operations of selecting the virtual object and rotating the virtual camera, and does not need to execute redundant operation processes such as moving the virtual camera, rotating the visual angle and positioning the virtual object, so that the editing efficiency of the virtual scene is improved.

A panoramic stitching method, a panoramic stitching apparatus, an electronic device, and a storage medium are provided. The panoramic stitching method comprises acquiring captured images from cameras in a same scene; obtaining a remapping lookup table and a fusion lookup table generated based on a stitching model; and mapping and fusing the captured images based on the remapping lookup table and the fusion lookup table to obtain a panoramic stitched output image. The stitching model comprises a hemisphere and a cylinder, and an origin of a virtual camera coordinate system associated with the cameras is located at a center of a bottom surface of the hemisphere. The bottom surface is where the hemisphere is in contact with the cylinder. This method effectively reduces camera parallax and alignment errors, significantly enhancing the quality of panoramic stitching, which is particularly well-suited for open horizontal scenes, delivering exceptional visual stitching results.

The invention provides a virtual camera projection method and system, electronic equipment and a medium, and belongs to the technical field of target detection, and the method comprises the steps: constructing a training data set according to sampling points in a three-dimensional space; modeling projection and back projection between the real camera view and the virtual camera view to obtain a two-stage multi-layer perceptron model; training a two-stage multi-layer perceptron model according to the training data set to obtain a trained model; and obtaining a pixel mapping table between the virtual camera view and the real camera view based on the trained model. According to the method, the reversible pixel mapping relation between the real camera and the virtual camera is constructed by adopting the two-stage multi-layer perceptron model, and the pictures under any group of camera parameters are uniformly converted to be under the virtual camera parameters, so that high-quality label data are efficiently generated through the automatic labeling model; the data utilization rate and the generalization ability of the automatic labeling model are remarkably improved, so that the model can directly process multi-source heterogeneous data, and the model adaptation cost is reduced.

A method, system, and computer-readable storage medium are disclosed for displaying virtual elements (e.g., AR content) in a physical environment by a client device using a virtual camera pose that is different from the pose of the physical camera used to capture images of the physical environment. The client device uses a three-dimensional (3D) map (e.g., a topographical mesh) of the physical environment to determine a pose (a position and orientation) of the camera of the client device. The 3D map can include geometry, colors, textures, or any other suitable information describing the physical environment.

The invention discloses a near-ground distance measurement method based on virtual large baseline four-eye vision, a medium and equipment, and the method comprises the steps: synchronously obtaining four images when a lifting appliance enters a near-ground operation range; based on the calibration relation between the cameras, the high-altitude images are respectively projected and transformed to the visual angles of the corresponding lifting appliance cameras and fused, two virtual camera images are generated, and therefore a virtual large base line far exceeding the physical distance is constructed; performing three-dimensional correction and cutting on the virtual image pair to obtain a row-aligned three-dimensional image pair; performing feature matching and triangulation on the virtual image pair to generate a sparse reference depth map; and inputting the stereo image pair and the sparse depth map into a pre-trained depth estimation neural network model together, outputting a dense depth map, and converting the dense depth map into a coordinate system taking the lifting appliance as an original point to obtain a vertical distance. According to the invention, the ultra-large baseline is virtually synthesized by using the existing camera, the remote distance measurement precision is improved, the cost is low, and the reliability is high.

The invention provides an automatic rendering method and system of a building block model, a medium and a product, and aims to solve the problems of low efficiency and high resource occupation in large-scale building block model rendering. The method comprises the following steps: dividing a model into logic blocks; generating a virtual camera array and a virtual light source matrix based on the blocks; performing multi-resolution rendering on each block according to the priority determined by the view angle area; and finally splicing a rendering result. According to the method, through blocking processing and differentiated multi-resolution rendering, the consumption of computing resources is effectively reduced, the rendering efficiency is remarkably improved, and meanwhile, the rendering image quality of the complex building block model is ensured.

The invention provides a stacked material volume identification and generation method fusing single view 3D reconstruction and BIM calibration, and relates to the technical field of stacked material volume generation, and the method employs a low-cost monocular camera to collect images, and generates a three-dimensional model of stacked materials by means of an advanced deep learning model and a three-dimensional reconstruction algorithm. Through scale calibration with a virtual camera in the BIM model, the problem of scale uncertainty which is difficult to overcome in traditional single-view reconstruction is solved, so that the real volume of the stacked material can be accurately calculated. In addition, through automatic target detection, three-dimensional coordinate calculation and volume calibration processes, the measurement efficiency is greatly improved, and manual intervention is reduced. The method is not only suitable for volume measurement of static stacked materials, but also can be expanded to real-time monitoring of dynamic construction scenes, and provides powerful technical support for construction management.