590 results about "Virtual camera" patented technology

The invention discloses a tunnel fracture recognition method based on three-dimensional live-action reconstruction and ordered re-acquisition of a virtual camera. The method specifically comprises the following steps: acquiring a multi-view rock fracture image; acquiring three-dimensional point cloud data of a tunnel wall surface according to the acquired rock mass fracture image, reconstructing a geometrical shape of the tunnel, mapping acquired texture information to a three-dimensional grid vertex through an image processing algorithm, and generating a three-dimensional tunnel model; setting parameters of the virtual camera, orderly collecting to form a virtual image, and enabling the position of the virtual camera to correspond to the real position of the model; and extracting information of the crack of the virtual image, and projecting the coordinates of the two-dimensional image back to the three-dimensional space through the projection matrix to obtain the space coordinates of the crack. A virtual image is generated by combining a texture mapping technology of a three-dimensional model, so that the limitation of a visual angle, illumination and space in actual image acquisition is overcome, and high-precision identification of tunnel cracks is realized.

The invention relates to the technical field of computer vision and virtual reality, and discloses a three-dimensional video fusion method based on camera self-calibration and projection texture mapping, and the method comprises the following steps: S1, obtaining video data, and collecting at least one frame of two-dimensional image in a to-be-fused video stream; optionally, the two-dimensional image is preprocessed; and S2, calibrating internal reference of the camera, detecting linear features in the two-dimensional image by using an image processing algorithm, estimating the position of a vanishing point by using a least square method or other optimization algorithms based on the detected linear segment, and calculating an internal reference matrix of the virtual camera according to the optimized vanishing point position. The perspective relation between the video image and the surface of the three-dimensional model is determined through the vanishing point detection technology, accurate fusion of the video image and the three-dimensional model is achieved, the sense of reality of a virtual scene is improved, and the tedious camera calibration process and the complex three-dimensional reconstruction process based on a calibration plate are avoided.

A shadow generation method for a three-dimensional model, the method comprising: acquiring light source information of a virtual light source and an observation parameter of a virtual camera, wherein the light source information is used for forming an illumination effect of the virtual light source in a virtual scene, and the observation parameter is used for forming a field-of-view range of the virtual camera in the virtual scene for a three-dimensional model; on the basis of the light source information and the observation parameter, generating a multi-angle contour map corresponding to the three-dimensional model, wherein the multi-angle contour map is used for representing an initial shadow; on the basis of the light source information and the observation parameter, projecting the multi-angle contour map onto a reference plane, so as to obtain a hard shadow image corresponding to the three-dimensional model; on the basis of the light source information and material attributes of the three-dimensional model, obtaining a shadow feature of the three-dimensional model; and inputting the hard shadow image and the shadow feature into a trained shadow prediction model based on a convolutional neural network, and on the basis of the hard shadow image and the shadow feature and by means of the shadow prediction model, outputting a soft shadow image corresponding to the three-dimensional model.

The invention relates to a three-dimensional reconstruction method, system and device of a supporting structure and a storage medium. The method comprises the following steps: acquiring two-dimensional medical images of a support structure implanted in a lumen of a living body at different visual angles; performing image segmentation on a target support in the two-dimensional medical image under each view angle to generate a two-dimensional target image corresponding to each view angle; performing parametric deformation on the original three-dimensional grid of the target support through a deformation model based on a graph neural network to generate a deformed three-dimensional grid; projecting the deformed three-dimensional grid according to a virtual camera pose parameter corresponding to the two-dimensional medical image under each view angle, and rendering to generate a two-dimensional rendered image corresponding to each view angle; and iteratively optimizing the three-dimensional grid according to an error between the two-dimensional rendering image corresponding to each view angle and the two-dimensional target image, and stopping iteration when a preset termination condition is met so as to output an optimal three-dimensional grid. According to the invention, the precision of three-dimensional reconstruction of the support structure can be improved.

There is provided an apparatus comprising a split rendering client that is configured for obtaining a predicted pose of a virtual camera at a first time and a time information indicative of a time at which a video frame is expected to be displayed on a display, providing rendering metadata to a split rendering server. The rendering metadata comprises the predicted pose and the time information. The split rendering client is further configured for receiving, from the split rendering server, a plurality of rendered and encoded frames and a render pose associated with each respective rendered and encoded frame of the plurality of rendered and encoded frames, obtaining a second predicted pose at a second time different than the first time, decoding at least two rendered and encoded frames of the plurality of the rendered and encoded frames to generate at least two decoded frames, based on the render pose associated with each respective decoded frame and the second predicted pose, selecting a primary decoded frame and one or more secondary decoded frames from the at least two decoded frames, composing a video frame to be displayed based on the primary decoded frame and at least one or more secondary decoded frames and causing the video frame that is composed to be displayed.

Computing an output image of a dynamic scene. A value of E is selected which is a parameter describing desired dynamic content of the scene in the output image. Using selected intrinsic camera parameters and a selected viewpoint, for individual pixels of the output image to be generated, the method computes a ray that goes from a virtual camera through the pixel into the dynamic scene. For individual ones of the rays, sample at least one point along the ray. For individual ones of the sampled points, a viewing direction being a direction of the corresponding ray, and E, query a machine learning model to produce colour and opacity values at the sampled point with the dynamic content of the scene as specified by E. For individual ones of the rays, apply a volume rendering method to the colour and opacity values computed along that ray, to produce a pixel value of the output image.

A virtual camera augmented reality (AR) system detects selection of a virtual camera added as an extension in an internet browser, generates a new canvas element for the virtual camera, and accesses video frames captured by a hardware camera coupled with a computing device. The virtual camera AR system detects selection of an AR option and applies the AR option to each video frame of at least a subset of the accessed video frames, at a predefined rate. The virtual camera AR system provides each video frame of at least the subset of video frames comprising the applied AR option to the new canvas element and causes display, of each video frame of at least the subset of video frames comprising the applied AR option in the new canvas element, on a user interface of the computing device.

An automatic visual data labeling framework for heterogeneous data types is described. An example method can include obtaining scan data and video data that each depict a scene including an object. The scan data can be generated by a scanner. The video data can be generated by a camera. The method can also include generating a virtual representation of the scene in a virtual environment based on the scan data. The virtual representation can include a virtual representation subset corresponding to the object. The virtual environment can be associated with a virtual camera. The method can also include applying label data to the virtual representation subset to create a labeled virtual representation subset corresponding to the object. The method can also include applying the labeled virtual representation subset to the object depicted in the video data based on a correlation of the scanner, the camera, and the virtual camera.

The invention discloses a stereoscopic vision optimization method and system for a naked-eye 3D large screen, and particularly relates to the technical field of naked-eye 3D vision optimizing.The method comprises the steps that environment illumination and audience positions are sensed in real time through multi-sensor fusion, and an environment light field model is established; glare crosstalk noise is predicted based on physical simulation, and self-adaptive suppression and compensation are carried out in combination with human eye visual sensitivity and image content features; a virtual camera is dynamically generated according to the real-time positions of the eyes of the audience, and a lightweight neural radiation field renderer is used for real-time re-rendering, so that motion parallax is realized; and finally, intelligently fusing the glare compensation layer and the perspective correction layer, coding and outputting to a screen. The system correspondingly comprises an environment perception module, a glare compensation module, a perspective rendering module and a fusion coding module. The naked-eye 3D large-screen display method effectively inhibits ambient light interference, improves the quality and immersion of a stereoscopic picture under different visual angles, and is suitable for naked-eye 3D large-screen display under outdoor and complex illumination environments.

To suppress jitter caused by a CG model that shows different motion from a camera image, and reduce discomfort of a user who views a composite image generated by synthesizing the CG model with the camera image.SOLUTION: An information processing apparatus includes: a determination means which determines position and attitude of a virtual camera; detection means which detects whether at least one of the position and attitude of the virtual camera determined by the determination means is stable or not; and acquisition means which acquires information on the stable position and attitude of the virtual camera, as reference position and attitude information. The determination means updates at least one of detected unstable position and attitude of the virtual camera, based on an attitude difference between the attitude of the virtual camera and the reference attitude.SELECTED DRAWING: Figure 8

A panoramic stitching method, a panoramic stitching apparatus, an electronic device, and a storage medium are provided. The panoramic stitching method comprises acquiring captured images from cameras in a same scene; obtaining a remapping lookup table and a fusion lookup table generated based on a stitching model; and mapping and fusing the captured images based on the remapping lookup table and the fusion lookup table to obtain a panoramic stitched output image. The stitching model comprises a hemisphere and a cylinder, and an origin of a virtual camera coordinate system associated with the cameras is located at a center of a bottom surface of the hemisphere. The bottom surface is where the hemisphere is in contact with the cylinder. This method effectively reduces camera parallax and alignment errors, significantly enhancing the quality of panoramic stitching, which is particularly well-suited for open horizontal scenes, delivering exceptional visual stitching results.

The invention provides a virtual camera projection method and system, electronic equipment and a medium, and belongs to the technical field of target detection, and the method comprises the steps: constructing a training data set according to sampling points in a three-dimensional space; modeling projection and back projection between the real camera view and the virtual camera view to obtain a two-stage multi-layer perceptron model; training a two-stage multi-layer perceptron model according to the training data set to obtain a trained model; and obtaining a pixel mapping table between the virtual camera view and the real camera view based on the trained model. According to the method, the reversible pixel mapping relation between the real camera and the virtual camera is constructed by adopting the two-stage multi-layer perceptron model, and the pictures under any group of camera parameters are uniformly converted to be under the virtual camera parameters, so that high-quality label data are efficiently generated through the automatic labeling model; the data utilization rate and the generalization ability of the automatic labeling model are remarkably improved, so that the model can directly process multi-source heterogeneous data, and the model adaptation cost is reduced.

The invention discloses a near-ground distance measurement method based on virtual large baseline four-eye vision, a medium and equipment, and the method comprises the steps: synchronously obtaining four images when a lifting appliance enters a near-ground operation range; based on the calibration relation between the cameras, the high-altitude images are respectively projected and transformed to the visual angles of the corresponding lifting appliance cameras and fused, two virtual camera images are generated, and therefore a virtual large base line far exceeding the physical distance is constructed; performing three-dimensional correction and cutting on the virtual image pair to obtain a row-aligned three-dimensional image pair; performing feature matching and triangulation on the virtual image pair to generate a sparse reference depth map; and inputting the stereo image pair and the sparse depth map into a pre-trained depth estimation neural network model together, outputting a dense depth map, and converting the dense depth map into a coordinate system taking the lifting appliance as an original point to obtain a vertical distance. According to the invention, the ultra-large baseline is virtually synthesized by using the existing camera, the remote distance measurement precision is improved, the cost is low, and the reliability is high.

The invention provides an automatic rendering method and system of a building block model, a medium and a product, and aims to solve the problems of low efficiency and high resource occupation in large-scale building block model rendering. The method comprises the following steps: dividing a model into logic blocks; generating a virtual camera array and a virtual light source matrix based on the blocks; performing multi-resolution rendering on each block according to the priority determined by the view angle area; and finally splicing a rendering result. According to the method, through blocking processing and differentiated multi-resolution rendering, the consumption of computing resources is effectively reduced, the rendering efficiency is remarkably improved, and meanwhile, the rendering image quality of the complex building block model is ensured.

An imaging processing apparatus includes one or more memories storing instructions, and one or more processors executing the instructions to acquire a first image captured by an imaging apparatus, generate a second image by performing processing of increasing the number of pixels with respect to the first image, and set information indicating a position and an orientation of a virtual camera corresponding to the second image.

The invention provides a method and equipment for generating a three-dimensional panoramic video, and a storage medium. The method comprises the following steps: S1, setting a binocular virtual camera at the position of an observer in an animation scene; s2, establishing a virtual ball surrounding the binocular virtual camera, and dividing a sphere skin into a plurality of view finding blocks; s3, running the animation frame by frame, and enabling the binocular virtual camera to poll and collect the image content in each view finding block; and S4, combining the puzzle blocks into a left monocular panoramic video and a right monocular panoramic video. Therefore, a panoramic video with a three-dimensional effect is produced.

The invention relates to a face and limb fusion capture method, a virtual live broadcast system, equipment and a medium, a rear end adopts a first virtual camera to collect and process a first image shot by the first camera to obtain a first video stream, and adopts a second virtual camera to collect and process a second image shot by the second camera to obtain a second video stream; the back end extracts face key point coordinate data from the first video stream, extracts skeleton node coordinate data from the second video stream, integrates the face key point coordinate data and the skeleton node coordinate data to obtain a single message body, and sends the single message body to the front end; the front end receives and analyzes the single message body to obtain a facial two-dimensional absolute pixel coordinate and a skeleton three-dimensional absolute pixel coordinate, and performs spatial registration on the facial two-dimensional absolute pixel coordinate and the skeleton three-dimensional absolute pixel coordinate to obtain a facial three-dimensional absolute pixel coordinate; the front end fuses the three-dimensional absolute pixel coordinates of the face and the three-dimensional absolute pixel coordinates of the skeleton so as to drive the virtual character to live; and the phenomena of key frame loss and skeleton-expression dislocation are improved.

The application provides an object recognition method and device, electronic equipment and computer storage medium. In the method, the virtual lens in the target application can call the camera of the terminal. Based on the calling operation, the zoom coefficient for zooming the camera of the terminal can be configured according to the focusing parameter information of the camera of the terminal. The zoom coefficient is a parameter for indicating the focusing setting of the camera with a macro function in the terminal. After the zoom coefficient for zooming the camera of the terminal is configured, the camera of the terminal is called to recognize the object to be recognized based on the zoom coefficient. In the actual process, the zoom coefficient for indicating the camera with a macro function in the terminal can realize the switching between the cameras, so that the camera with a macro function in the terminal can be called for focusing, and the object to be recognized can be clearly focused when the object to be recognized is recognized in the application.

The present disclosure relates to methods and systems for displaying composite image data. In one embodiment, a method includes: capturing real-time image data via a first camera of a mobile device, the real-time image data including an image of an object in a physical real-world environment; receiving depth data via a depth sensor of the mobile device, the depth data indicating a distance of the object from the camera in the physical real-world environment; receiving motion data via one or more motion sensors of the mobile device, the motion data indicating at least an orientation of the first camera in the physical real-world environment; generating a virtual camera transform based on the motion data, the camera transform being used to determine an orientation of the virtual camera in the virtual environment; and generating composite image data using the image data, a mask, and virtual background content selected based on the virtual camera orientation.

The invention relates to a shadow generation method and device for a three-dimensional model, computer equipment, a storage medium and a computer program product. The method comprises the following steps: acquiring light source information of a virtual light source and observation parameters of a virtual camera; according to the light source information and the observation parameters, a multi-view profile diagram corresponding to the three-dimensional model is generated, and the multi-view profile diagram is used for representing an initial shadow; projecting the multi-view profile diagram to a reference plane according to the light source information and the observation parameters to obtain a hard shadow image corresponding to the three-dimensional model; obtaining shadow features of the three-dimensional model according to the light source information and material attributes of the three-dimensional model; and inputting the hard shadow image and the shadow features into a trained shadow prediction model based on a convolutional neural network, and outputting a soft shadow image corresponding to the three-dimensional model through the shadow prediction model according to the hard shadow image and the shadow features. By adopting the method, the shadow of the three-dimensional model can be quickly generated.

Interactive methods and systems for converting three-dimensional (3-D) mixed-reality experiences into two-dimensional (2-D) representations are disclosed. A system comprises a 3-D-to-2-D decomposition engine that analyzes a data model of a 3-D environment to ascertain the positions, orientations, and activities of users, objects, and places within the environment. A 2-D user interface engine generates intuitive interface elements for display on conventional devices such as mobile phones, tablets, and laptops. The interface allows users to navigate the 3-D space efficiently through a series of interactive buttons representing conversation groups, individual users, and objects. Virtual cameras capture various perspectives within the 3-D environment, which are then processed to create a coherent 2-D representation for the user. This approach enables inclusive collaboration across a diverse range of devices and user capabilities, expanding the reach of mixed-reality technologies.

The invention relates to the technical field of neuroendoscopy, in particular to a neuroendoscopy under-endoscope guiding method based on multiple intraoperative scenes, which comprises the following steps: performing three-dimensional graph reconstruction based on a plurality of three-dimensional medical images shot by medical imaging equipment, and establishing a three-dimensional reconstruction scene; determining a space conversion matrix of the tail end of the neuroendoscope and the three-dimensional reconstruction scene according to a shooting result of the optical tracking locator and the three-dimensional medical image, and determining a virtual camera according to the space conversion matrix; adjusting the virtual camera according to the moving mode of the neuroendoscope, and obtaining a corresponding virtual endoscope picture after the neuroendoscope is moved; determining the axial direction of the endoscope according to the spatial transformation matrix, and determining a virtual section image along the axial direction of the endoscope according to the axial direction of the endoscope and the three-dimensional reconstruction scene; according to the invention, dynamic and high-precision three-dimensional guidance and augmented reality display of the neuroendoscope can be realized.

The invention belongs to the technical field of three-dimensional point cloud processing, and particularly provides a point cloud semantic segmentation method and system based on an adaptive virtual camera, and the method comprises the steps: adaptive virtual camera layout generation, soft visibility feature construction, a mixed distance neighbor search mechanism, a multi-modal camera embedding fusion mechanism, and sequential robust state space modeling. According to the method, the problems that in an existing point cloud semantic segmentation method, neighbor selection semantics are unrelated, the observation angle of a camera is fixed, visibility expression is rough, and a global feature fusion mode is simple are solved; specifically, a structure-adaptive virtual camera layout strategy is introduced, soft visibility features are constructed, a mixed distance measurement method fusing geometric and semantic information is designed, a multi-modal attention fusion mechanism and a semantic-guided sequential modeling mode are used, and a multi-modal attention fusion model is established. The local semantic consistency and the global structure perception capability in the point cloud segmentation process are effectively improved, and meanwhile, the calculation efficiency and the modeling stability are ensured.

The invention relates to a three-dimensional model and two-dimensional picture mapping method and device, a storage medium and electronic equipment. The method comprises the steps of obtaining virtual camera shooting parameters when a two-dimensional picture is generated through a virtual camera in a three-dimensional space; obtaining a projection matrix according to the camera external parameters and the camera internal parameters of the virtual camera in the projection mode, determining a direction vector coordinate system according to the camera external parameters, and storing the shooting parameters of the virtual camera, the projection matrix and the direction vector coordinate system; for each two-dimensional pixel point of the edited two-dimensional picture, converting a corresponding two-dimensional pixel coordinate into a projection coordinate, constructing a ray equation and calculating an intersection point of the ray equation and the surface of the three-dimensional model based on the virtual camera shooting parameter and the projection coordinate to obtain a three-dimensional coordinate point; and constructing a three-dimensional picture in the three-dimensional model according to each three-dimensional coordinate point, and mapping the labeling attribute of the two-dimensional pixel point in the edited two-dimensional picture to the three-dimensional coordinate point corresponding to the two-dimensional pixel point. The efficiency of cooperative communication can be improved.

The invention discloses a 6D pose estimation method and device for a target in a monocular RGB image, and relates to the technical field of pose estimation. According to the invention, based on the pre-constructed 3D Gaussian splash model of the target, multiple groups of virtual cameras surrounding the 3D Gaussian splash model of the target at different angles are formed, and multiple groups of virtual RGB images and corresponding depth images are generated through 3D Gaussian rendering based on the multiple groups of virtual cameras, so that 2D-2D feature matching is carried out on the multiple groups of virtual RGB images and the target RGB image. The pixel-level matching relation between the target RGB image and the depth image is established based on the corresponding depth image, so that matching from 2D to 3D is completed, the matching relation between the matching pixel of the target RGB image and the 3D point generated by the corresponding depth image is determined, the matching precision from 2D to 3D is improved with low cost, and the initial pose of the target can be solved and optimized based on the matching relation subsequently. And low cost is taken into account while the pose estimation precision is improved.

The invention relates to an AprilTag-based panoramic camera 3D calibration method and system and a medium, and the method comprises the steps: M1, obtaining the original images of a front vehicle-mounted fisheye camera, a rear vehicle-mounted fisheye camera, a left vehicle-mounted fisheye camera and a right vehicle-mounted fisheye camera, carrying out the fisheye distortion correction of each image, and obtaining the data information of a distortionless image; and M2, based on the data information of the distortionless image, detecting an AprilTag mark in each distortionless image, extracting a detected angular point coordinate as an image point set, querying a three-dimensional world coordinate corresponding to a predefined calibration plate physical size, constructing a matched three-dimensional point set, and obtaining the data information of the matched three-dimensional point set of the image. According to the method, flexible adjustment of the height and view parameters of the aerial view camera is achieved, repeated calibration is avoided, the parameters of the virtual camera are dynamically corrected by fusing the data of the vehicle attitude sensor, and view angle compensation under the vehicle body inclination state is achieved.

Embodiments of the present disclosure provide a video generation method, a motion video generation method for a virtual object, a video editing method, a video generation model training method, and a video generation model-based information processing method. The video generation method comprises: acquiring video generation data, the video generation data comprising a virtual camera parameter, a reference action sequence of a target object, and an object image of the target object (202); inputting the video generation data into a video generation model to obtain a target video of the target object, the video generation model being obtained by performing parameter adjustment on a parameter encoding unit, a parameter cross-attention unit, and a temporal attention unit in a pre-trained generation model on the basis of a sample video, and a sample object image, sample action sequence and sample camera parameter corresponding to the sample video, and the pre-trained generation model being obtained by performing parameter adjustment on an object encoding unit, an action encoding unit, and a generation unit in an initial generation model, on the basis of the sample video (204). The virtual camera parameter is inputted into a model comprising the temporal attention unit and the parameter cross-attention unit, so that a video that is temporally stable and conforms to a camera motion trajectory is generated.

The invention provides a splicing and rendering method and system for multi-source video streams based on a digital twin scene, and relates to the technical field of digital twin video rendering. Multi-channel video streams at different spatial positions in the real world are acquired, and a mapping relation between pixel coordinates of the multi-channel video streams and three-dimensional spatial coordinates of a digital twin model is established; and processing the multiple paths of video streams frame by frame to generate target area data and target tracking information, and further identifying a shielded target and generating an enhanced video stream. And then projecting the enhanced video stream as a dynamic texture to a corresponding surface of a twin model, determining a video overlapping region fusion strategy, converting three-dimensional related information of a real target, determining a shielding relationship between virtual information and the real target, finally performing fusion rendering according to the shielding relationship and the fusion strategy, and outputting a panoramic rendering result from a visual angle of a virtual camera. Shielding compensation and accurate fusion of multi-source video streams in a digital twinning scene can be realized, and a real target panoramic rendering result is output.

The embodiment of the application discloses a kind of adjustment method, device, equipment and medium of vehicle head-up display image, the method comprises: determining adjacent time current vehicle coordinate system based on the pose conversion matrix of world coordinate system;According to the first position transformation matrix between the first camera coordinate system of world coordinate system and vehicle coordinate system of vehicle head-up display, it is corrected;Determine the visual transformation matrix of driver's eyes in vehicle coordinate system;According to the second position transformation matrix between the rendering space coordinate system of virtual image and the virtual camera coordinate system in rendering space, it is corrected;Based on the first position transformation matrix after correction and the second position transformation matrix after correction, the position of virtual image displayed by HUD and the position of target object in virtual image are adjusted.By using the above technical solution, the problem that the object in the virtual image displayed by HUD cannot be attached to the actual object when the attitude of the vehicle changes is solved.

An example of an information processing apparatus performs, in a predetermined area in a virtual space, editing including at least one of selecting a placement object to be placed in the area, placing the placement object, and moving the placement object, based on an operation input. The information processing apparatus performs presentation upon completion, which includes at least one scene and displays, for each scene, an image of the area based on a virtual camera. In the scene, the information processing apparatus sets a gaze point of the virtual camera at any of a position of the placement object placed in the area, a predetermined position in the area, and a position of a character arranged in the area, and sets the virtual camera at a position at which the placement object placed in the area is not placed.