364 results about "Virtual camera" patented technology

There is provided an apparatus comprising a split rendering client that is configured for obtaining a predicted pose of a virtual camera at a first time and a time information indicative of a time at which a video frame is expected to be displayed on a display, providing rendering metadata to a split rendering server. The rendering metadata comprises the predicted pose and the time information. The split rendering client is further configured for receiving, from the split rendering server, a plurality of rendered and encoded frames and a render pose associated with each respective rendered and encoded frame of the plurality of rendered and encoded frames, obtaining a second predicted pose at a second time different than the first time, decoding at least two rendered and encoded frames of the plurality of the rendered and encoded frames to generate at least two decoded frames, based on the render pose associated with each respective decoded frame and the second predicted pose, selecting a primary decoded frame and one or more secondary decoded frames from the at least two decoded frames, composing a video frame to be displayed based on the primary decoded frame and at least one or more secondary decoded frames and causing the video frame that is composed to be displayed.

The invention discloses a stereoscopic vision optimization method and system for a naked-eye 3D large screen, and particularly relates to the technical field of naked-eye 3D vision optimizing.The method comprises the steps that environment illumination and audience positions are sensed in real time through multi-sensor fusion, and an environment light field model is established; glare crosstalk noise is predicted based on physical simulation, and self-adaptive suppression and compensation are carried out in combination with human eye visual sensitivity and image content features; a virtual camera is dynamically generated according to the real-time positions of the eyes of the audience, and a lightweight neural radiation field renderer is used for real-time re-rendering, so that motion parallax is realized; and finally, intelligently fusing the glare compensation layer and the perspective correction layer, coding and outputting to a screen. The system correspondingly comprises an environment perception module, a glare compensation module, a perspective rendering module and a fusion coding module. The naked-eye 3D large-screen display method effectively inhibits ambient light interference, improves the quality and immersion of a stereoscopic picture under different visual angles, and is suitable for naked-eye 3D large-screen display under outdoor and complex illumination environments.

A panoramic stitching method, a panoramic stitching apparatus, an electronic device, and a storage medium are provided. The panoramic stitching method comprises acquiring captured images from cameras in a same scene; obtaining a remapping lookup table and a fusion lookup table generated based on a stitching model; and mapping and fusing the captured images based on the remapping lookup table and the fusion lookup table to obtain a panoramic stitched output image. The stitching model comprises a hemisphere and a cylinder, and an origin of a virtual camera coordinate system associated with the cameras is located at a center of a bottom surface of the hemisphere. The bottom surface is where the hemisphere is in contact with the cylinder. This method effectively reduces camera parallax and alignment errors, significantly enhancing the quality of panoramic stitching, which is particularly well-suited for open horizontal scenes, delivering exceptional visual stitching results.

The invention provides a virtual camera projection method and system, electronic equipment and a medium, and belongs to the technical field of target detection, and the method comprises the steps: constructing a training data set according to sampling points in a three-dimensional space; modeling projection and back projection between the real camera view and the virtual camera view to obtain a two-stage multi-layer perceptron model; training a two-stage multi-layer perceptron model according to the training data set to obtain a trained model; and obtaining a pixel mapping table between the virtual camera view and the real camera view based on the trained model. According to the method, the reversible pixel mapping relation between the real camera and the virtual camera is constructed by adopting the two-stage multi-layer perceptron model, and the pictures under any group of camera parameters are uniformly converted to be under the virtual camera parameters, so that high-quality label data are efficiently generated through the automatic labeling model; the data utilization rate and the generalization ability of the automatic labeling model are remarkably improved, so that the model can directly process multi-source heterogeneous data, and the model adaptation cost is reduced.

The invention discloses a near-ground distance measurement method based on virtual large baseline four-eye vision, a medium and equipment, and the method comprises the steps: synchronously obtaining four images when a lifting appliance enters a near-ground operation range; based on the calibration relation between the cameras, the high-altitude images are respectively projected and transformed to the visual angles of the corresponding lifting appliance cameras and fused, two virtual camera images are generated, and therefore a virtual large base line far exceeding the physical distance is constructed; performing three-dimensional correction and cutting on the virtual image pair to obtain a row-aligned three-dimensional image pair; performing feature matching and triangulation on the virtual image pair to generate a sparse reference depth map; and inputting the stereo image pair and the sparse depth map into a pre-trained depth estimation neural network model together, outputting a dense depth map, and converting the dense depth map into a coordinate system taking the lifting appliance as an original point to obtain a vertical distance. According to the invention, the ultra-large baseline is virtually synthesized by using the existing camera, the remote distance measurement precision is improved, the cost is low, and the reliability is high.

The application provides an object recognition method and device, electronic equipment and computer storage medium. In the method, the virtual lens in the target application can call the camera of the terminal. Based on the calling operation, the zoom coefficient for zooming the camera of the terminal can be configured according to the focusing parameter information of the camera of the terminal. The zoom coefficient is a parameter for indicating the focusing setting of the camera with a macro function in the terminal. After the zoom coefficient for zooming the camera of the terminal is configured, the camera of the terminal is called to recognize the object to be recognized based on the zoom coefficient. In the actual process, the zoom coefficient for indicating the camera with a macro function in the terminal can realize the switching between the cameras, so that the camera with a macro function in the terminal can be called for focusing, and the object to be recognized can be clearly focused when the object to be recognized is recognized in the application.

The invention relates to the technical field of neuroendoscopy, in particular to a neuroendoscopy under-endoscope guiding method based on multiple intraoperative scenes, which comprises the following steps: performing three-dimensional graph reconstruction based on a plurality of three-dimensional medical images shot by medical imaging equipment, and establishing a three-dimensional reconstruction scene; determining a space conversion matrix of the tail end of the neuroendoscope and the three-dimensional reconstruction scene according to a shooting result of the optical tracking locator and the three-dimensional medical image, and determining a virtual camera according to the space conversion matrix; adjusting the virtual camera according to the moving mode of the neuroendoscope, and obtaining a corresponding virtual endoscope picture after the neuroendoscope is moved; determining the axial direction of the endoscope according to the spatial transformation matrix, and determining a virtual section image along the axial direction of the endoscope according to the axial direction of the endoscope and the three-dimensional reconstruction scene; according to the invention, dynamic and high-precision three-dimensional guidance and augmented reality display of the neuroendoscope can be realized.

The invention relates to a three-dimensional model and two-dimensional picture mapping method and device, a storage medium and electronic equipment. The method comprises the steps of obtaining virtual camera shooting parameters when a two-dimensional picture is generated through a virtual camera in a three-dimensional space; obtaining a projection matrix according to the camera external parameters and the camera internal parameters of the virtual camera in the projection mode, determining a direction vector coordinate system according to the camera external parameters, and storing the shooting parameters of the virtual camera, the projection matrix and the direction vector coordinate system; for each two-dimensional pixel point of the edited two-dimensional picture, converting a corresponding two-dimensional pixel coordinate into a projection coordinate, constructing a ray equation and calculating an intersection point of the ray equation and the surface of the three-dimensional model based on the virtual camera shooting parameter and the projection coordinate to obtain a three-dimensional coordinate point; and constructing a three-dimensional picture in the three-dimensional model according to each three-dimensional coordinate point, and mapping the labeling attribute of the two-dimensional pixel point in the edited two-dimensional picture to the three-dimensional coordinate point corresponding to the two-dimensional pixel point. The efficiency of cooperative communication can be improved.

Embodiments of the present disclosure provide a video generation method, a motion video generation method for a virtual object, a video editing method, a video generation model training method, and a video generation model-based information processing method. The video generation method comprises: acquiring video generation data, the video generation data comprising a virtual camera parameter, a reference action sequence of a target object, and an object image of the target object (202); inputting the video generation data into a video generation model to obtain a target video of the target object, the video generation model being obtained by performing parameter adjustment on a parameter encoding unit, a parameter cross-attention unit, and a temporal attention unit in a pre-trained generation model on the basis of a sample video, and a sample object image, sample action sequence and sample camera parameter corresponding to the sample video, and the pre-trained generation model being obtained by performing parameter adjustment on an object encoding unit, an action encoding unit, and a generation unit in an initial generation model, on the basis of the sample video (204). The virtual camera parameter is inputted into a model comprising the temporal attention unit and the parameter cross-attention unit, so that a video that is temporally stable and conforms to a camera motion trajectory is generated.

The invention provides a splicing and rendering method and system for multi-source video streams based on a digital twin scene, and relates to the technical field of digital twin video rendering. Multi-channel video streams at different spatial positions in the real world are acquired, and a mapping relation between pixel coordinates of the multi-channel video streams and three-dimensional spatial coordinates of a digital twin model is established; and processing the multiple paths of video streams frame by frame to generate target area data and target tracking information, and further identifying a shielded target and generating an enhanced video stream. And then projecting the enhanced video stream as a dynamic texture to a corresponding surface of a twin model, determining a video overlapping region fusion strategy, converting three-dimensional related information of a real target, determining a shielding relationship between virtual information and the real target, finally performing fusion rendering according to the shielding relationship and the fusion strategy, and outputting a panoramic rendering result from a visual angle of a virtual camera. Shielding compensation and accurate fusion of multi-source video streams in a digital twinning scene can be realized, and a real target panoramic rendering result is output.

The embodiment of the application discloses a kind of adjustment method, device, equipment and medium of vehicle head-up display image, the method comprises: determining adjacent time current vehicle coordinate system based on the pose conversion matrix of world coordinate system;According to the first position transformation matrix between the first camera coordinate system of world coordinate system and vehicle coordinate system of vehicle head-up display, it is corrected;Determine the visual transformation matrix of driver's eyes in vehicle coordinate system;According to the second position transformation matrix between the rendering space coordinate system of virtual image and the virtual camera coordinate system in rendering space, it is corrected;Based on the first position transformation matrix after correction and the second position transformation matrix after correction, the position of virtual image displayed by HUD and the position of target object in virtual image are adjusted.By using the above technical solution, the problem that the object in the virtual image displayed by HUD cannot be attached to the actual object when the attitude of the vehicle changes is solved.

A computer implemented method includes initiating, by at least one processor within a computing environment, operation of a virtual camera and an SFU; receiving, by the SFU, a plurality of audio streams from a plurality of remote devices; communicating, by the SFU, the plurality of audio streams to the virtual camera; receiving, by the virtual camera, the plurality of audio streams from the SFU; mixing, by the virtual camera, the plurality of audio streams into a single audio stream; communicating, by the virtual camera, the single audio stream to a physical image capture device; and receiving, by the virtual camera, an audiovisual stream from the physical image capture device.

The invention provides a three-dimensional effect rendering method and device and electronic equipment, and the method comprises the steps: carrying out the zooming processing of a target surface patch according to the model size of a target three-dimensional model and the camera orientation of a virtual camera, obtaining a zoomed target surface patch of which the size is matched with the model size, and keeping the zoomed target surface patch facing the virtual camera; stretching the scaled target surface patch according to the model shape of the target three-dimensional model to obtain a stretched target surface patch of which the shape is matched with the model shape; and moving the stretched target patch to the position of the target three-dimensional model in the virtual scene, deleting the target three-dimensional model in the virtual scene and rendering the stretched target patch to obtain a rendered target patch so as to simulate the three-dimensional effect of the target three-dimensional model through the rendered target patch. According to the method, the target surface patch is stretched in the camera space according to the visual angle, so that the surface patch fitting three-dimensional effect is realized, and a better three-dimensional effect is realized with extremely low rendering performance consumption.

An apparatus, method, and computer-readable medium for generating a view of an augmented reality environment includes: generating an augmented reality (AR) environment based at least in part on one or more audiovisual feeds of a set captured by one or more physical cameras and one or more three-dimensional graphics models of a virtual environment; tracking a current viewpoint location within the AR environment, the current viewpoint location corresponding to one or more of a physical location of a physical camera in the one or more physical cameras or a virtual location of a virtual camera in the virtual environment; and generating a view of the AR environment based at least in part on the current viewpoint location.

The application provides a real-time shooting system and method of digital animation. The real-time shooting system is constructed based on a motion capture system, a UE engine and a same-screen display application software. The real-time shooting system adopts the UE engine to generate a plurality of second virtual cameras with adjustable parameters to control corresponding entity virtual cameras (constructed based on a marker physical camera) in the motion capture system in real time, and transmit the shooting pictures to the same-screen display application software through the UE engine for real-time display. The real-time shooting system provided by the application can capture and display complete digital animation effects in real time, solve the delay caused by multi-role shooting and software adaptability in the digital animation shooting process, and facilitate timely adjustment of camera parameters and role resources according to the displayed shooting effects.

The application discloses a contour determination method and device of a BIM model, a terminal and a storage medium, and the method comprises the following steps: acquiring a BIM model to be calculated, and acquiring model parameters of the BIM model to be calculated; importing the BIM model to be calculated into a pre-set graphics engine; creating a virtual camera in the graphics engine; rendering the BIM model to be calculated by using the virtual camera in combination with the model parameters of the BIM model to be calculated, so as to obtain a binary image of the BIM model to be calculated; and performing edge calculation and vectorization processing on the binary image of the BIM model to be calculated, so as to obtain a two-dimensional contour of the BIM model to be calculated. According to the scheme, the contour of the BIM model is determined by using the image recognition method in the case that the volume of the BIM model is large, and the calculation difficulty can be reduced.

The embodiment of the invention provides a digital twinning method and device for a satellite optical system, and the method comprises the steps that a telemetering instruction is received by an interface layer in the satellite optical system, a virtual electric control box is a core logic layer in the satellite optical system, and a virtual camera is a unit model layer in the satellite optical system. The interface layer and the core logic layer receive the telemetering instruction and respond to the telemetering instruction with the unit model layer, so that a simulation result responding to the telemetering instruction in the satellite optical system is obtained. By adopting the layered architecture, the interface layer focuses on data interaction, the core logic layer is responsible for instruction processing and logic operation, and the unit model layer focuses on imaging simulation, so that the stability and reliability of the system are enhanced, and a stable and reliable virtual platform support is provided for testing and verification of the satellite optical system.

An electronic device includes a camera, a communication circuit, memory storing one or more computer programs, and at least one processor. The one or more computer programs include computer-executable instructions that, when executed by the one or more processors, cause the electronic device to obtain information on a second image comprising a plurality of visual objects and changed from a first image, transmit, to a server, the information on the second image, receive information on a three-dimensional (3D) model for a space comprising the plurality of external objects and information on a reference image, obtain a third image by removing, from the second image, at least one visual object among the plurality of visual objects, identify at least one feature point based on a comparison between the reference image and the third image, identify a pose of a virtual camera, and identify content superimposed on the first image.

The invention discloses a photoelectric pod damping control method and device based on a simulation environment, and relates to the technical field of simulation. The method comprises the following steps: constructing a three-dimensional simulation environment and each simulation period, based on a virtual camera imaging principle, obtaining a target miss distance, mapping the target miss distance into an angle tracking error of a pod control axis, independently designing a proportional-differential controller for the pod control axis, obtaining a continuous time control law, and converting the continuous time control law into discrete time for representation to obtain a discrete control quantity. Setting initial proportional gain and damping gain in a three-dimensional simulation environment, analyzing a time domain response curve of target miss distance based on discrete control quantity, iteratively adjusting parameters until preset performance indexes are met, enabling the time domain response of the miss distance to be optimal, and taking the corresponding proportional gain and damping gain at the moment as a parameter set for photoelectric pod control. A real object platform is replaced by a simulation environment, and the problems that a traditional damping parameter setting method is high in cost and low in efficiency are solved.

This invention relates to the field of 3D geometric modeling engine technology, specifically to a method for rapidly displaying Points of Interest (POI) information from a 3D real-world model. The invention first monitors the motion state of a virtual camera, reusing historical states during high-speed movement to avoid image jumps caused by aliasing. Second, it calculates the original occlusion distance of the POI and quantifies the physical depth difference between it and the scene. Then, by analyzing the local window depth distribution, it obtains discrete suppression weights to correct the original distance, adaptively distinguishing between solid and discrete structures, suppressing vegetation noise while maintaining high sensitivity to building edges. Finally, it constructs a state update cost based on the corrected distance and introduces hysteresis decision logic, setting a potential energy threshold for state switching, effectively reducing flickering in critical areas. This significantly improves the accuracy and stability of POI information display.

The invention discloses a special effect surface patch control method and device, equipment, a storage medium and a program product, and relates to the technical field of computers, and the method comprises the steps: obtaining a first distance between a target special effect surface patch and a virtual camera, a first view field angle corresponding to the virtual camera, and a reference height corresponding to the target special effect surface patch; in response to the change of the first view field angle, acquiring a changed second view field angle; based on the second view field angle and the reference height, determining a second distance between the target special effect patch and the virtual camera at the second view field angle; and controlling the movement of the target special effect patch based on the offset between the second distance and the first distance. By implementing the technical scheme of the invention, the position of the target special effect patch can be automatically adjusted when the visual field angle of the virtual camera changes so as to maintain the effect of filling the screen, the picture presentation effect is improved, and the manual adjustment workload is reduced.

A simulation data creation method, a preview image display method, and a simulation device that enable grasping of the contents of simulation before construction of a virtual space. The simulation data creation method includes the following steps: (a) causing a virtual robot disposed in a virtual work space to act based on an operation instruction by a user, and displaying a simulator image representing the action of the virtual robot in the virtual work space on a display device, the virtual work space being a virtual three-dimensional work space; (b) outputting a snapshot representing a two-dimensional image obtained by a virtual camera capturing a state in which a virtual three-dimensional object including the virtual robot is disposed in the virtual work space; and (c) saving, as simulation data, in a memory, an action program representing a target action of the virtual robot based on the operation instruction, layout data representing positions and postures of objects in the virtual work space, attribute data representing attributes of the objects, and the snapshot.

The application relates to the technical field of aviation assembly, in particular to a method for calculating the adjusting amount of a gasket in the process of adjusting the coaxiality of a helicopter tail transmission shaft, which comprises the following steps: constructing target key points and a target virtual camera of the helicopter tail transmission shaft to determine a first coaxiality error representation vector of the helicopter tail transmission shaft; obtaining target measurement data of the helicopter tail transmission shaft and equivalently mapping the target measurement data to the first coaxiality error representation vector to obtain a second coaxiality error representation vector meeting a preset format; and inputting the second coaxiality error representation vector into a pre-calibrated mathematical solution model to output the adjusting amount of each gasket of the helicopter tail transmission shaft. Thus, the problems that in the related art, only two single circular targets are measured, the coaxiality of the helicopter tail transmission shaft cannot be effectively observed, and the adjusting amount of the installed gasket is estimated only according to artificial experience, thereby increasing the artificial cost and reducing the accuracy of the gasket adjusting amount are solved.

The invention discloses a high-rise building curtain wall construction progress monitoring method and system based on a multi-view image, and belongs to the field of curtain wall construction progress monitoring, and the method comprises the following steps: S1, employing a network dome camera and a total station to obtain the multi-view image covering the full facade of a building and the real coordinates of ground control points; s2, generating a building appearance model by using a Poisson surface reconstruction algorithm based on the multi-view image and the real coordinates of the ground control points; s3, virtual camera parameters are configured based on the building appearance model, an image-building mapping relation is established, and a plurality of layered and blocked wall units are obtained; and S4, obtaining the construction progress and generating a visual report. By adopting the high-rise building curtain wall construction progress monitoring method and system based on the multi-view image, through the multi-view image fusion three-dimensional reconstruction and intelligent segmentation algorithm, the image distortion influence can be eliminated, and low-cost, efficient and accurate high-rise building curtain wall construction progress quantification and non-construction area positioning are realized.

The application discloses a near-ground ranging method based on virtual large baseline four-eye vision, a medium and equipment, the method synchronously acquires four images when a spreader enters a near-ground operation range; based on a calibration relationship between cameras, high-position images are respectively projected and transformed to a visual angle of a corresponding spreader camera and fused to generate two virtual camera images, so that a virtual large baseline far beyond a physical distance is constructed; virtual image pairs are subjected to stereo correction and cutting to obtain row-aligned stereo image pairs; feature matching and triangulation are performed on the virtual image pairs to generate a sparse reference depth map; the stereo image pairs and the sparse depth map are jointly input into a pre-trained depth estimation neural network model to output a dense depth map, and the vertical distance is obtained in a coordinate system with the spreader as an origin. The application utilizes existing cameras to virtually synthesize an ultra-large baseline, improves long-distance ranging precision, and is low in cost and high in reliability.

The application discloses an indoor parking lot vehicle positioning and reverse vehicle searching method based on visual semantic analysis, which comprises the following steps: dividing an image into a real physical surface and a mirror virtual image area, and calculating a ground reflection reference surface; establishing a virtual mirror camera symmetrical to the real main camera based on the reference surface, repairing the mirror virtual image feature to the effective observation of the virtual camera; introducing a symmetrical virtual observation factor in pose optimization, jointly optimizing the re-projection error of the real main camera and the virtual camera, converting the mirror interference into a geometric constraint for enhancing positioning, so as to solve the continuous vehicle pose, significantly improve the positioning accuracy and robustness in the mirror reflection scene such as the indoor parking lot, and realize reliable reverse vehicle searching application.

The game control method provided by the invention comprises the following steps: in response to a first sliding operation acting on a first area on a graphical user interface, generating a dynamic rocker control area at a first initial position of the first sliding operation; obtaining a sliding distance of the first sliding operation; and when the sliding distance is greater than the preset distance threshold, controlling the displacement of the controlled virtual object and the visual angle adjustment of the virtual camera at the same time according to the direction of the first sliding operation. The dynamic identification mechanism based on the sliding distance threshold value can intelligently distinguish the operation intention of the player and automatically switch between the short sliding mode and the long sliding mode, the technical problems that traditional double-rocker operation areas are overlapped and the mistaken touch rate is high are solved, and the interaction experience of single-hand operation in the mobile terminal 3D vertical screen game is improved.

The application discloses a virtual camera simulation data generation method and system based on a Gaussian point cloud model, relates to the technical field of three-dimensional point cloud processing and modeling, and comprises the following steps: after completing multi-source data interaction, a point cloud dataset is established according to the data interaction result; after completing space-time registration on the dataset, a fusion space dataset is constructed, and a Gaussian point cloud model is obtained through modeling; then, virtual camera parameters are configured, a frustum is constructed to identify invalid points, and point labels are established; center perception fusion is carried out by using the point labels to obtain a perspective simulation image; and after target perception feedback correction, simulation data is established. The application solves the technical problem that the prior art cannot accurately fuse multi-source information, the virtual camera simulation data generated has a large deviation from a real scene, and the data processing result is inaccurate, and achieves the technical effect that the virtual camera simulation data accurately fits the real scene and the accuracy of the data processing result is improved.

This application provides a method, apparatus, electronic device, and storage medium for calibrating the parameters of a virtual camera. The method includes: acquiring real image information of a preset real calibration location using a real camera; acquiring virtual image information of a preset virtual calibration location using a virtual camera, wherein the preset real calibration location and the preset virtual calibration location correspond one-to-one; parsing the real image information and the virtual image information respectively to obtain the real calibration pixel coordinates and the virtual calibration pixel coordinates; calculating the target parameters of the virtual camera based on the real calibration pixel coordinates and the virtual calibration pixel coordinates; and optimizing the virtual camera parameters based on the target parameters. Therefore, the calibrated virtual camera can directly output high-fidelity data, reducing the workload of repeated real-vehicle data acquisition and manual calibration, shortening the ADAS algorithm iteration cycle, reducing R&D costs, and enhancing the universality and reusability of the same set of simulation data across multiple platforms and vehicle models.