1032 results about "Virtual camera" patented technology

The invention provides a cross-modal adaptive matching online external parameter calibration method and system for a laser radar and a camera, and belongs to the technical field of online external parameter calibration of the laser radar and the camera, initial external parameters are obtained by using a static calibration result, a laser radar point cloud is projected to a visual angle of a virtual camera, an LIP image aligned with a camera image is generated, and a calibration result is obtained. Performing cross-modal segmentation on the LIP image and the RGB image, extracting a semantic mask, generating a cross-modal feature point pair through a strategy, and performing preliminary external parameter calculation in combination with a PnP algorithm; global optimization is carried out through a multi-modal loss function, and an inter-frame smoothing and increment optimization strategy is introduced to ensure accurate estimation of external parameters. Finally, the optimized external parameter matrix is applied to projection and matching of subsequent frames, and can adapt to environment change and data dynamic update in real time. The method has high precision and real-time performance, overcomes the defects that off-line calibration depends on a specific calibration target and cannot cope with dynamic environment changes, and adapts to application requirements in high-speed motion and complex environments.

The invention discloses a tunnel fracture recognition method based on three-dimensional live-action reconstruction and ordered re-acquisition of a virtual camera. The method specifically comprises the following steps: acquiring a multi-view rock fracture image; acquiring three-dimensional point cloud data of a tunnel wall surface according to the acquired rock mass fracture image, reconstructing a geometrical shape of the tunnel, mapping acquired texture information to a three-dimensional grid vertex through an image processing algorithm, and generating a three-dimensional tunnel model; setting parameters of the virtual camera, orderly collecting to form a virtual image, and enabling the position of the virtual camera to correspond to the real position of the model; and extracting information of the crack of the virtual image, and projecting the coordinates of the two-dimensional image back to the three-dimensional space through the projection matrix to obtain the space coordinates of the crack. A virtual image is generated by combining a texture mapping technology of a three-dimensional model, so that the limitation of a visual angle, illumination and space in actual image acquisition is overcome, and high-precision identification of tunnel cracks is realized.

A computer system implements a 3D virtual environment (312) configured to be accessed by a plurality of client devices each having corresponding a user graphical representation (322) within the 3D virtual environment, which includes positions for the user graphical representations (322) arranged in a geometry (330) and a virtual camera (314) positioned within the 3D virtual environment. The virtual camera (314) moves on a predetermined path (332) that maintains a distance between the virtual camera (314) and the positions (322) arranged in the geometry for the user graphical representations. The computer system captures a video stream from the perspective of the virtual camera on the predetermined path. The video stream includes video of the user graphical representations in the positions arranged in the geometry. The predetermined path maintains a constant viewing orientation angle between the virtual camera and the positions arranged in the geometry, which may include predefined seating positions at a virtual conference table.

The invention provides a video processing method, a recording and broadcasting system, electronic equipment and a computer readable storage medium, and relates to the technical field of video processing, the method comprises the following steps: obtaining a panoramic classroom video and video editing information, the video editing information comprising related description information of a target video expected to be generated through the panoramic classroom video by a user; generating a mirror moving parameter and a shooting parameter of the virtual camera according to the video editing information; determining a first interested picture which accords with the description of the video editing information from the panoramic classroom video according to the mirror operation parameters; performing image reconstruction processing on the first interested picture according to the mirror operation parameter and the shooting parameter to obtain a second interested picture; and generating a target video according to the second interested picture. According to the method and the device, the teaching video which has the mirror moving effect and meets the requirements of the user is quickly generated under the conditions that manpower is not consumed and a mirror moving technology is not needed.

The invention relates to the technical field of computer vision and virtual reality, and discloses a three-dimensional video fusion method based on camera self-calibration and projection texture mapping, and the method comprises the following steps: S1, obtaining video data, and collecting at least one frame of two-dimensional image in a to-be-fused video stream; optionally, the two-dimensional image is preprocessed; and S2, calibrating internal reference of the camera, detecting linear features in the two-dimensional image by using an image processing algorithm, estimating the position of a vanishing point by using a least square method or other optimization algorithms based on the detected linear segment, and calculating an internal reference matrix of the virtual camera according to the optimized vanishing point position. The perspective relation between the video image and the surface of the three-dimensional model is determined through the vanishing point detection technology, accurate fusion of the video image and the three-dimensional model is achieved, the sense of reality of a virtual scene is improved, and the tedious camera calibration process and the complex three-dimensional reconstruction process based on a calibration plate are avoided.

A video having a wide field of view (e.g., spherical field of view) may be presented within a graphical user interface. The graphical user interface may include multiple punchouts of the wide field of view, with the number of punchouts set based on user interaction with a punchout-number element of the graphical user interface. Individual punchouts may be used as output of a virtual camera to provide views of differential spatial parts of the video. Presentation of different spatial parts of the video within different punchouts may be automatically synchronized based on the different spatial parts originating from a single video.

The invention provides a method and a system for generating deviation guide information in an approaching landing section of an aircraft and a readable storage medium, and the method comprises the steps: S1, real image collection and processing: capturing an outboard environment image through an airborne visual sensor, and extracting key feature points of a target landing airport runway; s2, virtual image generation: based on the airport three-dimensional high-precision map, generating an approach glide image under a visual angle of a virtual camera; s3, image feature registration: carrying out spatial alignment on the key feature points of the runway in the real image and the key feature points of the runway in the virtual image; the pixel deviation of the runway center point in the real image and the virtual image is calculated, glide guidance is provided for a pilot or is directly input into an automatic flight system.The method can be used for the aircraft in the approaching landing section, the enhanced outboard visual display is provided for the pilot in the cockpit based on the equivalent ILS deviation guidance method, and the visual guidance effect is improved. And providing approaching glide deviation guide information.

The invention discloses a point cloud 2D-3D fusion-based building indoor scene deep learning reconstruction method and system, and the method comprises the steps: directly generating a two-dimensional image from a point cloud, combining the two-dimensional image with three-dimensional point cloud data, and achieving the more accurate semantic classification of the point cloud through a deep learning model. The method comprises the following steps: firstly, generating two-dimensional images of multiple angles by using a virtual camera, and carrying out semantic segmentation on the images through a YOLO model; thirdly, the classification result of the YOLO model is combined with the point-by-point classification result of the point cloud through the PointNet series model, and the final semantic label of each point is updated based on the confidence degree; according to the method, the point cloud data with complex or sparse shapes can be effectively processed, the point cloud components with similar shapes are effectively distinguished, and the reconstruction precision and semantic segmentation effect of the point cloud data are improved. Meanwhile, the method can be applied to a building information model BIM, and support is provided for operation, maintenance and intelligent construction management of a building.

A method includes identifying a passthrough transformation associated with a VST XR device. The VST XR device includes see-through cameras that are (i) offset from forward axes extending from expected locations of a user's eyes and / or (ii) rotated such that optical axes of the see-through cameras are angled relative to the forward axes. The method also includes obtaining images captured using the see-through cameras, applying the passthrough transformation to the images to generate transformed images, and displaying the transformed images on one or more display panels of the VST XR device. The passthrough transformation is based on (i) a first transformation between see-through camera viewpoints and viewpoint-matched virtual camera viewpoints and (ii) a second transformation that aligns principal points of the see-through cameras and principal points of the one or more display panels.

A shadow generation method for a three-dimensional model, the method comprising: acquiring light source information of a virtual light source and an observation parameter of a virtual camera, wherein the light source information is used for forming an illumination effect of the virtual light source in a virtual scene, and the observation parameter is used for forming a field-of-view range of the virtual camera in the virtual scene for a three-dimensional model; on the basis of the light source information and the observation parameter, generating a multi-angle contour map corresponding to the three-dimensional model, wherein the multi-angle contour map is used for representing an initial shadow; on the basis of the light source information and the observation parameter, projecting the multi-angle contour map onto a reference plane, so as to obtain a hard shadow image corresponding to the three-dimensional model; on the basis of the light source information and material attributes of the three-dimensional model, obtaining a shadow feature of the three-dimensional model; and inputting the hard shadow image and the shadow feature into a trained shadow prediction model based on a convolutional neural network, and on the basis of the hard shadow image and the shadow feature and by means of the shadow prediction model, outputting a soft shadow image corresponding to the three-dimensional model.

The invention provides a virtual training and evaluation method and device for intelligent focusing of an operating microscope, and relates to the technical field of intelligent control of operating microscopes, and the method comprises the steps: constructing a virtual operation scene based on real operation video data; obtaining a current virtual microscopic texture image which is collected by the virtual camera at the current position and aims at the virtual operation target; determining a region of interest in the current virtual microscopic texture image, and calculating the definition of the region of interest; based on the definition of the region of interest, controlling the virtual camera to perform focusing motion to obtain a focusing distance after the position of the virtual camera is updated; and calculating performance index data based on the real focusing distance and the updated focusing distance so as to evaluate the focusing algorithm of the operating microscope based on the performance index data. According to the method, organic unification among virtual operation scene depth, microscope motion control and image quality evaluation is realized, and a safe and reliable quantitative evaluation means is provided for development of an intelligent focusing algorithm of the operation microscope.

To compute an image of a dynamic 3D scene comprising a 3D object, a description of a deformation of the 3D object is received, the description comprising a cage of primitive 3D elements and associated animation data from a physics engine or an articulated object model. For a pixel of the image the method computes a ray from a virtual camera through the pixel into the cage animated according to the animation data and computes a plurality of samples on the ray. Each sample is a 3D position and view direction in one of the 3D elements. The method computes a transformation of the samples into a canonical cage. For each transformed sample, the method queries a learnt radiance field parameterization of the 3D scene to obtain a color value and an opacity value. A volume rendering method is applied to the color and opacity values producing a pixel value of the image.

The invention relates to a vehicle in-the-loop test method based on visual augmented reality. Firstly, a positioning and motion information acquisition system of a real vehicle is established, and high-precision motion information of the vehicle can be obtained; then image information and motion information of a vehicle-mounted camera are synchronously collected, and the purpose is to obtain camera output and vehicle motion information under the same timestamp; secondly, calculating the pose of the camera at each moment according to the internal participation initial external parameter information of the camera and the synchronously acquired motion information; synchronously inputting the calculated pose into simulation software, and obtaining virtual camera output consistent with the real camera in time and space; taking a really acquired camera image as a background, taking a traffic participant provided by a virtual simulation camera as a foreground, and obtaining fused image information of an augmented reality view angle through an image fusion algorithm; and finally, the fused image is input to the sensing layer, a sensing result is settled and then input to the decision control layer, a decision instruction is output to an execution mechanism of the vehicle, the vehicle motion information is changed and then is acquired by the high-precision information acquisition system, and thus a round of information circulation is completed.

The invention provides an aircraft simulation training method and system based on virtual-real combination, and relates to the technical field of aircraft simulation training. Accurate fusion of the actual optoelectronic device and the virtual scene is realized. Firstly, introduction of geographic information data provides a real scene basis for training; secondly, distortion correction of the two-dimensional picture ensures the accuracy of live-action imaging; and finally, by creating a virtual camera which is the same as the aircraft in position and has the same field angle, the system can always keep an output picture in an actual video range, and strictly follows the first view angle of a driver. According to the virtual-real combination mode based on position synchronization and visual angle matching, the real imaging characteristics of actual optoelectronic equipment are reserved, accurate superposition of the virtual targets is achieved, the real visual effect and the accurate space relation of training images are kept all the time, and therefore the vivid training experience is provided while the training cost is reduced.

A virtual environment (e.g., a game environment) is navigable by a user providing user input. The virtual environment is subject to one or more lighting effects from one or more virtual environment light sources therein. A view of the virtual environment is generated in connection with a virtual camera, with that view including at least a portion of a virtual object. The virtual object is defined in connection with an object model. Generation of the view includes: calculating color contributions for sections into which an image obtained from the virtual camera is partitioned; creating a composite image by blending (e.g., additively blending) the calculated color contributions onto the object model using an object mask, the object mask defining different effects to be applied to different parts of the virtual object; and integrating the created composite image in the view of the virtual environment.

The invention relates to a three-dimensional reconstruction method, system and device of a supporting structure and a storage medium. The method comprises the following steps: acquiring two-dimensional medical images of a support structure implanted in a lumen of a living body at different visual angles; performing image segmentation on a target support in the two-dimensional medical image under each view angle to generate a two-dimensional target image corresponding to each view angle; performing parametric deformation on the original three-dimensional grid of the target support through a deformation model based on a graph neural network to generate a deformed three-dimensional grid; projecting the deformed three-dimensional grid according to a virtual camera pose parameter corresponding to the two-dimensional medical image under each view angle, and rendering to generate a two-dimensional rendered image corresponding to each view angle; and iteratively optimizing the three-dimensional grid according to an error between the two-dimensional rendering image corresponding to each view angle and the two-dimensional target image, and stopping iteration when a preset termination condition is met so as to output an optimal three-dimensional grid. According to the invention, the precision of three-dimensional reconstruction of the support structure can be improved.

Depending on the positional relationship between the element configuring a 3D model and the camera viewpoint, it is not possible to color the fragment generated by UV development, and as a result of that, a texture image of low image quality including color voids is obtained. In the UV development, the fragment is generated by setting camera parameters of the virtual camera.

The invention discloses a multi-camera synchronous electronic image stabilization method and device based on virtual camera imaging, and the method comprises the steps: obtaining and splicing images captured by all cameras in a multi-camera cooperative imaging system, and constructing a virtual camera image; calculating the translation offset of the virtual camera according to the perspective transformation matrix between the adjacent cameras, and calculating the internal reference of the virtual camera; converting the pixel coordinate system into a to-be-stabilized virtual camera coordinate system according to the internal reference of the virtual camera; converting the compensation amount to a virtual camera coordinate system to be stabilized; calculating an image stabilization rotation matrix for converting the coordinate system of the virtual camera to be stabilized into the coordinate system of the stable virtual camera; and converting the to-be-stabilized image from the stable virtual camera coordinate system to the pixel coordinate system according to the image stabilization rotation matrix and the internal reference of the virtual camera to obtain a stabilized image. The method can be applied to multi-camera synchronous electronic image stabilization in a multi-camera collaborative imaging system, and high-quality anti-shake video real-time output is realized.

The embodiment of the invention discloses a vehicle-mounted head-up display image adjusting method and device, equipment and a medium. The method comprises the following steps: determining an attitude transformation matrix of a current vehicle coordinate system based on a world coordinate system at adjacent moments; correcting a first position transformation matrix between a vehicle-mounted first camera coordinate system and a vehicle coordinate system under a world coordinate system according to the attitude transformation matrix; determining a visual transformation matrix of the eyes of the driver under a vehicle coordinate system; correcting a second position transformation matrix between a rendering space coordinate system of the virtual image and a virtual camera coordinate system in the rendering space according to the visual transformation matrix; and adjusting the position of the virtual image displayed by the HUD and the position of the target object in the virtual image based on the corrected first position transformation matrix and the corrected second position transformation matrix. By adopting the technical scheme, the problem that the object in the virtual image displayed by the HUD cannot be attached to the actual object when the posture of the vehicle changes is solved.

There is provided an apparatus comprising a split rendering client that is configured for obtaining a predicted pose of a virtual camera at a first time and a time information indicative of a time at which a video frame is expected to be displayed on a display, providing rendering metadata to a split rendering server. The rendering metadata comprises the predicted pose and the time information. The split rendering client is further configured for receiving, from the split rendering server, a plurality of rendered and encoded frames and a render pose associated with each respective rendered and encoded frame of the plurality of rendered and encoded frames, obtaining a second predicted pose at a second time different than the first time, decoding at least two rendered and encoded frames of the plurality of the rendered and encoded frames to generate at least two decoded frames, based on the render pose associated with each respective decoded frame and the second predicted pose, selecting a primary decoded frame and one or more secondary decoded frames from the at least two decoded frames, composing a video frame to be displayed based on the primary decoded frame and at least one or more secondary decoded frames and causing the video frame that is composed to be displayed.

The invention relates to the technical field of aviation assembly, in particular to a method for calculating the gasket adjustment amount in the coaxiality adjustment process of a helicopter tail transmission shaft, and the method comprises the steps: constructing a target key point and a target virtual camera of the helicopter tail transmission shaft, so as to determine a first coaxiality error representation vector of the helicopter tail transmission shaft; obtaining target measurement data of the helicopter tail transmission shaft, and equivalently mapping the target measurement data to the first coaxiality error representation vector to obtain a second coaxiality error representation vector meeting a preset format; and inputting the second coaxiality error representation vector into a pre-calibrated mathematical solution model to output the adjustment amount of each gasket of the helicopter tail transmission shaft. Therefore, the problems that in the prior art, only two sets of single circular targets are measured, the coaxiality of the helicopter tail transmission shaft cannot be effectively observed, the adjustment amount of the installation gasket is estimated only according to manual experience, the labor cost is increased, and the accuracy of the adjustment amount of the gasket is reduced are solved.

Systems and methods are described for imparting dynamic, non-linear and realistic movement and look and feel to a player's / virtual character's first-person limbs and hand-held object model, and procedurally animating a first-person virtual camera such that it simulates the movement of a camera handheld by the player / virtual character. To impart chaotic or random motion to the first-person limbs and hand-held object model a first module defines and implements first and second two dimensional mass-spring-damper systems, each of which is linked to the player's / virtual vector's view vector. Procedural animation or rotational shake is implemented by a second module by applying a coherent noise function to each of the six axes of the first-person virtual camera.

The invention relates to the technical field of artificial intelligence, particularly provides a three-dimensional reconstruction method of a vehicle driving scene, electronic equipment and a medium, and aims to solve the problem of improving aerial view rendering quality in three-dimensional reconstruction of the vehicle driving scene. The method provided by the invention comprises the following steps: rendering by adopting a three-dimensional Gaussian sputtering model of a vehicle driving scene to obtain a rendered image under multiple view angles, and obtaining a reconstructed driving scene according to the rendered image; wherein a virtual camera with a bird's-eye view angle and a second camera parameter of the virtual camera at a moment t are set, and the moment t is any one of a plurality of moments; according to the vehicle-mounted camera data, generating a bird's-eye view under the visual angle of the virtual camera at the t moment, and according to the bird's-eye view and the second camera parameter, obtaining virtual camera data at the t moment; and training according to all the vehicle-mounted camera data and the virtual camera data of the virtual camera at the multiple moments to obtain a three-dimensional Gaussian sputtering model. Through the method, the rendering quality of the aerial view in three-dimensional reconstruction of the vehicle driving scene can be effectively improved.

Embodiments of the present invention provide a video generation method and apparatus, a medium, and a device. The method comprises: receiving a video description text, and generating a target text on the basis of the video description text, wherein the target text comprises at least one sub-text, each sub-text corresponds to one virtual scene, and the sub-text comprises a virtual character in the virtual scene and an interactive text corresponding to the virtual character; on the basis of the interactive text in the sub-text, generating a control instruction corresponding to the virtual character in the sub-text, and on the basis of the control instruction corresponding to each sub-text, generating a control instruction sequence corresponding to the target text; and controlling the virtual character to execute a control instruction in the control instruction sequence, and by means of a virtual camera, recording frames obtained by executing the control instruction by the virtual character to obtain a target video corresponding to the video description text.

Computing an output image of a dynamic scene. A value of E is selected which is a parameter describing desired dynamic content of the scene in the output image. Using selected intrinsic camera parameters and a selected viewpoint, for individual pixels of the output image to be generated, the method computes a ray that goes from a virtual camera through the pixel into the dynamic scene. For individual ones of the rays, sample at least one point along the ray. For individual ones of the sampled points, a viewing direction being a direction of the corresponding ray, and E, query a machine learning model to produce colour and opacity values at the sampled point with the dynamic content of the scene as specified by E. For individual ones of the rays, apply a volume rendering method to the colour and opacity values computed along that ray, to produce a pixel value of the output image.

The invention provides a point cloud rigid body registration method and device, electronic equipment and a readable storage medium, and the method comprises the steps: photographing a plurality of point cloud images for each to-be-registered point cloud data set through a virtual camera at different poses, and inputting the point cloud images into a preset three-dimensional visual processing tool for pose estimation; based on a virtual camera pose of a virtual camera during shooting of the point cloud image corresponding to each to-be-registered point cloud data set and shooting pose estimation of the point cloud image corresponding to each to-be-registered point cloud data set, a conversion relation between a coordinate space of each to-be-registered point cloud data set and a target coordinate space is obtained; and the point cloud data in each point cloud data set to be registered can be easily converted into the same coordinate space based on the conversion relation, so that the registration among the point cloud data sets to be registered is realized. The scheme of the invention has the characteristics of small calculation amount, high registration efficiency and high registration precision.

Example systems and methods for filming are disclosed. In an example, prop spatial data for a prop device and digital asset spatial data for a virtual asset can be received. The digital asset's spatial data can be updated based on the prop data, adjusting its position, movement, and orientation in a virtual space. In some examples, a scene model, a virtual representation of a scene, can be created using depth and color data. A miniaturized diorama of the scene model can be generated. A virtual camera can be configured to provide a perspective view of the diorama, visualized on an output device. In some examples, the scene model can include virtual points corresponding to scene locations, creating a waypoint scene model. Augmented video data can incorporate these virtual points based on the waypoint scene model. The waypoint scene model can also be miniaturized to provide the diorama.

The invention provides a game interaction method, a game interaction device, a computer storage medium and electronic equipment, and relates to the technical field of games. According to the method, a graphical user interface provided by a terminal device at least comprises a part of a game scene captured by a virtual camera and an attack control, the game scene comprises a controlled virtual character controlled by the terminal device, and the method comprises the following steps: in response to a first touch operation for the attack control, controlling the controlled virtual character to execute an attack action; in response to a second touch operation for the attack control, controlling the controlled virtual character to execute an attack action; after the controlled virtual character is controlled to execute an attack action or in the process of executing the attack action, a touch point responding to the second touch operation moves from the first position to a second position in the graphical user interface, and the virtual camera is adjusted to update the game scene displayed on the graphical user interface. The operation complexity can be reduced, so that the operation burden of players is reduced, and the game experience of the players is improved.

The invention provides a rendering display method, a rendering display device, electronic equipment and a storage medium. The rendering display method comprises the steps that area information of a target terrain area on a virtual earth surface is recorded in a Mask channel of a virtual texture RVT during operation, the area information comprises crop type information, and different crop type information corresponds to different combinations of crop types and crop growth states; and in the rendering stage, obtaining the region information from the Mask channel, if the distance between a target terrain region corresponding to the region information and the virtual camera is greater than or equal to a preset distance threshold value, obtaining a first map corresponding to the crop type and the crop growth state based on the crop type information, and generating a crop effect in the target terrain region based on the first map. According to the rendering display method provided by the embodiment of the invention, the rendering performance of real-time updating change of the target terrain area along with the change of the crop growth state is realized on the basis of the RVT during the long-range view.

A virtual camera augmented reality (AR) system detects selection of a virtual camera added as an extension in an internet browser, generates a new canvas element for the virtual camera, and accesses video frames captured by a hardware camera coupled with a computing device. The virtual camera AR system detects selection of an AR option and applies the AR option to each video frame of at least a subset of the accessed video frames, at a predefined rate. The virtual camera AR system provides each video frame of at least the subset of video frames comprising the applied AR option to the new canvas element and causes display, of each video frame of at least the subset of video frames comprising the applied AR option in the new canvas element, on a user interface of the computing device.