176 results about "World space" patented technology

Aspects of the present disclosure are directed to providing assisted scene capture for an artificial reality (XR) environment. Some implementations can obtain an XR space model corresponding to a real-world space, including walls, the ceiling, and the floor, using an XR system. Some implementations can then scan the real-world space and display a mesh corresponding to the scanned area overlaid onto a view of the real-world space. Some implementations can perform post-processing on the displayed mesh, which can include A) collapsing variations in the mesh corresponding to the walls, ceiling, or floor onto the XR space model; B) clipping the mesh to the XR space model for open doorways, windows, etc.; C) simplifying the mesh corresponding to the walls, ceiling, or floor; D) determining that the mesh is complete by identifying that a threshold percentage of the XR space model is covered by the mesh; or any combination thereof.

Techniques are disclosed for generating photorealistic images of objects, such as heads, from multiple viewpoints. In some embodiments, a morphable radiance field (MoRF) model that generates images of heads includes an identity model that maps an identifier (ID) code associated with a head into two codes: a deformation ID code encoding a geometric deformation from a canonical head geometry, and a canonical ID code encoding a canonical appearance within a shape-normalized space. The MoRF model also includes a deformation field model that maps a world space position to a shape-normalized space position based on the deformation ID code. Further, the MoRF model includes a canonical neural radiance field (NeRF) model that includes a density multi-layer perceptron (MLP) branch, a diffuse MLP branch, and a specular MLP branch that output densities, diffuse colors, and specular colors, respectively. The MoRF model can be used to render images of heads from various viewpoints.

Precision agriculture methods and systems where drone images of an agricultural field are captured by a UAV and analyzed to generate information regarding the agricultural field. Items of interest are identified in the images, the pixel-space locations of the items of interest are determined, and the world-space locations of the items of interest are then determined using the pixel-space locations. The transformation from pixel-space location to world-space location occurs without transforming the images or processing transformed images.

Provided in the present application are a target tracking method and apparatus based on image space positioning, and a device. The method comprises: acquiring first position coordinates of a plurality of target detection boxes at the current moment; on the basis of the first position coordinates, predicting second position coordinates of a candidate box of each target detection box at the next moment; on the basis of the second position coordinates, and third position coordinates of each target detection box at the next moment, calculating multi-dimensional similarities between each target detection box and a plurality of candidate boxes, wherein each multi-dimensional similarity comprises a multi-dimensional similarity determined on the basis of a dynamic threshold value of a centroid distance; and using a preset matching algorithm to output a target tracking result at the next moment according to the multi-dimensional similarities. The positions of targets in a world space are taken into consideration, thereby effectively solving the problem of positioning being inaccurate caused by single position information of a two-dimensional image; and a difference in the centroid distance between a distant target and a nearby target during consecutive imaging can be used, thereby improving the target positioning accuracy of a plurality of targets that overlap with each other.

This application relates to a shadow rendering method performed by a computer device. The method includes: obtaining a world space location of a pixel point of a screen space in a world space in which a virtual scene is located; determining, from a plurality of light sources in the virtual scene, a target light source whose illumination area comprises the world space location of the pixel point; obtaining a world space location of the target light source in the world space; performing ray tracing based on the world space location of the target light source and the world space location of the pixel point, to obtain light source access information of the target light source for the pixel point; and performing shadow rendering on the virtual scene based on the light source access information.

The invention provides a flight environment virtual simulation restoration method and system based on flight data, relates to the technical field of virtual reality and flight simulation, and solves the problem of how to accurately and smoothly restore real flight data to a virtual simulation environment. The method comprises the following steps: acquiring real flight data, converting the real flight data into a data format which can be identified by a UE rendering engine, analyzing and extracting key flight parameters, and smoothing process data corresponding to the key flight parameters, so that the process data smoothly changes when the flight environment of an aircraft is virtually simulated and updated; according to the smoothed process data, updating the aircraft pose in the UE virtual environment of the UE rendering engine to realize the reduction of the flight path; meanwhile, a world space UI is created, and instrument information corresponding to the flight path is updated and displayed in the world space UI. According to the invention, the flight path and instrument display under the simulation condition are close to the real flight condition, and the authenticity and immersion of flight training are effectively improved.

The invention provides a texture mapping method, a texture mapping device, a computer storage medium and electronic equipment, and relates to the technical field of computer graphic processing. The method comprises the following steps: acquiring a world space coordinate and a world space normal of a vertex of a target model; performing texture projection on the world space coordinates on a plurality of world coordinate axes to obtain target texture coordinates of the vertexes of the target model on the world coordinate axes; determining a texture weight value based on the included angles between the world space normal and the plurality of world coordinate axes; and carrying out texture sampling according to the target texture coordinate to obtain an initial map sampling result, and carrying out mixed sampling on the initial map sampling result and the texture weight value on the corresponding world coordinate axis to carry out texture mapping on the to-be-processed model based on the target map sampling result. The method can get rid of dependence on a model topological structure and UV expansion of a three-dimensional model, and realizes a more real, natural, accurate and flexible texture fitting effect without deformation stretching.

An apparatus, method and computer program is described comprising: obtaining a virtual audio signal (comprising virtual audio samples) from one or more virtual sound signals of a virtual environment for presentation to a user; processing the virtual audio samples based on first reverberation parameters to generate a first reverberation signal (based on a geometry of a real-world space) for samples up to a first threshold time following the respective virtual audio sample being obtained; processing the virtual audio samples based on second reverberation parameters to generate a second reverberation signal (based on a geometry of the virtual environment) for samples beyond a second threshold time following the respective virtual audio sample being obtained; and generating an audio output for presentation to the user based on a sum of the first and second reverberation signals.

In accordance with one aspect of the inventive concepts, provided is an autonomous mobile robot (AMR), comprising: at least one processor in communication with at least one computer memory device; at least one 3D sensor configured to collect 3D sensor data of an object; and a fixed scale image processing module configured to receive the 3D point cloud data; transform the 3D point cloud data into at least one fixed scale (FS) image; detect, identify, and localize the object in image space of the at least one FS image; select a plugin associated with an object type of the object; and apply the plugin for 6D (degrees of freedom) pose estimation of the object in real-world space to localize the object. In various embodiments, the object can be any type of forkable object, but in other embodiments the object need not be a forkable object.

The invention relates to an underground engineering three-dimensional reconstruction and grid extraction method based on 3D Gaussian splashing, and the method comprises the following steps: S1, obtaining a static scene image: photographing a plurality of static scene images from different angles and heights of an underground engineering through photographing equipment or an unmanned plane; s2, sparse point cloud generation: processing the image set collected in the step S1; s3, initializing to generate 3D Gaussian: taking the sparse point clouds obtained in the step S2 as input, dividing the point clouds into a plurality of regions, and generating one or more 3D Gaussian models in each region based on the world space position mean value and distribution characteristics of the point clouds; s4, optimizing a 3D Gaussian parameter; s5, extracting a grid; and S6, model lightweight: in order to reduce the storage demand of the model and accelerate the rendering speed, performing lightweight processing on the model. According to the method, efficient generation and real-time rendering of the three-dimensional model of the underground engineering can be realized while the reconstruction quality is maintained, and digital support is provided for design, construction and management of the underground engineering.

Systems, apparatuses, and methods for rendering content based on control points of a camera device are disclosed. In an example, a marker is attached to a camera device, and its pose is tracked over time. An offset between the marker and the control point is determined based on a camera model of the camera device. The attitude of the tracked marker may be translated and / or rotated according to the offset to estimate the attitude of the control point. Rendering of the content is adjusted over time based on the estimated attitude of the control point. When content is rendered in a real-world space (e.g., on a display component located therein), a camera device may capture the content as well as other objects in the real-world space and generate a video stream thereof.

The invention discloses automatic boundary creation and relocation. Aspects of the present disclosure relate to instant boundary creation for virtual reality (VR) experiences. An artificial reality (XR) system may scan and collect visual feature data of a user's real-world environment in the background while rendering an augmented reality (AR) environment or a mixed reality (MR) environment. When the XR system detects an intent to enter a VR mode (e.g., by launching a VR application), the XR system may generate a recommendation to a boundary of a real-world space, which may include a type of interaction mode (e.g., a movable mode or a stationary mode). Based on the user's response to the recommendation, the XR system may prompt the user to further scan the real-world space and / or manually adjust the boundaries while continuing to scan and collect visual feature data in the background. Alternatively or additionally, some embodiments may automatically relocate the real-world space based on the generated boundaries.

In various examples, one or more deep neural networks (DNNs) are executed to regress on control points of a curve, and the control points may be used to perform a curve fitting operation—e.g., Bezier curve fitting—to identify landmark locations and geometries in an environment. The outputs of the DNN(s) may thus indicate the two-dimensional (2D) image-space and / or three-dimensional (3D) world-space control point locations, and post-processing techniques—such as clustering and temporal smoothing—may be executed to determine landmark locations and poses with precision and in real-time. As a result, reconstructed curves corresponding to the landmarks—e.g., lane line, road boundary line, crosswalk, pole, text, etc.—may be used by a vehicle to perform one or more operations for navigating an environment.

In various examples, a machine learning model—such as a deep neural network (DNN)—may be trained to use image data and / or other sensor data as inputs to generate two-dimensional or three-dimensional trajectory points in world space, a vehicle orientation, and / or a vehicle state. For example, sensor data that represents orientation, steering information, and / or speed of a vehicle may be collected and used to automatically generate a trajectory for use as ground truth data for training the DNN. Once deployed, the trajectory points, the vehicle orientation, and / or the vehicle state may be used by a control component (e.g., a vehicle controller) for controlling the vehicle through a physical environment. For example, the control component may use these outputs of the DNN to determine a control profile (e.g., steering, decelerating, and / or accelerating) specific to the vehicle for controlling the vehicle through the physical environment.

A 3D neural radiance field (NeRF) is used to generate an impulse response (IR) characterization, which can then be input to a convolutional reverberation engine to create an audio experience that reflects the in-game world on a 2.0 stereo speaker system. The NeRF recreates a background geometry and the impulse response of a virtual 3D space generated using NeRF is input to the convolutional reverberation engine to transform game sounds / music to appear as though they are occurring inside the 3D space of the game. The same may be done for the player's real-world space in which the virtual IR and real IR are blended together in real-time, and the real-life player tracked as he moves around the room to create audio as it would sound were the player moving through the virtual space while adjusting for the acoustics of the real space.

Systems, apparatus, and methods of rendering content based on a control point of a camera device are disclosed. In an example, a marker is attached to the camera device and its pose is tracked over time. Based on a camera model of the camera device, an offset between the marker and the control point is determined. The tracked pose of the marker can be translated and / or rotated according to the offset to estimate a pose of the control point. The rendering of the content is adjusted over time based on the estimated poses of the control point. Upon presentation of the content in a real-world space (e.g., on a display assembly located therein), the camera device can capture the content along with other objects of the real-world space and generate a video stream thereof.

The invention discloses a vehicle tracking and re-identification system in an expressway scene, and the method comprises the steps: building a homography matrix to construct the mapping of a camera space coordinate and a real space coordinate, and then carrying out the pre-screening of a confidence coefficient + Laplacian variance pre-screening module; a set of end-to-end assembly line is formed by combining a hierarchical matching and self-adaptive histogram filtering matching module, a motion constraint and Kalman filtering module and Desection-Tracking-ReID Pipeline, so that the re-recognition accuracy and the track connectivity are improved. According to the method, systematic optimization is carried out for special challenges of a highway closed monitoring scene, the accuracy of vehicle re-identification and the connectivity and smoothness of the track are remarkably improved, and reliable technical support is provided for intelligent traffic monitoring.

Provided are an augmented reality device for providing an augmented reality service matched to a real-world space, and an operating method thereof. The augmented reality device may recognize a plurality of objects in a real-world space from an image obtained through a camera, recognize a relative position relationship including positions and directions between the plurality of recognized objects, and display, at a preset position, a virtual object representing information related to the plurality of recognized objects, based on information about a prestored relative position relationship.

Provided are an augmented reality device for providing an augmented reality service for controlling an object in a real world space, and an operating method for the same. The method may include recognizing a first plane comprising a wall and a second plane comprising a floor from a spatial image based on a photograph of the real world space; generating a three-dimensional (3D) model of the real world space by extending the wall and extending the floor along the respective planes and performing 3D in-painting on an area of the extended wall and the extended floor that is hidden by an obstructive object; segmenting an object selected by a user input from the spatial image based on two-dimensional (2D) segmentation; and segmenting the object on the spatial image from the real world space based on a 3D model or 3D position information of the object using 3D segmentation.

Aspects of the present disclosure relate to more accurate and quicker localization of an artificial reality (XR) system in a real-world space (e.g., a room). If a user enters a room and localization fails, the system can locate a corner that was designated in a previous localization. The corner could have been manually selected by the user or could have been automatically recommended by the XR system. In some implementations, the user or system can identify two adjacent corners in the room for further accuracy. Through later selection of the corner(s) for localization, the XR system can identify the saved room using depth sensors, with identification of corners being more reliable and detectable than other methods identifying walls.

In various examples, image space coordinates of an image from a video may be labeled, projected to determine 3D vehicle space coordinates, then transformed to 3D world space coordinates using known 3D world space coordinates and relative positioning between the coordinate spaces. For example, 3D vehicle space coordinates may be temporally correlated with known 3D world space coordinates measured while capturing the video. The known 3D world space coordinates and known relative positioning between the coordinate spaces may be used to offset or otherwise define a transform for the 3D vehicle space coordinates to world space. Resultant 3D world space coordinates may be used for one or more labeled frames to generate ground truth data. For example, 3D world space coordinates for left and right lane lines from multiple frames may be used to define lane lines for any given frame.

One example method includes receiving, by a display management device, one or more display configuration selections for one or more corresponding video conference displays for a hybrid conference location, the hybrid conference location providing a real-world space associated with a corresponding virtual conference hosted by a virtual conference provider; accessing, by the display management device, the corresponding virtual conference via the virtual conference provider; requesting, by the display management device, a plurality of video streams of the virtual conference from the virtual conference provider, each video stream corresponding to a virtual participant, the display management device not corresponding to any virtual participant; generating, for each video conference display of the one or more video conference displays, a real-time video rendering of a respective subset of the plurality of video streams according to the one or more display configurations; and providing, to the one or more video conference displays, the respective real-time video rendering.

The present disclosure relates, in part, to spatially aware media that includes three-dimensional (3D) spatial information pertaining to a real-world space. The spatially aware media may map this 3D spatial information to media such as an image, for example, to provide 3D spatial context for the media. This may allow users to more flexibly and efficiently interact with virtual content in real-world spaces that are relevant to them. According to one embodiment, spatially aware media is augmented to provide an image of a real-world space overlaid with a render of a 3D model defined relative to the 3D spatial features of the real-world space. Before augmenting the spatially aware media, a recommended position for the 3D model relative to the 3D spatial features of the real-world space may be determined based on the 3D model and / or on the spatially aware media.

The present application relates to a three-dimensional object detection method and device for an unmanned system based on multimodal fusion. The method includes the following steps: obtaining a two-dimensional image and three-dimensional point cloud data of a road scene to be detected and inputting them into a multimodal fusion detection model; the multimodal fusion detection model includes a temporal dynamic convolution fusion module, which uses a three-dimensional position embedding encoder to generate a three-dimensional position perception feature of the previous frame according to the three-dimensional world space coordinates of the previous frame and the two-dimensional image features of the previous frame. The three-dimensional position perception feature of the previous frame is used as a supplementary position perception feature of the current frame after pose transformation; generating a first fused frame three-dimensional point cloud feature according to the supplementary position perception feature of the current frame and the three-dimensional point cloud feature of the current frame; obtaining a preliminary fusion feature based on the first fused frame three-dimensional point cloud feature and the two-dimensional image feature of the current frame, and outputting a detection result based on the preliminary fusion feature. This method can solve the problem of difficult recognition of distant objects.

The application relates to a dynamic grid coordinate generation method, system, computer equipment and storage medium, and the technical scheme points are as follows: receiving window space coordinates of multiple vertices, inversely transforming the window space coordinates of the vertices into corresponding world space coordinates; generating a primitive using the window space coordinates of all the vertices; performing primitive assembly and primitive rasterization on the primitive to determine target pixel points; performing interpolation calculation on the primitive after the primitive rasterization according to the world space coordinates of the vertices to obtain the world space coordinates of the target pixel points; determining a scaling ratio of the world space coordinates of the vertices and the corresponding window space coordinates according to the inverse transformation, and determining a scale according to the scaling ratio; determining to-be-colored pixel points according to the scale, a preset grid line width and the world space coordinates of the target pixel points, coloring the to-be-colored pixel points to obtain a grid; and the application has the effect of dynamically adjusting the grid scale and better display effect.

The invention discloses a method for drawing a soft shadow of directional light. The method comprises the following steps: S1, establishing a space coordinate system in a forward direction by taking the direction of parallel light as a Z axis, and obtaining a conversion matrix; s2, an included angle range is set, and a new direction is sampled with the parallel light orientation as the center; s3, replacing the sampling direction to the world space through the conversion matrix and replacing the original parallel light orientation; and S4, judging whether shielding exists between the sampling direction and the irradiated point or not to complete soft shadow drawing. Cone angle sampling is carried out on each beam of parallel light to replace the original parallel light direction, the area is added for the parallel light, on the basis, whether the shielding object exists between the sampling direction and the irradiated object or not is judged, corresponding illumination calculation is carried out, and according to the mode, the soft shadow effect with light and dark changes can be obtained through rendering on the irradiated object. And the rendering result is closer to reality and is more realistic.

The invention relates to the technical field of point cloud data acquisition, and provides a method for accelerating acquisition of point cloud data in a simulation scene, which comprises the following steps: S1, setting main acquisition equipment for acquiring point cloud data of three-dimensional space information in the simulation scene, according to the simulation parameter information of the main acquisition device, setting an auxiliary acquisition device having the same visual field range as the main acquisition device; s2, in the simulation process of the simulation scene, according to the sampling parameter information of the main acquisition equipment, acquiring point cloud data with the same sampling interval as the main acquisition equipment from the sampling data of the auxiliary acquisition equipment, and mapping the point cloud data acquired by the auxiliary acquisition equipment into the point cloud data of the main acquisition equipment; and S3, calculating three-dimensional coordinates of the point cloud data in the world space coordinates, and calculating other information of the point cloud data. According to the technical scheme, the acquisition process of the point cloud data in the simulation scene is accelerated in a mode of combining two kinds of point cloud acquisition equipment, and the acquisition efficiency of the point cloud data in the simulation scene is improved.

In various examples, a two-dimensional (2D) and three-dimensional (3D) deep neural network (DNN) is implemented to fuse 2D and 3D object detection results for classifying objects. For example, regions of interest (ROIs) and / or bounding shapes corresponding thereto may be determined using one or more region proposal networks (RPNs)—such as an image-based RPN and / or a depth-based RPN. Each ROI may be extended into a frustum in 3D world-space, and a point cloud may be filtered to include only points from within the frustum. The remaining points may be voxelated to generate a volume in 3D world space, and the volume may be applied to a 3D DNN to generate one or more vectors. The one or more vectors, in addition to one or more additional vectors generated using a 2D DNN processing image data, may be applied to a classifier network to generate a classification for an object.

The present invention discloses a method and device for indicating the orientation of a three-dimensional model, and relates to the field of computer technology. A specific implementation of the method includes: determining whether the three-dimensional model is rendered on the screen; if not, converting the coordinates of the target position of the three-dimensional model in the world space coordinate system to the target coordinate system, thereby obtaining the coordinates of the target position in the target coordinate system; calculating the rotation angle according to the coordinates of the target position in the target coordinate system; and rotating the arrow indicator based on the rotation angle to indicate the orientation of the three-dimensional model. This implementation can solve the technical problem of being unable to determine the orientation of the model.

The present disclosure relates, in part, to spatially aware media that includes three-dimensional (3D) spatial information pertaining to a real-world space. The spatially aware media may map this 3D spatial information to media such as an image, for example, to provide 3D spatial context for the media. This may allow users to more flexibly and efficiently interact with virtual content in real-world spaces that are relevant to them. According to one embodiment, spatially aware media is augmented to provide an image of a real-world space overlaid with a render of a 3D model defined relative to the 3D spatial features of the real-world space. Before augmenting the spatially aware media, a recommended position for the 3D model relative to the 3D spatial features of the real-world space may be determined based on the 3D model and / or on the spatially aware media.