588 results about "3d space" patented technology

The invention discloses an image processing method and device and a storage medium. The method comprises the steps of obtaining a to-be-simulated 3D convolution model and training data; decomposing the 3D convolution model into cascading of a 3D space convolution model and a 3D time convolution model to obtain a pseudo 3D cascading convolution model; training a pseudo 3D cascade convolution modelby using the training data, and obtaining parameters of a 3D spatial convolution model and a 3D time convolution model; converting the 3D space convolution model and the 3D time convolution model intoa 2D space convolution model and a 2D time convolution model; setting a feature rearrangement rule for the 2D spatial convolution model and the 2D time convolution model; mapping model parameters ofthe 3D spatial convolution model and the 3D time convolution model into parameters of a 2D spatial convolution model and a 2D time convolution model to obtain a 2D cascaded convolution model; and performing convolution operation on the image by using the 2D spatial convolution model and the 2D time convolution model. By means of the mode, image processing conducted through 3D convolution operationcan be achieved through the 2D convolution model.

Apparatuses, systems, and techniques of using one or more machine learning processes (e.g., neural network(s)) to detect objects from a plurality of image frames. In at least one embodiment, a plurality of image frames are fused into a feature map using one or more neural networks. In at least one embodiment, a plurality of image frames are processed using one or more neural networks to detect objects in a 3D space.

The invention discloses a tunnel fracture recognition method based on three-dimensional live-action reconstruction and ordered re-acquisition of a virtual camera. The method specifically comprises the following steps: acquiring a multi-view rock fracture image; acquiring three-dimensional point cloud data of a tunnel wall surface according to the acquired rock mass fracture image, reconstructing a geometrical shape of the tunnel, mapping acquired texture information to a three-dimensional grid vertex through an image processing algorithm, and generating a three-dimensional tunnel model; setting parameters of the virtual camera, orderly collecting to form a virtual image, and enabling the position of the virtual camera to correspond to the real position of the model; and extracting information of the crack of the virtual image, and projecting the coordinates of the two-dimensional image back to the three-dimensional space through the projection matrix to obtain the space coordinates of the crack. A virtual image is generated by combining a texture mapping technology of a three-dimensional model, so that the limitation of a visual angle, illumination and space in actual image acquisition is overcome, and high-precision identification of tunnel cracks is realized.

An encoder projects a scene represented by 3D gaussian splats into 2D representation(s), where the 2D representation(s) store different parameters of the 3D gaussian splats, along with associated metadata describing transformation from 3D space into 2D representations. The encoder forms the 2D representation(s) and the associated metadata into bitstream(s). The encoder outputs the bitstreams. The decoder receives the bitstream(s) encoding 2D representation(s) and associated metadata. The decoder applies decoding to corresponding individual bitstreams to form the 2D representation(s) and the associated metadata. The decoder reprojects, as described by the associated metadata, the 2D representations) into the scene represented by the 3D gaussian splats. The decoder outputs the scene for rendering, or renders the scene, to a viewer.

Three-dimensional Gaussian splatting mechanisms that initialize a set of 3D Gaussian distributions, un-project pixels from two-dimensional (2D) planes to 3D space by applying queries to the 3D Gaussians at expected un-projected ray depth positions, and splat the 3D Gaussian distributions on the 2D planes based on the expected un-projected ray depth positions.

The invention provides a video data processing method and system based on a digital twin scene. The method comprises the following steps: performing spatial-temporal feature extraction on a video frame sequence to determine a first spatial-temporal feature matrix; segmenting a plurality of video elements from the video frame sequence, and generating an element segmentation mask corresponding to each video element; generating a second spatial-temporal characteristic matrix; obtaining a 2D motion track corresponding to the dynamic element based on the second spatial-temporal characteristic matrix; mapping the 2D motion trails of the dynamic elements into a 3D space to create or update a target virtual 3D scene model, and generating a predicted video frame sequence through the target virtual 3D scene model; and adjusting model parameters in the target virtual 3D scene model according to a comparison result of the predicted video frame sequence and the video frame sequence decomposed by the video data. According to the technical scheme provided by the invention, the video data processing efficiency can be effectively improved.

The invention discloses a low-illumination scene image three-dimensional reconstruction method and device based on a 3D Gaussian ball, and electronic equipment. The method comprises the following steps: 1) obtaining a low-illumination scene image; 2) initializing information of a 3D Gaussian ball in a 3D space according to camera parameters and a volume rendering technology to obtain an initial position, an initial radius and an initial color of the 3D Gaussian ball, and writing the initialized 3D Gaussian ball information into the enhanced image; 3) constructing an illumination adjustment network based on the convolutional neural network fused with the attention mechanism; 4) inputting the enhanced image obtained in the step 2) into an illumination adjustment network, and predicting an illumination weight value of each 3D Gaussian ball by using the illumination adjustment network; and 5) based on the initial information of the 3D Gaussian ball and the illumination weight value, performing three-dimensional scene reconstruction by using a three-dimensional reconstruction algorithm to obtain a three-dimensional scene reconstruction view Cnor under normal illumination. According to the invention, the illumination can be dynamically adjusted through the illumination adjusting network, and the real restoration of the low-illumination scene is realized.

Techniques are discussed herein for generating three-dimensional (3D) representations of an environment based on two-dimensional (2D) image data, and using the 3D representations to perform 3D object detection and other 3D analyses of the environment. 2D image data may be received, along with depth estimation data associated with the 2D image data. Using the 2D image data and associated depth data, an image-based object detector may generate 3D representations, including point clouds and / or 3D pixel grids, for the 2D image or particular regions of interest. In some examples, a 3D point cloud may be generated by projecting pixels from the 2D image into 3D space followed by a trained 3D convolutional neural network (CNN) performing object detection. Additionally or alternatively, a top-down view of a 3D pixel grid representation may be used to perform object detection using 2D convolutions.

A method and apparatus generate a 2-dimensional (2D) image. A method for generating a 2-dimensional (2D) image includes obtaining a time index and a view. The method further includes obtaining first coding indices and a first codebook for canonical 3D Gaussians, wherein the canonical 3D Gaussians are 3D Gaussians corresponding to a reference time index, and represent a 3D space corresponding to the reference time index. The method also includes obtaining second encoding indices and a second codebook for a parameter offset, wherein the parameter offset indicates a difference between the canonical 3D Gaussians and 3D Gaussians for the time index. The method further includes reconstructing the canonical 3D Gaussians based on the first coding indices and the first codebook. The method also includes reconstructing parameter offsets of the 3D Gaussians for the time index based on the second coding indices and the second codebook. The method further includes adding the reconstructed canonical 3D Gaussians and the reconstructed parameter offset to reconstruct the 3D Gaussians for the time index. The method also includes generating a second image for the view based on the reconstructed 3D Gaussians.

Methods, apparatus, devices, subsystems, and systems for holographically displaying live scenes are provided. In one aspect, a method includes optically generating an optical hologram of a live scene including one or more three-dimensional (3D) objects; capturing sequential optical holograms of the live scene and generating sequential hologram data associated with the sequential optical holograms of the live scene, each optical hologram being associated with respective hologram data; processing the at least part of the sequential hologram data based on a mathematical function to remove low frequency components to generate digital holograms associated with the live scene; and reconstructing the live scene in a 3D space based on the digital holograms.

The embodiment of the invention discloses a naked eye three-dimensional image display method and device, computer equipment and a storage medium. The method comprises the following steps: acquiring a plurality of three-dimensional virtual sub-objects and initial display parameters corresponding to the plurality of three-dimensional virtual sub-objects in a target naked-eye three-dimensional scene, wherein the plurality of three-dimensional virtual sub-objects have a logic association relationship; displaying initial naked-eye three-dimensional pictures corresponding to the plurality of three-dimensional virtual sub-objects in the naked-eye three-dimensional display screen according to the initial display parameters; generating space display parameters of the plurality of three-dimensional virtual sub-objects according to the logic association relationship; and dynamically adjusting the three-dimensional space positions of at least part of the three-dimensional virtual sub-objects in the plurality of three-dimensional virtual sub-objects according to the space display parameters so as to update the initial naked-eye three-dimensional picture to obtain a target naked-eye three-dimensional picture. In this way, the spatial display parameters are generated through the logic incidence relation to adjust the three-dimensional space position of the three-dimensional virtual sub-object, and simulation stimulation of the human brain in the three-dimensional space can be achieved when the human eyes watch the three-dimensional virtual sub-object.

Methods, systems, and apparatus, including medium-encoded computer program products, for searching through spatial recordings by constructing queries. One of the methods include constructing a query based on input received at a three-dimensional (3D) space displayed at a visual programming interface, wherein the input includes positioning of objects at 3D positions within the 3D space over time, wherein the input defines at least one spatial orientation between at least two objects from the objects in the 3D space; executing the query to search a database of 3D recordings to find at least one segment from at least one 3D recording that includes the at least two objects and matches the spatial orientation between the at least two objects, as defined in the input, at a specific point in time; and presenting the at least one segment from the at least one 3D recording via the visual programming interface.

The invention relates to the technical field of optical cable fault positioning, and discloses a cable fault spatial position positioning method, which comprises the following steps: acquiring 3D network data of an optical cable, and constructing a topological graph for describing the connection condition of the optical cable; carrying out OTDR detection on the optical cable, and calculating a fault distance between a fault point and a detection point based on an optical time domain reflection signal; exploring a fault path between a fault point and a detection point in the topological graph; if only one fault path exists, spatial positioning of a fault point is carried out; if at least two fault paths exist, extracting attenuation characteristics of the optical time domain reflection signals; screening a fault path based on the attenuation characteristics, and carrying out the spatial positioning of a fault point; according to the invention, the cable fault point can be accurately mapped to the 3D space model, and powerful support is provided for rapidly positioning and repairing the cable fault.

Aspects of the present disclosure relate to a system comprising a computer-readable storage medium storing at least one program and a method for rendering a virtual modification of a real-world environment depicted in image content. A reference surface is detected in a three-dimensional (3D) space captured within a camera feed generated by a camera of a computing device. An image mask is applied to the reference surface within the 3D space captured within the camera feed. A visual effect is applied to the image mask corresponding to the reference surface in the 3D space. Applying the visual effect to the image mask results in the modified surface being rendered in the camera feed being presented on a display of the computing device.

The invention discloses a 3D anomaly detection method and device based on self-distillation. According to the method, rapid and efficient zero sample anomaly detection is carried out on the 3D point cloud by using the self-distillation technology and the strong generalization ability of the 2D multi-modal model CLIP. According to the method, 3D point cloud features are learned and extracted in the following mode, and anomaly detection is carried out. Firstly, a lightweight learnable point cloud feature extraction module is designed; encoding the extracted point cloud features by using an encoder, and aligning the extracted point cloud features with encoding features of the point cloud rendering graph to a 3D space by using a self-distillation technology; and finally, performing zero sample anomaly detection of the 3D point cloud by using the feature. Compared with the prior art, the method can quickly and efficiently carry out zero sample anomaly detection of the 3D point cloud, and has relatively high practicability and innovativeness.

An audio conference system for automatically forming a single combined physical microphone array aperture across associated and / or disassociated ad-hoc microphone elements in a shared 3D space is provided. The audio conference system includes a plurality of microphone / speaker units, each including at least one microphone and / or at least one speaker and a system processor communicating with the microphone / speaker units. The system processor instructs the microphone / speaker units to transmit unique calibration signals sequentially or simultaneously and to calculate time difference of arrival (TDOA) between the microphone / speaker units. A physical array structure of the microphone / speaker units is obtained based on TDOA between the microphone / speaker units, and a consolidated target coverage zone common to the microphone / speaker units is generated based on the physical array structure.

Various implementations interpret user activity as user interactions with virtual elements (e.g., user interface elements) positioned within in a 3D space such as an XR environment. Some implementations enable a user to provide input using a direct input modality in which the user interacts with virtual content by virtually touching the virtual content. As examples, a user may move their finger to directly tap, pinch, swipe, or otherwise interact with a user interface (UI) element within a 3D space. Some implementations perform touch detection (e.g., detecting when a user's finger virtually makes, continues, and breaks virtual contact with a UI element in 3D space) using position and velocity information about the user.

The invention relates to the technical field of smart home and artificial intelligence, discloses a smart home equipment automatic arrangement method and system fusing 3D modeling and a neural network, and aims to solve the problems of complex manual operation, high technical threshold and high error rate in the traditional scheme design. The method comprises the following steps: acquiring original three-dimensional point cloud data of a target space through laser scanning equipment, and constructing a centimeter-level precision 3D space model; a deep learning semantic segmentation network is adopted to identify building components and functional areas, and a spatial semantic map is generated; performing multi-modal fusion on the spatial semantic features and user requirements to generate equipment type probability distribution and an optimal position thermodynamic diagram; performing iterative correction on the initial arrangement scheme based on a multi-objective optimization algorithm, and synchronously optimizing wireless signal coverage intensity, equipment energy consumption efficiency and project implementation cost; the optimized 3D arrangement scheme is automatically converted into a two-dimensional vector engineering drawing, and a construction drawing is generated; and extracting an equipment list and material consumption according to the final arrangement scheme, and automatically generating a project quotation and an installation process guidance document. The system comprises a spatial semantic analysis module, an equipment arrangement decision module, an engineering drawing generation module and a multi-objective optimization engine. Through adoption of the technical scheme, full-process automation from original spatial data to constructable drawings can be realized, and the efficiency and specialty of smart home scheme design are remarkably improved.

A computer-implemented method for 3D reconstruction of a target is provided, comprising obtaining an initial global reconstruction of the target in a 3D space, inferred by a global machine learning model; providing, to a user, an initial visualisation of the target based on the reconstruction; receiving, from the user, at least one indication of at least one point of interest in the visualisation; resampling at least one first subsection of the target based on the at least one point of interest to obtain local data, wherein the local data is associated with the subsection based on spatial information that associates the local data with a point in 3D space; inputting the resampled local data and spatial information into a local feature machine learning model to obtain at least one 3D reconstruction of the target, wherein the local feature machine learning model has been trained to output a target reconstruction from local data of resampled subsections, and wherein the 3D coordinate system of the local 3D reconstruction aligns with the global 3D reconstruction; and merging the global 3D reconstruction with the local 3D reconstruction. A corresponding computer system and computer readable medium may also be provided.

This invention relates to a method for visually recognising and tracking objects and events in 3D space with high accuracy, reduced computational load and fast setup, using virtual sensors that are designed to ensure high reliability and reduce costs by maximising their potential for accurate object recognition for given visual input data. The main task of a virtual sensor is to provide cost-effective and reliable data and replace the use of humans and / or hardware sensors to detect and recognise objects or events that are important to a given entity, such as an industrial enterprise and various business entities. The solution is well suited for industrial applications, especially for tracking larger objects of known shape or appearance, such as in large industrial halls, warehouses without fixed racking systems, container docks, train docks, car parks, etc. It is particularly suitable for tracking coils, metal pieces and larger building components. The method provides a reliable source of i data for obtaining a digital twin of the monitored objects, which represents real-time information about each stock keeping unit (SKU) in the warehouse (WH), bringing significant benefits to WH managers and thus saving human labour spent on searching for materials, improving management flow, reducing overall equipment effectiveness (OEE) of vehicles, improving throughput, quality, etc.

Disclosed herein are methods and system of implementing an AI-enabled high-fidelity occupancy network, allowing for the prediction of signed distances of voxelized objects in a 3D space surrounding an ego, thereby facilitating enhanced object shape refinement. Through the utilization of camera feeds only, an AI model accurately calculates signed distance values for various voxels in the 3D space. The predicted values can also be rendered by translating signed distance field values into image layers and subsequently stacking the layers to form a 3D representation of the space surrounding the ego. Furthermore, the system can predict and display various ground markings, expanding its functionality beyond traditional lane detection.

Provided herein are systems for determining a location of a gross target volume of a patient. In some examples, systems can include one or more processors that are configured to obtain image data associated with a plurality of images of a lesion of a patient. For each image, the one or more processors can be configured to backproject points representing the lesion into the 3D space to determine a plurality of distribution confidence values for a subset of voxels within the three-dimensional space. The one or more processors can be configured to determine a three-dimensional confidence distribution based on confidence values from the plurality of distribution confidence values corresponding to each voxel of the 3D space and determine a position of the lesion within the 3D space based on the 3D confidence distribution.

The invention provides a logistics track spatial data acquisition method based on a three-dimensional model, and relates to the technical field of data processing, and the method comprises the steps: 1, collecting the dynamic position data of a logistics object in a three-dimensional space in real time through a multi-source sensor, the three-dimensional space coordinate data comprising horizontal direction coordinates and vertical direction coordinates; 2, inputting the dynamic position data into a pre-constructed three-dimensional environment model, and generating a preliminary three-dimensional track through a space registration algorithm; and step 3, performing real-time updating processing on the preliminary three-dimensional trajectory, dynamically correcting trajectory data through a time sequence analysis algorithm, fusing obstacle information in the three-dimensional environment model, optimizing the precision of the preliminary three-dimensional trajectory, and obtaining optimized three-dimensional trajectory data. According to the invention, high-precision acquisition, dynamic optimization and precise correction of the logistics track data are realized, the accuracy and environmental adaptability of the logistics track data are improved, reliable data support is provided for intelligent logistics management, and the logistics efficiency is improved.

The invention provides an occupied grid prediction method and device, electronic equipment and a readable storage medium. The method comprises the following steps: acquiring continuously acquired multiple frames of visual images and laser point cloud data; converting the visual image to a point cloud scene to obtain image point cloud data; fusing the image point cloud data and the laser point cloud data to obtain fused voxel features; performing motion perception prediction on the dynamic voxels in the fused voxel features through the occupancy flow to obtain motion perception features corresponding to the dynamic voxels; and performing state prediction based on the motion perception features to obtain an occupation prediction result. Visual images are converted into image point cloud data, and the image point cloud data and laser point cloud data are fused in a 3D space to obtain voxel features, so that information loss caused by multi-mode mutual projection can be avoided, and meanwhile, motion perception is directly performed on fused voxels through an occupation stream, so that complete space information can be effectively captured and utilized, and the accuracy of the motion perception is improved. Therefore, more accurate and comprehensive occupied grid prediction is realized.

The invention relates to the technical field of human body breathing micro-motion modeling and holographic imaging, and particularly discloses a radar human body breathing micro-motion 4D modeling and high-resolution holographic imaging method, which is characterized by comprising the following steps of: performing irradiation perception on a human chest target by adopting a multiple-input multiple-output plane radar array; a thoracic cavity target is regarded as a'vector signal source ', namely the thoracic cavity target is equivalent to a set of scattering points of which the spatial positions are mutually discrete; in the geometric model, a graphical representation 3D space coordinate system is established by taking the center of a radar array as an original point and a plane where the array is located as an XOZ plane. According to the radar human body breathing micro-motion 4D modeling and high-resolution holographic imaging method, the micro-Doppler principle is combined, and the micro-Doppler information of each scattering point in the fourth dimension, namely the slow time dimension, is obtained based on the phase decomposition and extraction technology, so that 4D precise reduction of the fluctuation process of the thoracic cavity of the human body is realized.

The invention provides a cerebral hemorrhage hematoma enlargement prediction system and method based on deep learning, and the system comprises a data obtaining module which is used for obtaining a baseline CT image and clinical data of a patient; the image preprocessing module is used for carrying out standardization processing on the CT image; the feature extraction module is used for extracting image features from the CT image by adopting a deep convolutional neural network; the feature fusion module is used for fusing the image features, the clinical features and the radiology features; and the prediction module is used for predicting the hematoma expansion risk based on the fused features. The system captures 3D space information to the maximum extent by extracting the maximum lesion level, shows high reliability, high interpretability and clinical availability in the aspect of predicting hematoma expansion, is remarkably superior to prediction of clinicians, achieves reasonable consistency of prediction probability and actual probability on multiple data sets, and has good application prospects. The system can be used as a clinical decision support system and has potential to improve patient prognosis.

A cross-regional and cross-view learning (C2RV) framework is provided for sparse-view reconstruction in cone-beam computed tomography (CBCT) by advantageously leveraging cross-region and cross-view feature learning to enhance representation of a point in 3D space before estimating an attenuation coefficient of the point. Specifically, multi-scale 3D volumetric representations (MS-3DV) are first introduced, where features are obtained by back-projecting multi-view features at different scales to the 3D space. Explicit MS-3DV enable cross-regional learning in the 3D space, providing richer information that helps better identify different internal anatomy structures. Hence, features of the point can be queried in a hybrid way, i.e. multi-scale voxel-aligned features from MS-3DV and multi-view pixel-aligned features from projections. Instead of considering queried features equally, scale-view cross-attention (SVC-Att) is used to adaptively learn aggregation weights by self-attention and cross-attention. Finally, multi-scale and multi-view features are aggregated to estimate the attenuation coefficient.

Embodiments are disclosed for determining a spin rate and axis of a ball using deep learning. In some embodiments, a method comprises: training a deep learning network on training images of spinning balls, each spinning ball having at least one feature point in a time series that forms a two-dimensional (2D) ellipse image in a 2D plane; capturing, with at least one camera, a series of images of a ball; predicting, with the trained deep learning network, spin measurements associated with the ball based on the series of images; determining a spin rate of the ball based on the spin measurements; determining coefficients of a 2D ellipse model based on the spin measurements and the spin rate; and determining, with the at least one processor, a spin axis of the ball in 3D space based on the 2D ellipse model and the spin rate.

The invention relates to the technical field of virtual interaction, in particular to a virtuality and reality combined multi-dimensional display interaction method based on a digital technology, which comprises the steps of S1, virtual fusion scene construction, S2, user behavior capture and data fusion, S3, dynamic instruction generation and response, and S4, multi-modal feedback and dynamic adjustment. According to the method, holographic projection, atomization imaging and AR / VR are fused to construct a virtual-reality seamless combined multi-dimensional interaction scene, and a user is supported to freely pass through a naked-eye 3D space; personalized content is dynamically generated, and somatosensory and acousto-optic multi-mode interaction is realized in combination with a sensor network; and meanwhile, through a block chain reward mechanism and distributed intelligent control, the immersion, expansibility and commercialization potential are remarkably improved, and the core problems of single interaction, scene splitting, response lag and the like in the traditional technology are solved.