69 results about "Multiple viewpoint" patented technology

Example embodiments relate to techniques for volumetric performance capture with neural rendering. A technique may involve initially obtaining images that depict a subject from multiple viewpoints and under various lighting conditions using a light stage and depth data corresponding to the subject using infrared cameras. A neural network may extract features of the subject from the images based on the depth data and map the features into a texture space (e.g., the UV texture space). A neural renderer can be used to generate an output image depicting the subject from a target view such that illumination of the subject in the output image aligns with the target view. The neural render may resample the features of the subject from the texture space to an image space to generate the output image.

The invention discloses an anti-shielding three-dimensional scanning measurement field construction method for large-size measurement, and belongs to the technical field of optical three-dimensional measurement. The method comprises the following steps: firstly, performing adaptive sampling based on geometric features of a workpiece to generate a candidate mark point set; secondly, constructing a virtual measurement scene, simulating a dynamic scanning process by using a ray tracing technology, and calculating the visibility and observation quality of each point under multiple viewpoints; then measuring precision is quantified by using a Fisher information matrix, and a layout optimization model which takes maximization of information gain as a target and meets coverage rate and engineering constraint at the same time is established; and finally, solving the model to obtain an optimal mark point subset, and verifying and outputting the optimal mark point subset. According to the method, the problems of measurement interruption, low point distribution efficiency and difficulty in guaranteeing precision caused by shielding of a large-size complex component in scanning are solved, and automatic construction of a high-robustness and high-precision measurement field is realized.

The invention relates to the technical field of intelligent logistics storage, in particular to a tray box target detection and quantity statistics method based on a multi-view depth camera, and the method comprises the following steps: 1, building a system architecture; a pretreatment process; step 3, a target identification process; step 4, a depth camera continuous image acquisition process; and step 5, an output process: adopting the training model to carry out tray box body identification on a plurality of visual angles, and carrying out fusion and three-dimensional positioning on detection result information of the plurality of visual angles to complete quantity statistics. The positions, the depths and the stacking layers of the box bodies on the tray and the number of the box bodies on each layer can be quickly and accurately identified, and the cargo management efficiency in logistics storage and industrial production is improved.

The invention relates to a comment generation method and device, equipment, a storage medium and a program product. According to one embodiment of the disclosure, the method comprises: selecting a target personality portrait from a pre-constructed personality portrait database, the personality portrait database comprising a plurality of different personality portraits; selecting a target watching point matched with the target personality portrait from watching points of multiple angles corresponding to original contents, wherein the original contents comprise contents for generating comments; and inputting the target personality portrait and the target watching point into a pre-trained comment generation model to obtain a currently generated comment. According to the method and the device, the diversity of the generated comments can be improved, so that the network environment interaction atmosphere foiling effect can be improved.

This invention relates to the field of visual recognition technology, specifically disclosing a target recognition processing method based on sliding window and multi-view fusion. The method includes: acquiring video streams from multiple viewpoints of a target scene; extracting and recognizing visual features of video frames for each viewpoint video segment to obtain basic recognition results under a single viewpoint; determining whether the basic recognition results of multiple consecutive frames under each single viewpoint conform to preset business rules, generating a single-view result sequence; performing frame-level fusion of the visual features under each single viewpoint to obtain fused visual features of multiple frames under multiple viewpoints; determining whether the fused visual features of each frame conform to preset spatial rules, generating a multi-view result sequence; and performing smoothing and fusion processing on each single-view result sequence and the multi-view result sequence to generate the recognition result of the current video segment. This invention can fully utilize multi-view data to improve recognition accuracy.

The invention discloses a target hierarchical enhancement reconstruction method and reconstruction system based on a polarization camera array. The method comprises the following steps: acquiring a depth parallax model of the polarization camera array; collecting a multi-view target original view in multiple polarization directions; according to the target original view, performing occlusion layering based on a depth parallax model to obtain occlusion layering images of different depths; performing adaptive threshold segmentation, morphological processing and central viewpoint conversion on the shielding layered image of any depth to obtain a shielding mask of any viewpoint, further obtaining a shielding mask corresponding to a target layer, performing pixel-by-pixel reconstruction, and obtaining a calculation imaging result of the target layer after shielding is removed; and obtaining a final target enhanced reconstructed image through polarization state fusion. The image contrast and the layered sensing capability of the system under the shielding condition are improved, a higher-precision polarization imaging result can be generated, and the method is used for layered reconstruction of any object layer.

The invention relates to an exposure sequence adaptive optimization method and device for multi-view automatic measurement, computer equipment, a computer readable storage medium and a computer program product. The method comprises the following steps: acquiring an original exposure time sequence of a plurality of viewpoints; the original exposure time sequence comprises a plurality of exposure segments, and each exposure segment corresponds to one viewpoint; calculating similarity data among the exposure segments, and grouping the exposure segments according to the similarity data to obtain a plurality of initial exposure groups; for each initial exposure group, re-grouping the exposure segments at the boundary position to obtain a plurality of boundary grouping results, and calculating grouping cost data for each boundary grouping result; and selecting a target grouping result from the boundary grouping results according to the grouping cost data, optimizing the target grouping result according to the grouping cost data to obtain a plurality of target exposure groups, and calculating the exposure time of each target exposure group. The method can shorten the measurement period.

Items are identified in a waste stream for purposes of recycling, using deterministic and / or probabilistic techniques. Imagery of the waste stream from multiple viewpoints permit creation of a 3D depth draped image representation, from which one or more 2D planes can be synthesized. Phase-coherent patches of recoverable encoded data can be identified from among soiled and crumpled object surfaces, and used in combination to recover object identification information. Recognition of certain items can trigger further image processing that is specific to such items. (Detection of a catsup bottle, for example, can trigger image analysis to discern the presence of catsup residue.) Information about recognized objects can be provided to external data customers, e.g., to track grey market diversion of particular products into unlicensed territories. These and other features and advantages, which can be used alone or in combination, are detailed herein.

The invention discloses a large-curvature thin-wall component cladding layer feature recognition method based on three-dimensional point cloud information, and the method comprises the steps: carrying out the filtering and simplification preprocessing of the collected point cloud data of a repaired blade profile at each visual angle, and obtaining the blade point cloud data at each visual angle; performing multi-view point cloud splicing on the blade point cloud data to obtain point cloud data of a repaired blade profile; calculating an initial point cloud normal vector and an initial point cloud curvature of each point position, and redirecting the initial point cloud normal vector to obtain a repaired blade initial point cloud normal vector consistent in direction; re-calculating the normal vector of the repaired blade point cloud to perform feature enhancement on the normal vector of the initial point cloud so as to obtain the normal vector feature enhanced repaired blade profile point cloud data; clustering the cladding layer features in each region to obtain point cloud data of the cladding layer in each region; and carrying out classification and combination on the cladding layer point cloud of each region by using Euclidean distance clustering to obtain completely spliced repaired blade cladding layer point cloud data.

We estimate a three-dimensional field capable of generating virtual viewpoint images that do not exhibit any noticeable inconsistencies in image quality when switching virtual viewpoints. [Solution] The image processing device 102 acquires multiple captured images obtained from multiple viewpoints and camera parameters corresponding to each viewpoint, acquires the shooting resolution which is the resolution of each captured image, generates multiple training images corresponding to each captured image by converting the resolution of each captured image based on the resolution of each of the multiple training images corresponding to each captured image which is determined based on the shooting resolution of each captured image, generates camera parameters corresponding to each training image by converting the camera parameters corresponding to each viewpoint based on the resolution of each training image, and learns a learning model concerning the three-dimensional field of the target space based on the multiple training images and the camera parameters corresponding to each training image.

The embodiment of the invention provides a video, image and 3D information generation method and device, a medium and a product, and belongs to the field of AI.The method comprises the steps that multiple to-be-edited images corresponding to multiple view angles are obtained, a to-be-edited area in a first to-be-edited image of an initial view angle is edited to obtain a first edited image, and the first edited image is edited to obtain a second edited image; and generating a plurality of sheltered second to-be-edited images corresponding to the other plurality of second to-be-edited images according to the to-be-edited area in the first to-be-edited image. And inputting the first edited image and the plurality of occluded second to-be-edited images into a target image restoration model to obtain a plurality of second edited images, wherein the target image restoration model diffuses an editing result in the first edited image to the plurality of occluded second to-be-edited images. And generating a video according to the plurality of edited images. Through the scheme, multi-view consistent image editing can be realized.

This application relates to a high-fidelity real-time rendering method, apparatus, device, and storage medium for dynamic digital humans. It includes receiving multimodal driving signals, mapping them to expression and pose parameters, inputting a parameterized human model to generate standard spatial geometry and skinning weights, and constructing a three-dimensional Gaussian sputtering field to deform to the target pose to obtain dynamic geometric data; arranging sparse virtual cameras in the target pose space, and simultaneously rendering and generating a sparse reference view set containing color and depth within a single rendering call; determining dense virtual viewpoints based on the display device's viewpoint parameters, and generating depth range textures for each dense viewpoint through downsampling depth reprojection and point sputtering techniques; determining ray step intervals using the depth range textures, sampling along the rays and fusing signed distance values ​​truncated from multiple viewpoints to generate surface position textures; projecting the surface positions onto sparse view sampled colors, fusing to generate a dense view color image, and adapting it for output to different display devices.

To acquire a three dimensional field capable of suppressing deterioration of reproducibility of an image of an object included in a virtual viewpoint image even when a position or a direction of a viewpoint used in learning is greatly different from a position of a virtual viewpoint or a direction of a line of sight at the virtual viewpoint.SOLUTION: The image processing apparatus 102 acquires a plurality of captured images obtained by image capturing from a plurality of positions and a plurality of camera parameters of a plurality of viewpoints corresponding to the plurality of positions. A camera parameter of an auxiliary viewpoint different from the plurality of viewpoints is generated, shape data of an object estimated based on the plurality of acquired camera parameters and the plurality of captured images is acquired, an auxiliary viewpoint image is generated based on the shape data and the generated camera parameter, and information regarding a three dimensional field corresponding to at least a partial space in an image capturing space captured from a plurality of positions is generated based on the plurality of acquired camera parameters, the plurality of captured images, the generated camera parameter, and the auxiliary viewpoint image.SELECTED DRAWING: Figure 10A

This invention belongs to the field of computer vision and action recognition, specifically relating to a robust multi-view skeleton fusion method based on information priority selection and effective joint masking. It includes: acquiring skeleton data from multiple viewpoints, performing data preprocessing and effective joint extraction; skeleton registration based on information content to obtain a registered skeleton; obtaining the skeleton most suitable for action recognition based on the registered skeleton and a GCN model; obtaining the action category based on the most suitable skeleton using a modern action recognition model; and adjusting model parameters using cross-entropy loss based on the action category results to optimize iterative recognition accuracy. This invention effectively solves the problem of low-quality noise interfering with high-fidelity data in multi-view fusion through an information content evaluation mechanism and a Boolean mask completion strategy; combined with GCN adaptive viewpoint adjustment, it enhances the model's robustness to different camera layouts and significantly improves the accuracy and stability of human action recognition in complex occlusion environments.

The application discloses a three-dimensional indoor scene generation method and related equipment, and the method comprises the steps of obtaining a text description of an indoor scene; generating an indoor layout through a preset diffusion model; converting the indoor layout into a two-dimensional height field and a two-dimensional semantic graph; predicting a neural radiance field based on the two-dimensional height field and the two-dimensional semantic graph; sampling multiple viewpoints in the predicted neural radiance field, rendering the multiple viewpoints, and obtaining an RGB-D image corresponding to each viewpoint, wherein RGB is color information of a two-dimensional image, and D is depth information of the two-dimensional image; constructing a truncated signed distance field body according to the RGB-D image corresponding to each viewpoint, generating a grid triangular facet based on the truncated signed distance field body, and obtaining an indoor three-dimensional scene. The application can generate a high-quality three-dimensional indoor scene without real multi-view images, improve the efficiency of generating a three-dimensional scene, and can be widely applied in the technical field of computer vision.

Example embodiments relate to techniques for volumetric performance capture with neural rendering. A technique may involve initially obtaining images that depict a subject from multiple viewpoints and under various lighting conditions using a light stage and depth data corresponding to the subject using infrared cameras. A neural network may extract features of the subject from the images based on the depth data and map the features into a texture space (e.g., the UV texture space). A neural renderer can be used to generate an output image depicting the subject from a target view such that illumination of the subject in the output image aligns with the target view. The neural render may resample the features of the subject from the texture space to an image space to generate the output image.

This document discloses a holographic communication method, electronic device, storage medium, and program product. The method includes: acquiring a three-dimensional model of a first object; the three-dimensional model is obtained based on multiple depth images and multiple attribute images, the multiple depth images and multiple attribute images being obtained based on multiple first images, the multiple first images being acquired from multiple viewpoints of the first object, the multiple attribute images including attribute data of multiple primitives under the multiple viewpoints, the primitives being basic units representing the first object; and rendering and displaying the three-dimensional model based on the three-dimensional model and the client's viewpoint. Therefore, high-fidelity stereoscopic image reconstruction can be achieved on the client using the acquired multi-view images, while ensuring viewpoint integrity.

A system and method for viewing aerial imagery and derived content on a web-based imagery browser that can be accessed by at least two users simultaneously. In addition, a system for interactively generating 3D geometry data for a location imaged from multiple viewpoints.

A system is provided.SOLUTION: A system, comprising: means for receiving a request from a user; means for retrieving and generating information based on the request; and means for providing the generated information to the user, wherein the means for receiving the request from the user includes means for receiving the request from the user, means for generating the information based on the request, and means for providing the information to the user. AI AI AI.SELECTED DRAWING: Figure 1

According to an embodiment of the present disclosure, a method of generating a stereoscopic video includes: obtaining video data; extracting at least one multiple viewpoint video data from the video data; identifying an object associated with the video data; generating tracking information for the object based on the at least one multiple viewpoint video data; and performing rendering on the object based on a depth value corresponding to the tracking information.

The application provides a kind of adaptive scanning visual angle planning method of medical auxiliary robot, mainly includes four steps: first, the three-dimensional grid model of patient is normalized to simulation environment, and the operating space of camera is limited;Second, according to three-dimensional grid model, target site identification and historical scanning information, construct composite state vector;Then, input the vector through strategy network, generate three-dimensional action vector representing the next optimal viewpoint coordinates;Finally, generate a series of viewpoints composed of multiple viewpoints, based on this sequence, use path planning algorithm to generate collision-free trajectory, and collect data at each viewpoint, finally obtain the three-dimensional point cloud of target site.The application has the beneficial effects: through conditional learning, a model with strong generalization ability for multiple parts is realized, and multiple targets such as scanning integrity, quality and safety can be automatically weighed, and finally a safe, efficient and high-precision automatic scanning scheme far beyond traditional methods is generated.

To enable the progress of work and the like to be easily grasped from videos.SOLUTION: A video analysis device 1 comprises: a three-dimensional data feature amount conversion unit 15 for assuming two dimensions by perspective projection at a plurality of viewpoints for three-dimensional data 14 obtained by measuring an object with a three-dimensional measuring device and extracting a feature amount; a three-dimensional data feature amount holding unit 16 for holding the feature amount; a video feature amount conversion unit 12 for extracting the feature amount of video data 11 obtained by capturing an image of the same object with a camera; a video feature amount holding unit 13 for preserving the feature amount; a spatial alignment unit 17 for comparing the feature amount of the three-dimensional data preserved in the three-dimensional data feature amount holding unit 16 with the feature amount of the video data preserved in the video feature amount holding unit 13, and determining two-dimensional projection data whose field angle is closest to the video data; and three-dimensional data generation unit 19 for associating the video data with the three-dimensional data on the basis of the two-dimensional projection data and generating associated three-dimensional data.SELECTED DRAWING: Figure 6

An image synthesis device includes a memory storing a virtual viewpoint image synthesis program, and a processor configured to execute the program. The program receives images of multiple viewpoints obtained by filming around a user, position information on the user, and information on a direction in which scene is viewed, estimates color and density by inputting the position information of the user and the direction in which the object is viewed to a neural radiance field model constructed by using the images of the multiple viewpoints, and synthesizes images of a virtual viewpoint by performing volume rendering by using estimated color and density, the neural network radiance field model is constructed based on a quasi-uniform spherical coordinate system, and the quasi-uniform spherical coordinate system includes a sum of a Yin grid and a Yang grid and has a grid structure that increases exponentially in a radial direction.

This invention relates to a method and electronic device for playing six-degree-of-freedom video in the field of immersive video technology. The method includes obtaining a first position, which is the position of a target object in the six-degree-of-freedom video in a virtual three-dimensional space at a given time, and a second position, which is the viewpoint position of the user in the virtual three-dimensional space at the same time; obtaining the line of sight direction according to the first and second positions; obtaining a target viewpoint stream corresponding to the line of sight direction from among multiple viewpoint streams of the six-degree-of-freedom video; and playing the six-degree-of-freedom video in a virtual three-dimensional space based on the target viewpoint stream.

This invention relates to a cross-view online action detection method based on probabilistic temporal mask attention. The specific steps include: S1, acquiring training data by obtaining video streams from multiple viewpoints in a target scene; S2, data preprocessing by extracting RGB and optical flow features from the video streams based on a pre-trained model; S3, constructing a dual-branch network structure, where the probabilistic branch compresses viewpoint-level features into latent viewpoint-specific codes, and the classification branch uses these codes for autoregressive classification; S4, constructing a GRU-TMA module to reduce interference from distant historical information on current action detection; S5, model training and optimization by minimizing reconstruction loss and KL divergence loss to optimize the latent representation of the probabilistic branch, while simultaneously optimizing the action classification capability of the classification branch through cross-entropy loss; S6, online action detection by classifying sequential frames based on the output of the dual-branch network and evaluating performance according to actual action labels. This invention improves the accuracy and robustness of action detection.

This invention discloses a collaborative scanning method, device, medium, and product for aero-engines. The method includes: controlling a continuous inspection robotic arm to perform a coarse scan of the aero-engine to obtain a coarse point cloud model; forming multiple candidate viewpoints based on the coarse point cloud model; grouping the candidate viewpoints to obtain multiple viewpoint groups and forming a joint viewpoint visibility matrix; obtaining a sequence of target viewpoint groups based on the joint viewpoint visibility matrix; controlling each industrial camera of the robotic arm to position itself at a target viewpoint in each target viewpoint group for high-resolution imaging by adjusting the bending and / or rotating traction ropes of the continuous inspection robotic arm, thereby obtaining a local fine point cloud; and obtaining a high-precision 3D solid model based on the coarse point cloud model and the local fine point cloud. This invention fundamentally avoids mechanical interference and collision damage, achieves full coverage of complex curved surfaces without blind spots, and improves the accuracy and efficiency of 3D reconstruction of aero-engines.

This invention relates to a multi-view 3D reconstruction method incorporating edge priors, belonging to the field of computer vision technology, and solves the problem of poor accuracy in existing 3D reconstruction results. The 3D reconstruction method includes: acquiring multiple viewpoint images of the scene to be reconstructed; selecting one viewpoint image and its corresponding edge image from the multiple viewpoint images as a reference image and its edge image, and selecting multiple source images from the multiple viewpoint images to obtain multiple image combinations; inputting each of the multiple image combinations into a pre-trained depth prediction model to obtain a depth map corresponding to the reference image in each image combination; and converting the pixels in the reference image into spatial points in the scene to be reconstructed based on the intrinsic and extrinsic parameter matrices of the reference image and the corresponding depth map in each image combination, thereby obtaining the reconstructed scene. This improves the accuracy of 3D reconstruction.

To reduce the influence of a photographer's shadow appearing on a subject when generating a three-dimensional model by capturing the subject from multiple directions.SOLUTION: Included are acquisition unit that acquires multiple images of a subject captured from multiple viewpoints, a generation unit that generates a three-dimensional model of the subject using multiple images, and an extraction unit that extracts areas on the subject where the position changes at each of the multiple viewpoints using the multiple images. The generation unit generates the three-dimensional model from the multiple images without using the image information of the areas extracted by the extraction unit.SELECTED DRAWING: Figure 2

This invention provides a technology that reduces user discomfort with light emanating from pixels in a display device capable of displaying different images from multiple viewpoints. [Solution] The display device includes a quasi-planar substrate including a flat or curved surface; a group of pixels including a plurality of display pixels at different positions in a first direction parallel to the tangent plane of the substrate; a first lens element configured to refract light incident from a plurality of the display pixels of the pixel group in a third direction perpendicular to the tangent plane of the substrate in separate directions within a plane including the first and third directions; and a second lens element configured to refract light incident from the display pixels in the third direction within a plane including a second direction and a third direction that intersect the first direction at a specific angle within the tangent plane of the substrate.

This invention belongs to the field of defect detection and specifically discloses a method for annotating defect data of complex parts, a defect detection method, and a multi-view, multi-illumination data acquisition device. In the data acquisition device, a turntable provides multiple viewpoints of the part under test, and the light source design provides multiple illumination conditions for the camera, ensuring the acquisition of information about the complete surface image of the part under test. For defect data, to avoid manual annotation of image pixels, fluorescent markings are applied to the defect locations beforehand. Therefore, under fluorescent illumination conditions during the acquisition process, clear defect locations can be obtained. A three-level defect segmentation method is used to obtain high-precision pixel-level mask information of the defect locations in the image. This invention can realize data acquisition of complex parts under multiple viewpoints and illumination conditions, and can automatically annotate defect data.