98 results about "Multiple viewpoint" patented technology

The invention relates to an image processing technology, and discloses an efficient panoramic image splicing method and system based on multi-view fusion, and the method comprises the steps: collecting a plurality of images from different views; performing multi-scale feature extraction on each image, generating a feature descriptor for each extracted feature point, and matching a corresponding feature point pair; performing multi-view geometric constraint screening on the feature point pairs; according to the screened feature point pairs, estimating a homography matrix between adjacent images, and carrying out global optimization on the homography matrix; aligning all the images into the same coordinate system; determining an overlapping region between adjacent images; and according to the pixel information in the overlapping areas, fusing the overlapping areas by adopting a self-adaptive weighted fusion algorithm so as to splice the plurality of images into a panoramic image. The invention further discloses a control device and a computer readable storage medium. The invention aims to improve the efficiency and accuracy of generating the multi-view fused panoramic image.

Example embodiments relate to techniques for volumetric performance capture with neural rendering. A technique may involve initially obtaining images that depict a subject from multiple viewpoints and under various lighting conditions using a light stage and depth data corresponding to the subject using infrared cameras. A neural network may extract features of the subject from the images based on the depth data and map the features into a texture space (e.g., the UV texture space). A neural renderer can be used to generate an output image depicting the subject from a target view such that illumination of the subject in the output image aligns with the target view. The neural render may resample the features of the subject from the texture space to an image space to generate the output image.

The invention discloses an anti-shielding three-dimensional scanning measurement field construction method for large-size measurement, and belongs to the technical field of optical three-dimensional measurement. The method comprises the following steps: firstly, performing adaptive sampling based on geometric features of a workpiece to generate a candidate mark point set; secondly, constructing a virtual measurement scene, simulating a dynamic scanning process by using a ray tracing technology, and calculating the visibility and observation quality of each point under multiple viewpoints; then measuring precision is quantified by using a Fisher information matrix, and a layout optimization model which takes maximization of information gain as a target and meets coverage rate and engineering constraint at the same time is established; and finally, solving the model to obtain an optimal mark point subset, and verifying and outputting the optimal mark point subset. According to the method, the problems of measurement interruption, low point distribution efficiency and difficulty in guaranteeing precision caused by shielding of a large-size complex component in scanning are solved, and automatic construction of a high-robustness and high-precision measurement field is realized.

A method of generating a three-dimensional (3D) object includes generating, with a multi-view stereo (MVS) neural reconstruction network, a feature volume from images that include multiple viewpoints of a subject, and applying score distillation sampling (SDS) fine-tuning to the feature volume resulting in a 3D object of the subject. A non-volatile computer-readable medium with instructions configured to cause performance of operations of said method. A system for providing a 3D object including an input to receive a text prompt from a user that corresponds to a subject. The system including a 3D object engine configured to generate, using a multi-view diffusion model, one or more images of the subject from different viewpoints, generate, with a MVS reconstruction neural network, a feature volume from the images of the subject, and apply SDS fine-tuning to the feature volume resulting in the 3D object of the subject.

The invention relates to the technical field of robot vision, in particular to a three-dimensional reconstruction viewpoint planning method based on collaborative robot 3D vision, which comprises the following steps: S1, performing visual calibration on a 3D camera on a collaborative robot; s2, whether a workpiece standard model exists or not is judged, if yes, a candidate viewpoint set is constructed according to the workpiece standard model, and if not, the workpiece standard model is established according to preset fixed multiple viewpoints, and then the candidate viewpoint set is constructed; and S3, a deep reinforcement learning mechanism obtains a planning result according to the candidate viewpoint set, controls an actual six-axis collaborative robot to move according to the planning result, carries out data acquisition, obtains 3D point cloud data of the workpiece at the visual angle, carries out preprocessing including noise point removal and filtering, and generates a three-dimensional model of the workpiece based on the fused point cloud data. The three-dimensional reconstruction method is high in automation degree, almost does not need manual operation in the whole process, and can effectively perform efficient and rapid three-dimensional reconstruction on a large batch of different types of non-standard customized small-batch workpieces.

Machine learning is used to transform screen renders from the viewpoint of the player's character to be from the viewpoint of other non-player characters (NPC) in the room. One or more neural networks are trained using game images captured during a human play session, and then subsequently the neural networks are used to create realistic video from the NPC viewpoints. To avoid rendering multiple viewpoints simultaneously, a single viewpoint is rendered, and neural networks are used to transform it for the viewpoints of other NPCs in the area. A group of NPCs may be treated as a batch and a single viewpoint transformed to multiple viewpoints in a single inference pass. For cloud gaming the game video can be rendered / sent immediately for the player but the actions of the NPCs are delayed for a frame while the neural network generates the behavior.

The invention relates to the technical field of intelligent logistics storage, in particular to a tray box target detection and quantity statistics method based on a multi-view depth camera, and the method comprises the following steps: 1, building a system architecture; a pretreatment process; step 3, a target identification process; step 4, a depth camera continuous image acquisition process; and step 5, an output process: adopting the training model to carry out tray box body identification on a plurality of visual angles, and carrying out fusion and three-dimensional positioning on detection result information of the plurality of visual angles to complete quantity statistics. The positions, the depths and the stacking layers of the box bodies on the tray and the number of the box bodies on each layer can be quickly and accurately identified, and the cargo management efficiency in logistics storage and industrial production is improved.

The invention provides a multi-view autonomous cooperative positioning method and system. The method comprises the following steps: determining a plurality of local maps and a plurality of key frame information of an environment where a plurality of agents are located based on a preset synchronous positioning and mapping algorithm; the plurality of intelligent agents are used for acquiring local environment information of multiple viewpoints from different positions and visual angles through carried visual sensors; the multi-view local environment information corresponds to a plurality of local maps and a plurality of key frame information; determining a constraint relationship among the plurality of local maps according to the information of the plurality of key frames, and determining a local map coordinate system transformation matrix according to the constraint relationship; the constraint relation comprises pose constraints of adjacent key frames; and fusing the plurality of local maps according to the local map coordinate system transformation matrix to obtain a global map, and determining global positioning information according to the global map. The method can adapt to a complex environment, and realizes high-precision and high-robustness unmanned aerial vehicle cluster positioning.

Various embodiments of the present disclosure relate generally to systems and methods for generating multi-view interactive digital media representations in a virtual reality environment. According to particular embodiments, a plurality of images is fused into a first content model and a first context model, both of which include multi-view interactive digital media representations of objects. Next, a virtual reality environment is generated using the first content model and the first context model. The virtual reality environment includes a first layer and a second layer. The user can navigate through and within the virtual reality environment to switch between multiple viewpoints of the content model via corresponding physical movements. The first layer includes the first content model and the second layer includes a second content model and wherein selection of the first layer provides access to the second layer with the second content model.

The invention relates to a comment generation method and device, equipment, a storage medium and a program product. According to one embodiment of the disclosure, the method comprises: selecting a target personality portrait from a pre-constructed personality portrait database, the personality portrait database comprising a plurality of different personality portraits; selecting a target watching point matched with the target personality portrait from watching points of multiple angles corresponding to original contents, wherein the original contents comprise contents for generating comments; and inputting the target personality portrait and the target watching point into a pre-trained comment generation model to obtain a currently generated comment. According to the method and the device, the diversity of the generated comments can be improved, so that the network environment interaction atmosphere foiling effect can be improved.

The present invention discloses a method for simulating defects in scanning point clouds of physical mechanical parts taking into account multiple factors. The method comprises: obtaining a defect-free scanning point cloud of a physical mechanical part and constructing a point cloud ray under a single viewpoint, and then constructing a single viewpoint visible point cloud; obtaining a simulated reflective defect point cloud according to the reflection intensity of each visible point; constructing a multi-viewpoint visible point cloud under multiple viewpoints, and obtaining a simulated blind area defect point cloud after intersecting and merging the multi-viewpoint visible point cloud and the reflective defect point cloud; then removing the defect circle and the bottom defect point to obtain the final simulated defect point cloud. The method of the present invention can simulate common missing or damaged situations in the actual point cloud acquisition process, can cover a wide range of scenarios, provide diversified defect simulations, enhance the realism and complexity of point cloud missing processing, reduce the time limit problem that may be encountered in the actual acquisition process, improve the efficiency of point cloud processing, and provide strong support for research and technological development in related fields.

This invention relates to the field of visual recognition technology, specifically disclosing a target recognition processing method based on sliding window and multi-view fusion. The method includes: acquiring video streams from multiple viewpoints of a target scene; extracting and recognizing visual features of video frames for each viewpoint video segment to obtain basic recognition results under a single viewpoint; determining whether the basic recognition results of multiple consecutive frames under each single viewpoint conform to preset business rules, generating a single-view result sequence; performing frame-level fusion of the visual features under each single viewpoint to obtain fused visual features of multiple frames under multiple viewpoints; determining whether the fused visual features of each frame conform to preset spatial rules, generating a multi-view result sequence; and performing smoothing and fusion processing on each single-view result sequence and the multi-view result sequence to generate the recognition result of the current video segment. This invention can fully utilize multi-view data to improve recognition accuracy.

The invention discloses a target hierarchical enhancement reconstruction method and reconstruction system based on a polarization camera array. The method comprises the following steps: acquiring a depth parallax model of the polarization camera array; collecting a multi-view target original view in multiple polarization directions; according to the target original view, performing occlusion layering based on a depth parallax model to obtain occlusion layering images of different depths; performing adaptive threshold segmentation, morphological processing and central viewpoint conversion on the shielding layered image of any depth to obtain a shielding mask of any viewpoint, further obtaining a shielding mask corresponding to a target layer, performing pixel-by-pixel reconstruction, and obtaining a calculation imaging result of the target layer after shielding is removed; and obtaining a final target enhanced reconstructed image through polarization state fusion. The image contrast and the layered sensing capability of the system under the shielding condition are improved, a higher-precision polarization imaging result can be generated, and the method is used for layered reconstruction of any object layer.

The invention relates to an exposure sequence adaptive optimization method and device for multi-view automatic measurement, computer equipment, a computer readable storage medium and a computer program product. The method comprises the following steps: acquiring an original exposure time sequence of a plurality of viewpoints; the original exposure time sequence comprises a plurality of exposure segments, and each exposure segment corresponds to one viewpoint; calculating similarity data among the exposure segments, and grouping the exposure segments according to the similarity data to obtain a plurality of initial exposure groups; for each initial exposure group, re-grouping the exposure segments at the boundary position to obtain a plurality of boundary grouping results, and calculating grouping cost data for each boundary grouping result; and selecting a target grouping result from the boundary grouping results according to the grouping cost data, optimizing the target grouping result according to the grouping cost data to obtain a plurality of target exposure groups, and calculating the exposure time of each target exposure group. The method can shorten the measurement period.

Items are identified in a waste stream for purposes of recycling, using deterministic and / or probabilistic techniques. Imagery of the waste stream from multiple viewpoints permit creation of a 3D depth draped image representation, from which one or more 2D planes can be synthesized. Phase-coherent patches of recoverable encoded data can be identified from among soiled and crumpled object surfaces, and used in combination to recover object identification information. Recognition of certain items can trigger further image processing that is specific to such items. (Detection of a catsup bottle, for example, can trigger image analysis to discern the presence of catsup residue.) Information about recognized objects can be provided to external data customers, e.g., to track grey market diversion of particular products into unlicensed territories. These and other features and advantages, which can be used alone or in combination, are detailed herein.

The invention discloses a three-dimensional point cloud reconstruction method and system based on ultra-wideband signal multi-view fusion. A commercial pulse wireless ultra-wideband radar is adopted to collect signals of an object under multi-view; background reflection reduction and effective distance box positioning are carried out on the reflection signals; the radar signal is subjected to data enhancement through virtual antenna offset and phase rotation; designing a signal multi-view depth reconstruction network model, extracting three-dimensional structure information of an object from a radar signal, and generating a multi-view object depth map; and performing inverse projection on the multi-view depth map, converting the depth map into local point cloud, and generating global three-dimensional reconstruction point cloud of the target object through point cloud fusion. The method provided by the invention still has relatively high robustness in a scene in which the object is shielded, and can be widely applied to the fields of logistics sorting, security check and the like.

The invention discloses a large-curvature thin-wall component cladding layer feature recognition method based on three-dimensional point cloud information, and the method comprises the steps: carrying out the filtering and simplification preprocessing of the collected point cloud data of a repaired blade profile at each visual angle, and obtaining the blade point cloud data at each visual angle; performing multi-view point cloud splicing on the blade point cloud data to obtain point cloud data of a repaired blade profile; calculating an initial point cloud normal vector and an initial point cloud curvature of each point position, and redirecting the initial point cloud normal vector to obtain a repaired blade initial point cloud normal vector consistent in direction; re-calculating the normal vector of the repaired blade point cloud to perform feature enhancement on the normal vector of the initial point cloud so as to obtain the normal vector feature enhanced repaired blade profile point cloud data; clustering the cladding layer features in each region to obtain point cloud data of the cladding layer in each region; and carrying out classification and combination on the cladding layer point cloud of each region by using Euclidean distance clustering to obtain completely spliced repaired blade cladding layer point cloud data.

We estimate a three-dimensional field capable of generating virtual viewpoint images that do not exhibit any noticeable inconsistencies in image quality when switching virtual viewpoints. [Solution] The image processing device 102 acquires multiple captured images obtained from multiple viewpoints and camera parameters corresponding to each viewpoint, acquires the shooting resolution which is the resolution of each captured image, generates multiple training images corresponding to each captured image by converting the resolution of each captured image based on the resolution of each of the multiple training images corresponding to each captured image which is determined based on the shooting resolution of each captured image, generates camera parameters corresponding to each training image by converting the camera parameters corresponding to each viewpoint based on the resolution of each training image, and learns a learning model concerning the three-dimensional field of the target space based on the multiple training images and the camera parameters corresponding to each training image.

The embodiment of the invention provides a video, image and 3D information generation method and device, a medium and a product, and belongs to the field of AI.The method comprises the steps that multiple to-be-edited images corresponding to multiple view angles are obtained, a to-be-edited area in a first to-be-edited image of an initial view angle is edited to obtain a first edited image, and the first edited image is edited to obtain a second edited image; and generating a plurality of sheltered second to-be-edited images corresponding to the other plurality of second to-be-edited images according to the to-be-edited area in the first to-be-edited image. And inputting the first edited image and the plurality of occluded second to-be-edited images into a target image restoration model to obtain a plurality of second edited images, wherein the target image restoration model diffuses an editing result in the first edited image to the plurality of occluded second to-be-edited images. And generating a video according to the plurality of edited images. Through the scheme, multi-view consistent image editing can be realized.

This application relates to a high-fidelity real-time rendering method, apparatus, device, and storage medium for dynamic digital humans. It includes receiving multimodal driving signals, mapping them to expression and pose parameters, inputting a parameterized human model to generate standard spatial geometry and skinning weights, and constructing a three-dimensional Gaussian sputtering field to deform to the target pose to obtain dynamic geometric data; arranging sparse virtual cameras in the target pose space, and simultaneously rendering and generating a sparse reference view set containing color and depth within a single rendering call; determining dense virtual viewpoints based on the display device's viewpoint parameters, and generating depth range textures for each dense viewpoint through downsampling depth reprojection and point sputtering techniques; determining ray step intervals using the depth range textures, sampling along the rays and fusing signed distance values ​​truncated from multiple viewpoints to generate surface position textures; projecting the surface positions onto sparse view sampled colors, fusing to generate a dense view color image, and adapting it for output to different display devices.

To acquire a three dimensional field capable of suppressing deterioration of reproducibility of an image of an object included in a virtual viewpoint image even when a position or a direction of a viewpoint used in learning is greatly different from a position of a virtual viewpoint or a direction of a line of sight at the virtual viewpoint.SOLUTION: The image processing apparatus 102 acquires a plurality of captured images obtained by image capturing from a plurality of positions and a plurality of camera parameters of a plurality of viewpoints corresponding to the plurality of positions. A camera parameter of an auxiliary viewpoint different from the plurality of viewpoints is generated, shape data of an object estimated based on the plurality of acquired camera parameters and the plurality of captured images is acquired, an auxiliary viewpoint image is generated based on the shape data and the generated camera parameter, and information regarding a three dimensional field corresponding to at least a partial space in an image capturing space captured from a plurality of positions is generated based on the plurality of acquired camera parameters, the plurality of captured images, the generated camera parameter, and the auxiliary viewpoint image.SELECTED DRAWING: Figure 10A

This invention belongs to the field of computer vision and action recognition, specifically relating to a robust multi-view skeleton fusion method based on information priority selection and effective joint masking. It includes: acquiring skeleton data from multiple viewpoints, performing data preprocessing and effective joint extraction; skeleton registration based on information content to obtain a registered skeleton; obtaining the skeleton most suitable for action recognition based on the registered skeleton and a GCN model; obtaining the action category based on the most suitable skeleton using a modern action recognition model; and adjusting model parameters using cross-entropy loss based on the action category results to optimize iterative recognition accuracy. This invention effectively solves the problem of low-quality noise interfering with high-fidelity data in multi-view fusion through an information content evaluation mechanism and a Boolean mask completion strategy; combined with GCN adaptive viewpoint adjustment, it enhances the model's robustness to different camera layouts and significantly improves the accuracy and stability of human action recognition in complex occlusion environments.

The embodiment of the invention provides a video stream processing method and device, equipment and a storage medium, and the method comprises the steps: obtaining a to-be-packaged video code stream which comprises a video coding parameter set and video coding data corresponding to a plurality of viewpoints; determining decoding parameter configuration information of each viewpoint according to the video coding parameter sets corresponding to the plurality of viewpoints respectively; generating a video parameter packaging packet of each viewpoint based on the decoding parameter configuration information of each viewpoint, and generating a video data packaging packet based on the video coding data corresponding to the plurality of viewpoints; and generating a video packaging file corresponding to the video code stream to be packaged according to the video parameter packaging packet and the video data packaging packet of each viewpoint. According to the technical scheme provided by the embodiment of the invention, the multi-view video can be packaged based on the video coding parameter set and the video coding data, and the packaged file meeting the transmission and storage requirements of the multi-view video is obtained, so that the method and the device can be suitable for more playing scenes of the multi-view video.

Various embodiments of the present disclosure relate generally to systems and methods for generating multi-view interactive digital media representations in a virtual reality environment. According to particular embodiments, a plurality of images is fused into a first content model and a first context model, both of which include multi-view interactive digital media representations of objects. Next, a virtual reality environment is generated using the first content model and the first context model. The virtual reality environment includes a first layer and a second layer. The user can navigate through and within the virtual reality environment to switch between multiple viewpoints of the content model via corresponding physical movements. The first layer includes the first content model and the second layer includes a second content model and wherein selection of the first layer provides access to the second layer with the second content model.

The application discloses a three-dimensional indoor scene generation method and related equipment, and the method comprises the steps of obtaining a text description of an indoor scene; generating an indoor layout through a preset diffusion model; converting the indoor layout into a two-dimensional height field and a two-dimensional semantic graph; predicting a neural radiance field based on the two-dimensional height field and the two-dimensional semantic graph; sampling multiple viewpoints in the predicted neural radiance field, rendering the multiple viewpoints, and obtaining an RGB-D image corresponding to each viewpoint, wherein RGB is color information of a two-dimensional image, and D is depth information of the two-dimensional image; constructing a truncated signed distance field body according to the RGB-D image corresponding to each viewpoint, generating a grid triangular facet based on the truncated signed distance field body, and obtaining an indoor three-dimensional scene. The application can generate a high-quality three-dimensional indoor scene without real multi-view images, improve the efficiency of generating a three-dimensional scene, and can be widely applied in the technical field of computer vision.

Example embodiments relate to techniques for volumetric performance capture with neural rendering. A technique may involve initially obtaining images that depict a subject from multiple viewpoints and under various lighting conditions using a light stage and depth data corresponding to the subject using infrared cameras. A neural network may extract features of the subject from the images based on the depth data and map the features into a texture space (e.g., the UV texture space). A neural renderer can be used to generate an output image depicting the subject from a target view such that illumination of the subject in the output image aligns with the target view. The neural render may resample the features of the subject from the texture space to an image space to generate the output image.

The invention discloses an interface display method and device, equipment and a medium, and belongs to the field of man-machine interaction. The method comprises the steps that a plurality of mutually independent content areas are displayed in a topic interaction interface, the area proportion of each content area in the plurality of content areas shows a first distribution condition, the plurality of content areas are in one-to-one correspondence with a plurality of viewpoints, the plurality of viewpoints are different viewpoints aiming at the same topic, and the first distribution condition shows the first distribution condition. Each content area in the plurality of content areas is used for accommodating the content published by the supporting object of the corresponding viewpoint; and in response to completion of the voting operation of the current object, displaying the plurality of updated content regions, the area proportion of each content region in the plurality of updated content regions presenting a second distribution condition, and the second distribution condition being associated with the quantity distribution condition of the voted object at the plurality of viewpoints. According to the method and the device, the information of the quantity distribution condition of each viewpoint support object is additionally provided for the user through the plurality of content areas, so that the information acquisition efficiency of the user is improved.

3D objects in the form of human faces are recognised at or adjacent an entrance to a building. Cameras are spaced horizontally from one another in an array to provide multiple viewpoints of a common scene with overlapping fields of view. For each of the cameras, a sequence of images of a face is generated and each of those images is normalised to a canonical view. One or more best normalised image for each camera is selected and feature data is extracted from the selected image. A recognition processor compares the extracted feature data of each selected image with stored, corresponding feature data of known 3D objects. A recognition result is output when the extracted feature data of at least one of the selected images corresponds to stored, corresponding feature data of at least one of the known 3D objects.

This document discloses a holographic communication method, electronic device, storage medium, and program product. The method includes: acquiring a three-dimensional model of a first object; the three-dimensional model is obtained based on multiple depth images and multiple attribute images, the multiple depth images and multiple attribute images being obtained based on multiple first images, the multiple first images being acquired from multiple viewpoints of the first object, the multiple attribute images including attribute data of multiple primitives under the multiple viewpoints, the primitives being basic units representing the first object; and rendering and displaying the three-dimensional model based on the three-dimensional model and the client's viewpoint. Therefore, high-fidelity stereoscopic image reconstruction can be achieved on the client using the acquired multi-view images, while ensuring viewpoint integrity.

A system and method for viewing aerial imagery and derived content on a web-based imagery browser that can be accessed by at least two users simultaneously. In addition, a system for interactively generating 3D geometry data for a location imaged from multiple viewpoints.