61 results about "Spherical image" patented technology

The invention relates to the technical field of panoramic video processing, in particular to a multi-modal fusion panoramic video intelligent content analysis method, which comprises the following steps of: acquiring a panoramic video frame and a synchronous audio thereof; performing spherical projection transformation on the panoramic image data, and converting the equidistant projection image into a spherical image; generating a plurality of plane images based on the spherical image; performing candidate target position detection based on the plurality of plane images; based on a candidate target position detection result, an image target detection model is utilized to realize instance segmentation of a target, and target features are extracted to generate a target embedded description vector sequence; based on the synchronous audio, an audio recognition processing module is used for carrying out time sequence information extraction, and an acoustic attention sequence is formed; a spectrum sensor is adopted to realize association fusion of the target embedded description vector sequence and the acoustic attention sequence, and an audio and video collaborative attention sequence is generated; and based on the audio and video collaborative attention sequence, establishing an audio and visual event identification model, and obtaining event content described by the panoramic content.

A computing device is provided. The computing device includes a memory configured to store one or more instructions; and a processor configured to execute the one or more instructions stored in the memory, in which the processor may execute the one or more instructions to generate a first predicted depth map image for a learning red / green / blue (RGB) image using a neural network, and train the neural network based on a difference between a learning depth map image having depth information and the first predicted depth map image. The learning depth map image may match 1:1 and may be generated based on the same setting change for a basic spherical image.

Auto exposure processing for spherical images improves image quality by reducing visible exposure level variation along a stitch line within a spherical image. An average global luminance value is determined based on luminance values determined for first and second images, which are based on auto exposure configurations of first and second image sensors used to obtain those first and second images. Delta luminance values are determined for the first and second images using the average global luminance value. The first and second images are then updated using the delta luminance values, and the updated first and second images are used to produce a spherical image.

The embodiment of the invention discloses a camera group calibration method and device, equipment and a medium, and relates to the technical field of camera calibration. The method comprises the following steps: acquiring a calibration object image of a to-be-calibrated object in a candidate camera coordinate system, and determining candidate calibration object coordinates in the candidate camera coordinate system according to the calibration object image; judging whether the candidate camera coordinate system is a reference camera coordinate system or not, and if not, converting the candidate calibration object coordinates under the candidate camera coordinate system into reference calibration object coordinates under the reference camera coordinate system according to a preset camera group transformation matrix; according to the reference calibration object coordinate and a preset calibration error function, determining corresponding calibration compensation data, and according to the reference calibration object coordinate and the calibration compensation data, determining a target calibration object coordinate of the object to be calibrated; wherein the calibration error function is determined based on sphere images of the calibration sphere at different acquisition positions in the view field overlapping area of the camera group and a preset sphere radius. And the space consistency and robustness of camera group calibration are improved.

Provided are a depth map generation method, a device and a storage medium, which belong to the technical field of image processing. The method includes: generating, according to a first spherical image acquired by a first fisheye lens and a second spherical image acquired by a second fisheye lens, a first disparity map of a spatial region where a terminal device is located; generating a second disparity map of the spatial region according to depth information of the spatial region acquired by an active depth sensor; and generating a target depth map of the spatial region according to the first disparity map and the second disparity map.

The invention discloses a spherical panoramic image feature matching method and system, a terminal and a storage medium, and the method comprises the steps: obtaining a spherical panoramic image, and carrying out the preprocessing of the spherical panoramic image; screening the preprocessed spherical panoramic image based on a spatial constraint model, and constructing a scene graph based on the screened spherical panoramic image; segmenting the scene graph into a plurality of sub-scene graphs, and performing priority ranking on the sub-scene graphs based on a graph sequence scheduling algorithm; and performing geometric correction and feature extraction on all the screened spherical panoramic images, and executing a feature matching task of the spherical panoramic images in the sub-scene images according to the priority ranking of the sub-scene images. Through a spatial constraint model, a graph order scheduling algorithm, geometric correction and a cascade Hash matching algorithm, the problem that feature matching efficiency and precision are difficult to consider when a spherical image feature matching technology processes large-scale high-resolution spherical images is solved.

Embodiments of the present application provide three-dimensional scene reconstruction method, apparatus, device, and a storage medium. The method comprises the following steps: constructing a neural radiance field for a three-dimensional scene based on a multi-view image sequence in the three-dimensional scene; for each given point location center, determining ray sampling points at the point location center and color information of the ray sampling points based on the neural radiance field; performing multi-layer rendering on the color information of the ray sampling points to obtain a multi-sphere image reconstructed for the three-dimensional scene at the point location center.

The present disclosure relates to live streaming methods and apparatuses, an electronic device, and a storage medium. A method comprises: acquiring a source live video stream, and acquiring the video projection format of the source live video stream; when the video projection format of the source live video stream is a first projection format, converting the video projection format of the source live video stream into a second projection format to obtain a target live video stream, the first projection format being used for projecting a spherical image into a rectangular image, the second projection format being used for projecting a spherical image to each face of an unfolded cube, and the cube being circumscribed about the spherical image; and pushing the target live video stream to a terminal.

The invention is applicable to the technical field of image processing, and provides an image processing method, the image processing method is applied to terminal equipment, the terminal equipment is provided with a display screen, and the image processing method comprises the following steps: controlling the display screen to display a local spherical surface of a target ball, and controlling the display screen to display an image picture corresponding to the target position in the panoramic spherical image, if the touch operation on the target ball is detected, adjusting the target position according to the touch operation; and controlling a display screen to display an image picture corresponding to the adjusted target position. Therefore, the terminal device can finely adjust the target position and further control the display screen to display the image corresponding to the adjusted target position, so that the user can see the expected image.

A system for capturing omnidirectional imaging data includes a sensor unit, an array of two or more digital cameras, and a processor. The digital cameras are each adapted for capturing subspherical imaging data. The processor is operatively connected to the array of digital cameras and is adapted for combining the subspherical imaging data into spherical imaging data. The sensor unit is detachably mounted to an intermediate mounting member. The intermediate mounting member includes a substantially hollow body and a battery pack. The intermediate mounting member is detachably mounted to a base bracket.

In order to efficiently estimate the position of an object in a three-dimensional space, a position estimation system includes a position estimation device and three imaging devices. The three imaging devices are arranged such that imaging positions of the imaging devices form a triangle. The position estimation device acquires images from the three imaging devices, performs transformation from image coordinates to stereo spherical image coordinates, then determines whether or not the epipolar constraint is satisfied in each pair of the stereo spherical image coordinates, and, if the epipolar constraint is satisfied, estimates that such coordinate points correspond to the coordinate position of an object.

Devices and methods related to capturing 360 degree spherical images are provided. A computing device can capture, substantially simultaneously, a first image with the first image sensor and a second image with a second image sensor. The first image sensor can be positioned on a first side of the computing device and oriented at a first orientation with respect to an axis of rotation. The second image sensor can be positioned on a second side of the device substantially opposite the first side of the device and oriented at a second orientation that is axially rotated from the first orientation with respect to the axis of rotation. The computing device can stitch together the first image and the second image to create an output image that captures a 360 degree field of view with respect to the computing device.

A system may obtain a plurality of experimental diffraction patterns. A system may identify a crystallographic orientation of each diffraction pattern of the plurality of experimental diffraction patterns. A system may build one or more pole figures. A system may select a reference spherical function having a reference frame. A system may correlate spherical images from the pole figures and the reference spherical function. A system may determine a sample frame of the crystallographic orientations. A system may rotate the crystallographic orientations of the plurality of experimental diffraction patterns into alignment with the reference frame to produce a plurality of rotated crystallographic orientations.

Obtaining a processed spherical image includes obtaining a first luminance thumbnail image for a first hemispherical input image, obtaining a second luminance thumbnail image for a second hemispherical input image, obtaining a first distortion correcting weight map, obtaining a second distortion correcting weight map, obtaining, as an aggregate weighted mean value for the input spherical image, an aggregate of a first normalized weighted mean value for the first luminance thumbnail image and a second normalized weighted mean value for the second luminance thumbnail image, obtaining at least one of a target exposure value, a target aggregate gain value, or a region of interest ratio value in accordance with the aggregate weighted mean value, and obtaining the processed spherical image from the input spherical image in accordance with at least one of the target exposure value, the target aggregate gain value, or the region of interest ratio value.

Embodiments of the present invention disclose a panoramic image processing method and a panoramic camera, which are used to improve the anti-shake effect of the panoramic camera. The method of the embodiments of the present invention includes: obtaining a panoramic image, the acquisition timestamp of the pixels of the panoramic image, and a plurality of rotation matrices. Converting the panoramic image into a spherical image, and for each pixel of the spherical image, respectively performing position correction using the plurality of rotation matrices to obtain a plurality of corrected pixels for each pixel. For each pixel of the spherical image, according to the acquisition timestamp of the pixels of the spherical image, determining a first corrected pixel and a second corrected pixel from the plurality of corrected pixels. For each pixel of the spherical image, performing weighted average calculation using the first corrected pixel and the second corrected pixel to obtain a target corrected pixel. Through the panoramic image processing method of the embodiments of the present invention, each pixel of the spherical image can be corrected to the position where the pixel should be at the initial moment as much as possible, thereby achieving a good anti-shake effect.

Obtaining a processed spherical image includes obtaining a first luminance thumbnail image for a first hemispherical input image, obtaining a second luminance thumbnail image for a second hemispherical input image, obtaining a first distortion correcting weight map, obtaining a second distortion correcting weight map, obtaining, as an aggregate weighted mean value for the input spherical image, an aggregate of a first normalized weighted mean value for the first luminance thumbnail image and a second normalized weighted mean value for the second luminance thumbnail image, obtaining at least one of a target exposure value, a target aggregate gain value, or a region of interest ratio value in accordance with the aggregate weighted mean value, and obtaining the processed spherical image from the input spherical image in accordance with at least one of the target exposure value, the target aggregate gain value, or the region of interest ratio value.

Embodiments of the present specification provide a panoramic image processing method and device, the method comprising inputting a target panoramic image into an image processing model, obtaining panoramic image features of the target panoramic image through an encoding layer of the image processing model; converting the panoramic image features into spherical image encoding features according to a panoramic image spherical conversion algorithm; inputting the spherical image encoding features into a decoding layer of the image processing model, and obtaining a spherical depth image of the target panoramic image through the decoding layer; and converting the spherical depth image into a panoramic depth image of the target panoramic image according to the panoramic image spherical conversion algorithm. The method obtains a depth image of a panoramic image by using an image processing model, saves costs without additional instruments and equipment; and runs an image processing model specially designed for a sphere on a sphere, solves distortion generated in an ERP projection process of a panoramic image, and improves the accuracy of obtaining a depth image of a panoramic image.

The embodiment of the present application provides a kind of depth map generation method, equipment and storage medium, belong to image processing technical field.The method comprises: according to the first spherical image of the first fish-eye lens collected and the second spherical image of the second fish-eye lens collected, the first parallax map of the space region where the terminal device is located is generated;And according to the depth information of the space region collected by the active depth sensor, the second parallax map of the space region is generated;Then according to the first parallax map and second parallax map, the target depth map of the space region is generated.The technical scheme of the present application can improve the field view angle of depth image.

A three-dimensional reconstruction image production program is provided that can achieve both efficient data collection and high accuracy of the reconstruction model in three-dimensional reconstruction. [Solution] The image production program for three-dimensional restoration disclosed herein includes an exclusion criterion detection procedure, an image exclusion procedure, and a restoration image output procedure, wherein the exclusion criterion detection procedure detects exclusion criteria from a spherical image, and the exclusion criteria are objects included in an image combination area of ​​the spherical image, the image exclusion procedure excludes a predetermined area from the spherical image based on the exclusion criterion, and the restoration image output procedure outputs the image from which the predetermined area has been excluded as a three-dimensional restoration image.

A system may obtain a plurality of experimental diffraction patterns. A system may identify a crystallographic orientation of each diffraction pattern of the plurality of experimental diffraction patterns. A system may build one or more pole figures. A system may select a reference spherical function having a reference frame. A system may correlate spherical images from the pole figures and the reference spherical function. A system may determine a sample frame of the crystallographic orientations. A system may rotate the crystallographic orientations of the plurality of experimental diffraction patterns into alignment with the reference frame to produce a plurality of rotated crystallographic orientations.

The invention relates to the field of computer vision and image processing, in particular to a panoramic image editing method and system for spherical geometry, and the method comprises the steps: obtaining a to-be-edited panoramic image and an editing instruction of a user, drawing a mask of a user editing dragging region on the panoramic image, and marking a control point and a target point; rotating an area edited by the user to the center of the spherical image; image editing is carried out on the panoramic image after rotation transformation by using a trained point editing diffusion model, and the point editing diffusion model adaptively adjusts an optimization area and an editing track according to spherical latitude information of a control point; and outputting the edited panoramic image. Compared with the prior art, the method has the advantages that the distortion area of the panoramic image is effectively corrected, editing path planning with higher consistency in the panoramic space is realized, and the problem that an existing editing method is difficult to apply to the spherical image is solved.

A mechanism for processing video data is disclosed. The mechanism includes determining to process multi-sphere image (MSI) based video with a two-dimensional (2D) codec, wherein multi-sphere images pertaining to a particular time instance are converted between a set of sphere images and projected 2D images acting as MSI layers, and wherein the projected 2D images are frame-packed. A conversion is performed between a visual media data and a bitstream based on the 2D codec.

An isometric image construction method suitable for standard CNN network processing, which is characterized by a large number of perspective images and a variety of types, is used in the semantic segmentation of panoramic images, the two-dimensional plane representation form of the panoramic image is called isometric image, the conversion relationship between spherical image and tangent plane image and the conversion relationship between spherical image and isometric image are used to realize the conversion of perspective image to isometric image, the best projection position of the perspective image data set converted into the isometric image data set is given by taking the isometric image semantic segmentation as an example, and compared with other methods of processing isometric image, it is verified that the method has better performance.

The present invention belongs to the field of image processing technology, and specifically relates to a low-complexity panoramic video encoding method that considers the propagation of time-domain distortion. The present invention provides a panoramic video encoding method that considers the propagation of time-domain distortion. The present invention analyzes the inter-frame reference relationship under the random access coding structure and establishes a corresponding distortion propagation chain, and uses the distortion influence factor of the current coding unit to adjust the coding parameters to achieve the goal of optimizing the coding. According to the problem of oversampling caused by different degrees of geometric deformation of the spherical image of the panoramic video in different latitudes during the projection process, the present invention adjusts the coding parameters according to the area change ratio of different latitudes. It can greatly improve the coding performance.

Various aspects of visual servoing based on use of a generative diffusion model are described. One technique includes obtaining a current view image from a robot camera that captures a portion of an environment of the robot, and obtaining a spherical image from a previous view that depicts a surrounding area of the environment of the robot. The current view image is combined with the previous spherical image to produce a modified spherical image, and inpainting is performed in a mask area of the modified spherical image with a generative diffusion model. The inpainting blends the current view image with the previous spherical image, providing a view of the entire environment of the robot that can be used for visual servoing. The robot can perform path planning and visual servoing of an end effector based on the modified spherical image, independent of any use of an external positioning or localization system.

The present application is applicable to the technical field of image processing, and provides an image stitching method, applied to a panoramic camera. The panoramic camera is provided with a first fisheye lens and a second fisheye lens. The image stitching method comprises: acquiring a first fisheye image by means of the first fisheye lens, and acquiring a second fisheye image by means of the second fisheye lens (S101); mapping the first fisheye image onto a unit spherical surface to obtain a first spherical image, and mapping the second fisheye image onto the unit spherical surface to obtain a second spherical image (S102); rotating the first spherical image by a specified angle to obtain a rotated first spherical image, and rotating the second spherical image by the specified angle to obtain a rotated second spherical image (S103); unwrapping the rotated first spherical image into a first ERP image, and unwrapping the rotated second spherical image into a second ERP image (S104); and stitching the first ERP image and the second ERP image to obtain a target image (S105). Thus, the image stitching accuracy is effectively improved.

To provide an image display device that can display a spherical image (including a spherical video image) and is easy to assemble, disassemble and move. The display device comprises a plurality of display units 2 and 3 each having the outline of a regular polygon that constitutes all or part of a polyhedron 1 surrounded by only two or less types of regular polygons, multiple Display unit Each of the a display surface corresponding to the inner peripheral surface of the polyhedron 1 and on which input image data is displayed; The relevant a connecting means for connecting the outer edges of the display units 2 and 3 to and detaching from the outer edges of other adjacent display units; The relevant and a position specifying means S for specifying the position of the display unit on the polyhedron, and each of the display units 2, 3 corresponds to the position of the display unit 2, 3 specified by the position specifying means. do Image data own Display it on the display surface.

A machine is configured to generate a representation of a rectangular image, which may depict a projection of a spherical image. The machine generates a first triangular array of pixels based on an upper rectangular region of the rectangular image. The machine generates a second triangular array of pixels based on a lower rectangular region of the rectangular image, and the machine generates a rectangular array of pixels based on a middle rectangular region of the rectangular image. The machine then generates an output image that includes the first triangular array of pixels, the second triangular array of pixels, and the rectangular array of pixels. After the output image is generated, the machine may provide the generated output image for use as a representation of rearranged image data of the spherical image.

The invention provides an image processing method based on a spherical convolutional neural network, and the method comprises the following steps: projecting input two-dimensional image data to a spherical substrate through an inverse north pole projection method, and forming a spherical image; mapping the spherical image into a spherical grid; and carrying out spherical convolution calculation according to the image in the spherical grid, specifically, converting the image in the spherical grid from a spatial domain to a frequency domain space through generalized Fourier transform, fitting the image and a convolution kernel in the spherical grid by adopting a spherical harmonic function and a Wigner D matrix as orthogonal bases, and carrying out convolution calculation on the image in the spherical grid. Performing inner product summation operation in the frequency domain space to complete spherical convolution calculation; and reducing a frequency domain convolution result to a geometric space through inverse Fourier transform to obtain an output feature map.