31 results about "Projector camera systems" patented technology

Aspects of the present invention include systems and methods for recalibrating projector-camera systems. In embodiments, systems and methods are able to recalibrate automatically the projector with arbitrary intrinsic and pose, as well as render for arbitrarily desired viewing point. In contrast to previous methods, the methods disclosed herein use the observing camera and the projector to form a stereo pair. Structured light is used to perform pixel-level fine reconstruction of the display surface. In embodiments, the geometric warping is implemented as a direct texture mapping problem. As a result, re-calibration of the projector movement is performed by simply computing the new projection matrix and setting it as a camera matrix. For re-calibrating the new view point, the texture mapping is modified according to the new camera matrix.

A method for self-recalibration of a structured light vision system including a camera and a projector. A camera plane and a projector plane are defined, a Homography matrix between the camera plane and the projector plane is computed, and a translation vector and a rotation matrix are determined from Homography-based constraints. A computer vision system implementing the method is also described.

Disclosed are embodiments of systems and methods for identifying features using color information in an image. The image may be formed from one or more display images comprising color information and features or feature components. Because color information may be used to identify features, more than one feature or feature component may be displayed in a display image. Because a plurality of features may be identified in a calibration image, an image system, such as a projector-camera system, can reduce the number of display images needed to calibrate the system.

A rigidly-coupled projector-camera system is augmented with laser pointers. The projector-camera system and lasers can be used to determine homography between the projector and a display surface, so that a projection of any desired geometry can be made on the display surface.

Light distortion due to light noise or light scattering object in the path of a light beam from a projector to a camera's image sensor in projector-camera systems are mitigated, or eliminated, by a simple modification to a light transport matrix T. For each row in light transport matrix T, matrix entries along a common row are compared, and all but the highest valued entry in the row are zeroed out, i.e. nullified. It is preferred that an array and record notation be used in order to reduce the size of the light transport matrix T, and its modified version to two arrays.

Two projected images from two projector-camera systems, each defined by characteristics c1=T1p1 and c2=T2p2, respectively, are used to create mosaic composite of a desired image c. Formulas p1=T1−1 (c−T2p2) and p2=T2−1(c−T1p1) are iteratively repeated with p2 initially set to zero until p1 converges to a first mosaic image and p2 converges to a second mosaic image.

The invention provides a color calibration optimization algorithm based on a projector-camera system, which is used for eliminating influence on immersion and reality sense of an observer caused by color difference of a display system of a large multi-projection screen. In the invention, for the problem of neglecting influence of a camera in the previous algorithm, a high dynamic range image (HDRI) synthesis algorithm based on Pedro image segmentation is proposed so as to establish a mapping relation between an input pixel and an output pixel of the camera; brightness and chrominance of a projector are calibrated at the same time by virtue of a combined function approximation method for a YUV (luminance and chrominance) color space transformation model so as to generate a color lookup table (CLUT); graphic processing unit (GPU) rendering for color calibration is realized based on shading language; and finally the three-channel annular multi-projection screen is taken as an experimental platform to verify effectiveness of the algorithm.

A rigidly-coupled projector-camera system is augmented with laser pointers. The projector-camera system and lasers can be used to determine homography between the projector and a display surface, so that a projection of any desired geometry can be made on the display surface.

The invention provides a real light source obtaining and relighting method based on images, belonging to the technical field of computer graphics. The method comprises the following steps that: step 1, a projector is utilized to project a basic light source by use of a light ray scanning method, so as to establish the corresponding relation between light rays of the basic light source and light rays reflected from an object surface; step 2, a homography matrix is utilized to establish the point-to-point corresponding relation between a visual angle of a camera and a visual angle image of the projector, and under the irradiation of the basic light source and a real light source, the camera respectively collects photos of a diffuse reflection plane with homogeneous materials, so as to establish the corresponding relation between the real light source and the basic light source; and step 3, basal images which are obtained in the step 1 and respectively correspond to the light rays are multiplied by the corresponding relation between the real light source and the basic light source, which is obtained in the step 2, and all images are superposed so as to obtain a result of relighting. The method utilizes a projector-camera system to realize the relighting of the real light source, and simplifies the collection of the real light source.

The invention discloses an augmented reality device applied to a stereotactic surgical robot and a method of the augmented reality device. A projector-camera system is composed of a camera and a projector, so that a surface point of an imaging target can be acquired, and color and texture information of the imaging target can also be acquired; after registration is carried out, the color and texture information of the surface of the imaging target is overlapped to surface image data obtained by nuclear magnetic imaging extraction, and a three-dimensional model which is more really and has colors and textures can be viewed on a computer; the camera and the projector are mounted at a tail end of a mechanical arm through connecting parts, so that a more complete three-dimensional surface of the imaging target is facilitated to obtain, and the registration accuracy is improved; on the basis of high-precision registration, a one-to-one corresponding relation of an image coordinate system and a mechanical arm base coordinate system is obtained, and basis is provided for subsequently calculating a position where the tail end of the mechanical arm needs to reach behind an appointed reality position to be augmented in the image coordinate system; augmented reality display on the surface of the imaging target is realized.

Aspects of the present invention include systems and methods for recalibrating projector-camera systems. In embodiments, systems and methods are able to recalibrate automatically the projector with arbitrary intrinsic and pose, as well as render for arbitrarily desired viewing point. In contrast to previous methods, the methods disclosed herein use the observing camera and the projector to form a stereo pair. Structured light is used to perform pixel-level fine reconstruction of the display surface. In embodiments, the geometric warping is implemented as a direct texture mapping problem. As a result, re-calibration of the projector movement is performed by simply computing the new projection matrix and setting it as a camera matrix. For re-calibrating the new view point, the texture mapping is modified according to the new camera matrix.

The invention discloses a projection display color reproduction method under normal lighting environment, which comprises three parts, namely projection display color reproduction characterization, equivalent reproduction target color value determination, and equivalent reproduction of color. The projection display color reproduction characterization is used for analyzing and describing color characteristics of a projector camera system, a display surface and environmental lighting; the equivalent reproduction target color value determination is used for analyzing and determining the reproduction capacity of the whole system under the current display environment, and obtaining a target color value in the reproduction capacity range and equivalent with the input original image color according to the reproduction capacity; and the equivalent reproduction of color is used for processing the target color value and obtaining an input color value required by the reproduction target color value according to the input / output color conversion relationship. The method does not depend on the measurement of other physical equipment, has low calculation cost and is favorable for developing the projection display to the portable and flexible mobile application direction.

Disclosed are embodiments of systems and methods for identifying features using color information in an image. The image may be formed from one or more display images comprising color information and features or feature components. Because color information may be used to identify features, more than one feature or feature component may be displayed in a display image. Because a plurality of features may be identified in a calibration image, an image system, such as a projector-camera system, can reduce the number of display images needed to calibrate the system.

The invention discloses a method for reproducing the projection display color of a colorful surface. The method comprises the steps of characterizing a projector camera system and a display surface color, determining a chroma color reproduction target value and calculating and realizing the color reproduction. The step of characterizing the projector camera system and a display surface color is used for representing the conversion relation of the input / output colors of the projector camera system and the color of the display surface; the step of determining the chroma color reproduction target value is used for setting a proper chroma reproduction target according to the input color value and a color reproduction range of the system on the colorful surface; and the step of calculating and realizing the color reproduction is used for correcting the color value of an original input image according to the conversion relation of the input / output colors of the system, the display surface color and the color reproduction target value so as to obtain the input color value. The system can automatically finish the entire chroma color reproduction process without needing any other measurement equipment or calibration color cards, the reproduction process is simple and hardware integration is easy to realize.

This invention describes a special type of drone called “Flying User Interface” device comprised of a robotic projector-camera system, an onboard digital computer connected with Internet, sensors, and a hardware interface to stick to any surface such as wall, ceilings, etc. Computer further consists of other subsystems, devices, and sensors such as accelerometer, compass, gyroscope, flashlight, etc. Drone flies from one places to another, detects a surface, and sticks to it. After successful sticking mechanism, device stops all its rotators and projects or augments images, information, and user interfaces on the near by surfaces. User interface may contain applications, information about object being augmented and information from Internet. User can interact with user-interface using command and gestures such as hand, body, feet, voice, etc.

A method for self-recalibration of a structured light vision system including a camera and a projector. A camera plane and a projector plane are defined, a Homography matrix between the camera plane and the projector plane is computed, and a translation vector and a rotation matrix are determined from Homography-based constraints. A computer vision system implementing the method is also described.

The invention relates to an adaptive illumination optimizing method adapting to a complicated illumination scene. The adaptive illumination optimizing method adapting to the complicated illumination scene comprises the following steps: establishing a projector-camera system; simulating a complicated illumination environment, selecting measured objects with different surface reflectance to preventoverexposure of an interference light source, and controlling a CCD camera by a computer to obtain an original object image; projecting reference stripes generated by the computer to an object througha projector, selecting black and white stripes of which the phase difference is between 0 and 2 pi as the reference stripes which can distort after the strips are deeply modulated by a three-dimensional object, and acquiring the distorted strips by a camera; recovering the height information of the object; carrying out inverse operation on the original object image, and projecting an inverse image to the surface of the object by a projector and keeping pixel matching; running an iterative algorithm until an iterative termination criterion is met to ensure that an overexposure portion is partially repaired; calibrating a camera response curve; and fitting a high-dynamic range image, namely a HDR image, according to the camera response curve, and obtaining and outputting a final result.

Application of dual photography is simplified by reducing the number of captured images needed to generate a light transport matrix T of (p×q) projection pixel array from (p×q) images to (p+q) images. Manipulation of the light transport matrix is also simplified by replacing the use of a matrix T with an index associating each projection pixel to only non-zero light transport values. By eliminating the use of zero-valued light transport coefficients, the memory and processing requirements for implementing dual photography are greatly reduced. This dual photography technique is applied to the calibration of projector-camera systems. A second method for calibrating projector-camera systems uses a known projection test pattern and a calibrated camera to associate projected markers on a real image to a captured image. Since the real location of the projected markers are determined by the camera, and the relative location of the markers on the test pattern is also known, the projector may be calibrated using homography techniques.

Light distortion due to light noise or light scattering object in the path of a light beam from a projector to a camera's image sensor in projector-camera systems are mitigated, or eliminated, by a simple modification to a light transport matrix T. For each row in light transport matrix T, matrix entries along a common row are compared, and all but the highest valued entry in the row are zeroed out, i.e. nullified. It is preferred that an array and record notation be used in order to reduce the size of the light transport matrix T, and its modified version to two arrays.

The invention provides an adaptive light compensation method based on a projector-camera system. The method comprises the steps that a projector and a camera are placed on the same side of a screen and arranged in a non-coaxial mode, and the screen serves as a reference plane; a sinusoidal grating stripe is generated to serve as a reference stripe, the reference stripe is projected to the surface of a three-dimensional object through the projector, the reference stripe will be distorted under depth modulation, and the reference grating stripe and the distorted grating stripe are acquired through the CCD camera; the relation between object depth and stripe phase change is derived; Fourier transformation is utilized to extract fundamental components of a distorted grating and a reference grating from a frequency domain, then Fourier inverse transformation is utilized to transfer the extracted fundamental components back to a time domain, phases folded in the interval from -pi to pi are obtained, and the folded phases in discontinuous distribution are changed into decompressed phases in continuous distribution in space through a phase decompression algorithm; the decompressed phases are substituted into a phase and depth conversion formula to obtain the depth of each pixel point; images collected by the camera are processed.

This invention introduces a mobile robotic arm equipped with a projector camera system, and computing device connected with interne and sensors, which can stick to any nearby surface using a sticking mechanism. Projector camera system displays the user interface on the surface. Users can interact with the device using the user interface using voice, body gestures, remote device, wearable or handheld device. We call this device “Stick Device” or “Stick User Interface”. In addition, the device can execute application specific tasks using reconfigurable tools and devices. With these functionalities, the device can be used for various human-computer interaction or human-machine interaction applications.

The invention relates to a projector network intelligent upgrading module and control platform system. The projector network intelligent upgrading module and control platform system comprises a PROCAMS (projector-camera system), a synchronization control bus and a control platform system. The PROCAMS comprises a projector, a digital camera, an FPGA and an embedded machine. Multiple PROCAMSs are connected through a control bus to form a projector network intelligent upgrading module. A control platform is cross platform software, can run on a PC or a tablet computer, and controls and coordinates each PROCAMS through the control bus. The main research content of the system comprises "a PROCAMS self-calibrating technology", "a multi-PROCAMS automatic stacking enhancement technology", "a multi-projection multi-camera free curved surface splicing fusion correction technology", "multi-PROCAMS control platform software development" and "a multi-projection display phase synchronization technology". According to the system provided by the invention, conventional low-level projectors with excessive production power are upgraded to high-quality extensible PROCAMS equipment, a high-end engineering projector can be replaced on many occasions, new applications can also be developed, and its application can promote upgrading, innovation and development of the projection product industry.

The invention discloses a projector out-of-focus correction method based on edge perception. According to the method, a projector-camera system is adopted, and a lens of a projector and a lens of a camera face the projection surface; inputting the special input image into a projector, projecting and irradiating the special input image onto a projection surface, and acquiring a projection result projected and irradiated onto the projection surface by the projector as an output image by a camera; and performing out-of-focus convolution kernel calibration by using the special input image and theoutput image to obtain a compensation picture, and performing correction compensation on the to-be-projected picture of the projector by taking the compensation picture as the input of the projector to obtain a clear projection result after out-of-focus compensation. The physical focal length of the projector does not need to be adjusted; according to the technical scheme, the clearness of the out-of-focus projection result of the projector can be realized only by processing the original picture input into the projector, the requirements of using the projector in complex application scenes such as thermal out-of-focus and large-amplitude height change of the projection surface of the projector can be met, and the applicability of the projector equipment is expanded to a certain extent.

A projector-camera system includes a projector coupled to back project a first image on a translucent diffusing screen. A camera is coupled to capture a second image from a back side of the translucent diffusing screen. The second image includes the first image back projected on the translucent diffusing screen and a shadow of a pointing device cast on a front side of the translucent diffusing screen. The pointing device is on the front side of the translucent diffusing screen and is in close proximity to the translucent diffusing screen. A processing block is coupled to the projector and the camera to generate a third image including the shadow of the pointing device. The processing block is further coupled to activate a command in a main computer coupled to the processing block in response to a relative position of the shadow of the pointing device in the third image.

The invention provides a real light source obtaining and relighting method based on images, belonging to the technical field of computer graphics. The method comprises the following steps that: step 1, a projector is utilized to project a basic light source by use of a light ray scanning method, so as to establish the corresponding relation between light rays of the basic light source and light rays reflected from an object surface; step 2, a homography matrix is utilized to establish the point-to-point corresponding relation between a visual angle of a camera and a visual angle image of the projector, and under the irradiation of the basic light source and a real light source, the camera respectively collects photos of a diffuse reflection plane with homogeneous materials, so as to establish the corresponding relation between the real light source and the basic light source; and step 3, basal images which are obtained in the step 1 and respectively correspond to the light rays are multiplied by the corresponding relation between the real light source and the basic light source, which is obtained in the step 2, and all images are superposed so as to obtain a result of relighting. The method utilizes a projector-camera system to realize the relighting of the real light source, and simplifies the collection of the real light source.

The invention discloses an augmented reality device applied to a stereotactic surgical robot and a method of the augmented reality device. A projector-camera system is composed of a camera and a projector, so that a surface point of an imaging target can be acquired, and color and texture information of the imaging target can also be acquired; after registration is carried out, the color and texture information of the surface of the imaging target is overlapped to surface image data obtained by nuclear magnetic imaging extraction, and a three-dimensional model which is more really and has colors and textures can be viewed on a computer; the camera and the projector are mounted at a tail end of a mechanical arm through connecting parts, so that a more complete three-dimensional surface of the imaging target is facilitated to obtain, and the registration accuracy is improved; on the basis of high-precision registration, a one-to-one corresponding relation of an image coordinate system and a mechanical arm base coordinate system is obtained, and basis is provided for subsequently calculating a position where the tail end of the mechanical arm needs to reach behind an appointed reality position to be augmented in the image coordinate system; augmented reality display on the surface of the imaging target is realized.

The invention provides a color calibration optimization algorithm based on a projector-camera system, which is used for eliminating influence on immersion and reality sense of an observer caused by color difference of a display system of a large multi-projection screen. In the invention, for the problem of neglecting influence of a camera in the previous algorithm, a high dynamic range image (HDRI) synthesis algorithm based on Pedro image segmentation is proposed so as to establish a mapping relation between an input pixel and an output pixel of the camera; brightness and chrominance of a projector are calibrated at the same time by virtue of a combined function approximation method for a YUV (luminance and chrominance) color space transformation model so as to generate a color lookup table (CLUT); graphic processing unit (GPU) rendering for color calibration is realized based on shading language; and finally the three-channel annular multi-projection screen is taken as an experimental platform to verify effectiveness of the algorithm.

The invention discloses a method for reproducing the projection display color of a colorful surface. The method comprises the steps of characterizing a projector camera system and a display surface color, determining a chroma color reproduction target value and calculating and realizing the color reproduction. The step of characterizing the projector camera system and a display surface color is used for representing the conversion relation of the input / output colors of the projector camera system and the color of the display surface; the step of determining the chroma color reproduction target value is used for setting a proper chroma reproduction target according to the input color value and a color reproduction range of the system on the colorful surface; and the step of calculating andrealizing the color reproduction is used for correcting the color value of an original input image according to the conversion relation of the input / output colors of the system, the display surface color and the color reproduction target value so as to obtain the input color value. The system can automatically finish the entire chroma color reproduction process without needing any other measurement equipment or calibration color cards, the reproduction process is simple and hardware integration is easy to realize.