74 results about "Light-field camera" patented technology

The invention belongs to the technical field of three-dimensional imaging, and particularly relates to an active light field three-dimensional imaging method based on phase guidance. The phase information is introduced into the light field stereo matching process, and the spatial resolution loss of the light field camera is compensated through phase guidance measurement, so that the bottleneck problem that the existing light field camera is limited by the sub-aperture resolution to cause the low resolution of the depth map is solved. Based on effective fusion of the light field and the structured light, three-dimensional reconstruction with high pixel resolution can be realized in single shooting, the robustness and precision of depth estimation can be ensured, and the method has wide application potential in the fields of precision measurement, industrial detection, three-dimensional modeling and the like.

The invention discloses an additive manufacturing process high dynamic range imaging method and system based on a multi-view light field. Firstly, an original light field image in the additive manufacturing process is collected through a light field camera; extracting R, G and B single-wavelength channel images from the original light field image by the established light field Bayer model; then, taking the R-channel image as a reference, and adopting a K-means clustering algorithm for segmentation to obtain a low-light area and a high-light area in the R-channel image; and finally, fusing the pixel value of the G channel image corresponding to the low-light area with the R channel image to enhance weak light details, and fusing the pixel value of the B channel image corresponding to the highlight area with the R channel image to suppress highlight, thereby reconstructing to obtain an HDR image. According to the method, the image quality and the information integrity monitored in the additive manufacturing process are effectively improved, and high-quality visual data are provided for additive manufacturing process optimization and defect recognition.

The invention provides a robot touch and vision fused double-loop coupling self-adaptive grabbing control method, which relates to the technical field of robot grabbing control, and comprises the following steps of: acquiring surface characteristic data through a light field camera array, acquiring dynamic contact information and extracting stability characteristics by a sensing array, and acquiring the surface characteristic data through a touch sensor array; cross-modal coupling is constructed based on Tucker decomposition, tactile and visual information is fused, mapping parameters are established, an adaptive time window is constructed to optimize grabbing parameters, and the grabbing posture and the clamping force are adjusted in combination with a Lyapunov stability criterion and a dynamic compensation mechanism. According to the method, the grabbing stability and robustness of the robot in a complex environment can be improved.

The invention discloses a light field time domain super-resolution method based on an event camera, and belongs to the field of optical engineering. A light field camera and an event camera are used for collecting light field event data, decoding the light field data, denoising the event data and generating an event image, the light field data and an event grayscale image are subjected to joint calibration in a sub-aperture image mode, joint optimization is carried out through calibration results of multiple viewpoints, a checkerboard is shot, and a target object is obtained. And obtaining an affine transformation matrix and a perspective transformation matrix, and obtaining a super-resolution result of projecting the event image onto a light field image plane. Starting from the expression mode of the light field macro pixel, the method has applicability to light field data of any macro pixel size; compared with a traditional time domain super-resolution method, the time domain super-resolution of the light field camera at any frame rate can be realized by segmenting the event stream at different scales.

This invention discloses a method, device, storage medium, and program product for calibrating and aligning a light field camera and an RGB camera. The deviation value is calculated by matching the images from the two cameras and fed back to the engineer, allowing for more precise fine-tuning. The deviation value provided by this invention is accurate and objective, unaffected by human factors. With objective guidance, engineers can significantly improve the efficiency of fine-tuning camera positions, thereby enhancing the efficiency of this task in actual production assembly.

The present invention discloses a simulation system and method for a high-dimensional light field event camera. The simulation method comprises the following steps: (1) constructing a world coordinate system, building a three-dimensional object model to be simulated, and setting the object's motion state to meet the dynamic scene requirements of a dynamic vision sensor; (2) constructing an optical camera model in front of the three-dimensional object model according to a focused Kepler light field structure, so that the light field camera can capture the angle and position of light and obtain high-dimensional light field information including the light position and incident angle; (3) mapping the obtained high-dimensional light field information to a two-dimensional plane to form an image frame, simulating it as an event according to the brightness difference between each frame, and finally outputting a light field event stream. The present invention can accurately simulate the light field event stream, avoid errors caused by processes such as microlens array calibration in actual hardware construction, and provide a simple and feasible method for the production of large-scale light field event data sets.

The invention belongs to the technical field of computer vision and light field imaging, and discloses a large-area highlight detection and repair method and system. The method comprises the following steps: imaging by using a light field camera array to obtain a multi-view-angle light field image, and calculating an initial parallax; the method comprises the following steps: obtaining a multi-view highlight area mask through highlight detection, removing the highlight area mask to obtain a light field image without a highlight area, carrying out light field refocusing to obtain a non-highlight refocusing image stack, applying the highlight area mask obtained through central view angle detection to an initial parallax image, and constructing a reliable parallax set to realize parallax layering; according to parallax layer information of a non-highlight area pixel adjacent to a highlight pixel space in the highlight area, carrying out adaptation on a parallax layer of the highlight pixel; and constructing a hierarchical restoration strategy based on observation conditions of different viewpoints when no-highlight refocusing image of the highlight pixel in the corresponding parallax layer is superposed and averaged at the pixel position, thereby realizing restoration of the highlight area. According to the method, high-quality and large-area highlight repair can be realized.

Provided is an imaging module having a biometric authentication function, which is suitable for a wearable electronic device, is thin, and can be made compact. An imaging module is included in a wearable electronic device, the imaging module including: a light emitting element that is provided to face a measurement target side in a worn state and that emits light to a measurement target; a light field camera that is provided to face the measurement target side in a worn state and that images the light emitted from the light emitting element to the measurement target; and a processor configured to process an image obtained by imaging the light emitted from the light emitting element to the measurement target using the light field camera, in which the light field camera includes a liquid crystal lens array and an image sensor array in order from the measurement target side.

The invention belongs to the technical field of optical camera products, and provides a high-duty-ratio micro-lens array preparation method and a light field camera. The method comprises the following steps: S1, preparing a photoetching mask plate with a high duty ratio, and performing contact photoetching processing to obtain a spin coating chromium plate photoetching mask plate; s2, cleaning the micro lens array substrate; s3, spin-coating a hydrophobic layer; s4, patterning the sacrificial layer; s5, performing plasma ashing and selective surface modification; s6, removing the part, which is not modified, of the photoresist; and S7, performing blade coating of optical cement to form a micro lens, thereby obtaining a wettability micro lens array with a high duty ratio. By adopting the cheap CMOS image sensor and by virtue of the advantages of the wettability micro-lens array with high duty ratio, the light field camera is constructed, so that the brightness, the signal-to-noise ratio, the spatial resolution and the three-dimensional perception capability of a picture shot by the light field camera are remarkably improved, and accurate analysis of the image is facilitated; the method is used in the fields of automatic driving, cultural relic digitization, industrial detection, medical images and the like.

The application provides a robot tactile and visual fusion double-ring coupling adaptive grasping control method, relates to the technical field of robot grasping control, and comprises the following steps: acquiring surface characteristic data through a light field camera array, collecting dynamic contact information and extracting stability features by a sensor array, constructing cross-modal coupling and fusing tactile and visual information based on Tucker decomposition, establishing mapping parameters and constructing an adaptive time window to optimize grasping parameters, and adjusting grasping posture and clamping force in combination with Lyapunov stability criterion and a dynamic compensation mechanism. The method can improve the stability and robustness of robot grasping in a complex environment.

The application discloses a three-dimensional light-thermal parameter reconstruction method and system for a combustion process, and belongs to the field of combustion diagnosis.The application solves the problems of low measurement accuracy in a high-temperature environment and poor accuracy of extracting effective signals in a complex background existing in the prior art light field parameter measurement.The application adopts a polarization spectrum camera and a light field camera to synchronously perform multi-angle data acquisition on a combustion area, and acquires polarization spectrum data and light field data of the combustion area;light field depth information is extracted from the light field data, a three-dimensional structure of the combustion area is constructed by combining the light field depth information with a light field reconstruction algorithm, spectral analysis is performed on the polarization spectrum data, and polarization degree and polarization angle information of the polarization spectrum are acquired;the three-dimensional structure is combined with a CT reconstruction algorithm; a three-dimensional distribution field of particulate matters in the combustion process is acquired; and based on the polarization spectrum information and the preprocessed light field data, the three-dimensional temperature field and the concentration field distribution of gases in the combustion area are calculated by using the optimized three-dimensional structure.The application is mainly used for combustion product parameter acquisition.

The invention discloses a tea bud picking obstacle avoidance path planning method and system based on a multi-modal information fusion diffusion model. A tea tree canopy image is acquired through a light field camera, a dense three-dimensional point cloud is constructed, tea bud target detection and positioning are realized in combination with an improved YOLOV8 and a PointNet + + network, a path generation method based on a U-net architecture condition denoising diffusion model is provided, path planning is modeled as a condition track generation problem, and the problem is solved. According to the method, multi-modal condition information input and a target guiding steering function are combined, multi-step denoising iteration is executed, a continuous, smooth, collision-free, dynamic and feasible mechanical arm picking path is directly generated, the sawtooth-shaped defect and post-optimization dependence of a path generated through traditional sampling algorithms such as RRT are avoided, and the path quality and execution stability are improved. The system comprises a light field acquisition module, a three-dimensional reconstruction module, a visual positioning module, a path planning module, an execution control module and the like, realizes accurate obstacle-avoiding picking of tea buds, is suitable for path planning of picking operation of tea leaves and other fruits and vegetables, and provides technical support for agricultural picking and harvesting.

The invention discloses a naked-eye 3D imaging method, and relates to the technical field of naked-eye 3D, and the method comprises the steps: employing a binocular camera or a light field camera to obtain multi-angle data, enabling the distance between cameras to change according to a target distance through dynamic baseline adjustment, increasing a baseline in a long distance to improve the depth-of-field resolution, and reducing the baseline in a short distance to enhance the local stereoscopic impression; meanwhile, in combination with an AI depth-of-field prediction model, a low-light environment and a high-reflectivity surface are subjected to deep optimization treatment, and errors caused by light interference are reduced; in the depth calculation process, an optical flow analysis method is adopted, a pixel motion vector field is calculated through a Horn-Schunck optical flow, the stability of optical flow calculation is optimized in combination with Kalman filtering, parallax matching or optical field reconstruction is performed based on an optical flow calculation result, for a binocular camera mode, a block matching method is adopted to calculate parallax, and the optical field is reconstructed. And in the light field camera mode, depth reconstruction is carried out by using an EPI polar plane parallax estimation algorithm.

The present application belongs to the technical field of camera calibration, and discloses a focusing light field camera geometric parameter calibration method, which comprises the following steps: using a focusing light field camera to collect and acquire a white image and a corresponding checkerboard original light field image under different object distances; determining the focal plane of the camera based on the original light field image; using an edge diffusion function information fitting-based corner point detection algorithm to calculate the checkerboard corner point coordinate information in the original light field image on the focal plane; calculating the center coordinate information of each microlens in the microlens array of the camera and the size information of a single microlens in the microlens array based on the white image; calculating the pixel block size in the microlens sub-image taken by the center sub-aperture image according to the above information; and calculating the distance from the virtual imaging surface to the microlens array and the distance from the microlens array to the plane where the photosensitive element is located based on the pixel block size and in combination with a Gaussian imaging formula. The present application can improve the calibration accuracy of the geometric parameters of the focusing light field camera.

The invention discloses a spectrum optimization-based double-layer medium wide spectrum parameter simultaneous reconstruction method, relates to the technical field of spectral imaging, and aims to solve the problems that when wide optical parameter reconstruction is carried out based on a multi-layer medium model in the prior art, detection radiation optical signals are limited by a traditional contact type measurement mode, and the reconstruction precision is low. On the basis of a traditional participating medium optical parameter measurement and reconstruction method, a light field camera is introduced to more accurately detect a reflected radiation measurement signal of a medium boundary; a reconstruction optimization algorithm of searching an initial point by a steepest descent method and searching an optimal solution by a conjugate gradient method is combined, so that the efficiency of accurate reconstruction of the double-layer medium is greatly improved, the limitation of a traditional contact measurement mode is broken through, and the reconstruction precision is improved; in view of the problem that a radiation measurement signal is extremely low in sensitivity to a reduced scattering coefficient of a bottom-layer medium, prior information of a spectrum dimension is introduced, reconstruction results of an absorption coefficient and a reduced scattering coefficient of a double-layer medium, especially the bottom-layer medium, are improved, and simultaneous reconstruction of a double-layer medium absorption spectrum and a reduced scattering spectrum is realized.

The application discloses a kind of light field imaging parameter control method and device of underwater light field camera, the method includes: S1, the underwater light field camera including light field camera and waterproof spherical shell is built;S2, the physical parameter of light field camera is acquired, and the light propagation model of the waterproof spherical shell is built;S3, according to light propagation analysis, the equivalent object distance of light field camera is acquired;S4, based on the disparity between the microlens array of light field camera and equivalent object distance, the shortest distance between the center of the waterproof spherical shell of underwater light field camera and the lens optical center of light field camera is calibrated;S5, based on equivalent object distance and the shortest distance between the center of the waterproof spherical shell of underwater light field camera and the lens optical center of light field camera, the light field imaging parameter of underwater light field camera is optimized;The application can improve the light field imaging definition of underwater light field camera;Improve the field of view angle of underwater light field camera, visual effect is better, improve the light field imaging quality of underwater light field camera.

To address the shortcomings of existing methods for detecting surface texture defects in complex curved parts, such as low detection efficiency, poor stability and recognition, and difficulty in rapidly and accurately quantitatively characterizing sub-millimeter-level surface texture defects in three dimensions, this invention provides a rapid three-dimensional quantitative characterization method for surface defects in curved parts. This invention utilizes a light field camera to simultaneously acquire two-dimensional images and three-dimensional point cloud data, improving the consistency and stability of subsequent three-dimensional quantitative characterization of defects. It employs a deep learning semantic segmentation model to extract two-dimensional contour information of defects, providing precise semantic guidance for subsequent three-dimensional spatial mapping. It establishes a coordinate mapping relationship between pixels in the two-dimensional image and physical points in the three-dimensional point cloud coordinate system, enabling precise selection of subsets of the defect's three-dimensional point cloud. Finally, it uses principal component analysis (PCA) to establish a local coordinate system, and within this coordinate system, employs the minimum bounding box method to calculate the three-dimensional geometric dimensions of the defects, achieving rapid and accurate three-dimensional quantitative characterization of the defects.

The invention discloses a double-lens cooperative light field camera system and a local light field analysis method, and relates to the technical field of light field cameras, the system comprises a main lens, a spectroscope, a micro lens array, a single lens, a main CMOS sensor, a preview CMOS sensor and a processor which are modularly designed; wherein the main lens is used for focusing scene light to capture full-light-field information; the spectroscope distributes light output by the main lens to a main light path and a preview light path according to a preset splitting ratio; the micro-lens array is used for decomposing transmission light into light field four-dimensional information containing direction and intensity; the main CMOS sensor is used for recording full-light-field data after the micro-lens array is decomposed; the single lens processes the reflected light to generate a preview image; the preview CMOS sensor is used for recording a preview image output by the single lens; and the processor extracts corresponding full-light-field data recorded by the main CMOS sensor according to the target point coordinates in the preview image, and analyzes the full-light-field data to generate a high-definition image. The calculation amount and the hardware cost can be remarkably reduced.

The present application relates to the technical field of light field imaging measurement, and particularly relates to an absolute depth measurement method based on a parallax angle of a polarized microlens light field image. The method comprises the following steps: 1) image pixel level classification based on multi-view sub-aperture depth maps, which provides a benchmark for optimization of a parallax map; 2) constructing a matching cost equation based on pixel classification results, so as to realize global optimization of a relative depth map from coarse to fine; 3) establishing a linear mapping model from the relative depth map to the absolute depth map based on the hardware structure of a microlens light field camera, so as to realize depth calibration; 4) combining the depth calibration with internal parameters of the light field camera, so as to calculate absolute depth physical parameters of an observed target. Thus, physical measurement and characterization of three-dimensional absolute size of a real observed object are realized, and absolute depth information of a target to be measured in a light field image can be obtained, in particular, absolute depth information of a weak texture and a blocked target can be obtained.

This invention relates to a continuous three-dimensional particle field reconstruction method based on single-light-field imaging using EM deconvolution, belonging to the field of optical imaging technology. This method employs a light field camera to capture light field images of a discretized three-dimensional flow field, acquiring the light intensity information of scattered light emitted by particles on each pixel of a CCD detector. Based on the light intensity information, a weight matrix for each voxel is calculated, and the entire weight matrix is ​​loaded only once. The nth frame of the light field image is loaded, and the non-zero pixels in the nth frame are identified. The non-zero pixel sequence formed by the scattered light from tracer particles is extracted. An EM deconvolution algorithm is used to iteratively calculate the non-zero pixel sequence, updating the intensity of each voxel, thereby reconstructing the three-dimensional particle field of a single frame. The above steps are repeated to complete the reconstruction of the three-dimensional particle field from multiple frames of high temporal resolution light field images, resulting in higher reconstruction accuracy and consistency, and less time consumption.

This invention provides a method and system for multispectral temperature measurement based on an infrared sensor. The method includes: calculating the optical parameters of a microlens array and a filter array according to optical formulas; then fabricating the microlens array through photolithography, development, pressing, and atomic force scanning; next, constructing an infrared multispectral radiation temperature measurement system; calibrating the system using a blackbody furnace to obtain the correspondence between the camera output signal intensity and the standard temperature; decoupling the multispectral light field image based on the imaging characteristics of the light field camera; and finally, using the decoupled data to sequentially calculate the true temperature of points on the high-temperature component, obtaining the two-dimensional true temperature field of the high-temperature component. This invention overcomes the limitations of visible light multispectral temperature measurement by combining multispectral light field measurement with infrared imaging technology to achieve two-dimensional temperature field measurement, effectively overcoming the shortcomings of existing optical temperature measurement systems, such as complex structure and inability to simultaneously achieve both high measurement accuracy and wide range.

This invention discloses a modular light field microscope system based on a Lytro light field camera and its debugging method, belonging to the field of optical imaging and microscopy technology. The system includes a microscope objective, a first lens, a second lens, and a Lytro light field camera arranged sequentially along the optical path. The focal length of the first lens is shorter than that of the second lens, forming a relay optical path. The system also includes a modular mechanical support assembly composed of a cage rod and a cage plate, and each optical component can be slidably adjusted along the axial direction of the cage rod. This invention also discloses a step-by-step debugging method for the system. This invention utilizes the advantages of the commercial Lytro light field camera—small size and low cost—and overcomes the technical bottleneck of Lytro cameras being unsuitable for microscopic observation through a focal length gradient dual-lens relay optical path design. The system is compact, portable, and easy to debug, enabling high-quality four-dimensional light field imaging of microscopic samples. It is suitable for scenarios such as three-dimensional velocimetry of fluid particles, biological fluorescence microscopy, and in-situ microscale detection.

This invention belongs to the field of logistics management technology, and specifically relates to an automatic weighing system and method for ore loading vehicles based on light field technology. The system includes a light field intelligent detection device, a weighing system, a loading vehicle, and a barrier control system. It acquires three-dimensional vehicle data through a light field camera array, combines this data with a deep learning algorithm to calculate the material volume and weight in real time, and performs dual verification with weighbridge data to achieve high-precision load measurement. An adaptive grid algorithm and density deviation detection are employed to effectively identify theft or overloading. This system can operate stably even in dusty environments, achieving a volume measurement accuracy of ±0.8% and a weight verification error of ≤±1%, significantly improving the efficiency and safety of mine weighing and reducing manual intervention.

The invention relates to the technical field of optical imaging, in particular to an image highlight removing method and system based on virtual polaroid and virtual exposure fusion, and the method comprises the steps: obtaining a Stokes parameter image of a target scene, dividing highlight region blocks, preliminarily removing highlight of the highlight region blocks, and removing highlight in combination with a virtual exposure image. According to the method, the virtual polarization filtering technology and the virtual exposure fusion technology are creatively combined, the advantages of the virtual polarization filtering technology and the virtual exposure fusion technology are fully exerted, a brand-new and efficient image highlight removal method is formed, the projector and a light field camera do not need to be combined in the whole process, and the image highlight removal efficiency is improved. According to the invention, the method does not need to employ a plurality of light sources or detectors to obtain images under different visual angles or illumination conditions, overcomes the harsh implementation conditions of the traditional highlight removal technology, improves the reliability and uniformity of highlight removal, and solves the problems of harsh highlight region removal conditions and poor universality in the prior art.

This invention provides a dual-modal real-time microscopic imaging device, comprising: an illumination component configured to illuminate a sample; an imaging component configured to receive light signals from the sample and emit scattered light signals and fluorescence signals via different optical paths; a full-field camera configured to receive scattered light signals emitted from the imaging component; a fluorescence imaging two-dimensional array detector and a GISC camera configured to switchably receive fluorescence signals emitted from the imaging component; and an image processing processor electrically connected to the full-field camera, the fluorescence imaging two-dimensional array detector, and the GISC camera. The dual-modal real-time microscopic imaging device and method of this invention solve the problems of low spatial resolution, inability to simultaneously perform real-time dual-modal imaging, and difficulties in multi-color real-time imaging in existing dual-modal phase and fluorescence imaging methods.

Systems, methods, and apparatuses are provided herein for changing the positions and / or shapes of microlenses of a light field camera to generate light field images with enhanced depth of field and / or dynamic range. This may be accomplished by a light field camera determining a plurality of focus measurements for a plurality of microlenses, wherein one or more of the plurality of microlenses vary in distance from a main lens of the light field camera. The light field camera may use the plurality of focus measurements to determine a microlens of the plurality of microlenses that captures information that is the most focused. The light field camera can then determine defocus functions for the microlenses that are not capturing information that is the most focused. The light field camera can then generate a light field image using the determined defocus functions and the information captured by the plurality of microlenses.

This application discloses a color difference detection method, apparatus, device, and storage medium. The method includes: acquiring images to be calibrated obtained by a light field camera taking pictures of a black and white calibration object with no filter and different color filters respectively; performing scale calibration processing on the images to be calibrated to obtain a color calibration matrix for each color; acquiring an image to be detected taken by the light field camera, in which an extended reality device displays an image of the color calibration object; and detecting the color difference of the extended reality device based on the color calibration matrix and the image to be detected for the corresponding color.

The invention provides an optical detection device and method based on BGA packaging ball mounting defect detection, and relates to the technical field of optical detection. According to the optical detection device, a light field camera is used for extracting the three-dimensional contour morphology of a BGA ball, annular light sources at different angles are used for lighting, appearance detection of solder balls with different heights is achieved, a combined lighting mode is used for achieving accurate detection of the diameter and the offset distance of the BGA ball, the coplanarity of a BGA ball array is calculated according to information of the three-dimensional morphology of the BGA ball, and the detection accuracy of the BGA ball array is improved. The light source and the lens are comprehensively utilized to obtain two-dimensional and three-dimensional information of different positions for the BGA packaging ball mounting array, so that the coplanarity measurement of the BGA ball array is completed on the same set of optical detection device, and one-time detection of all kinds of BGA ball defects such as BGA ball bridging, tin shortage, ball shortage, multiple balls, ball damage, local offset and overall offset is realized; the detection is efficient, and the digital defect detection standard which cannot be realized under a microscope in the prior art is realized.

The application discloses a three-dimensional positioning method based on a light field camera and belongs to the field of visual positioning. The method comprises the following steps: a light field camera is used to shoot a light field image of a three-dimensional object; a multi-view sub-aperture image is obtained by converting the light field image; and parallax of adjacent views is calculated. A mapping relationship model from a three-dimensional point in space to a sensor light field coordinate is established; a parallax-depth mapping relationship is established by combining the multi-view parallax and the physical meaning of the mapping relationship model; the depth is solved from the parallax; the spatial coordinate is solved from the sensor light field coordinate contained in the parallax map; and finally, the three-dimensional coordinate of the object is obtained. The method realizes panoramic three-dimensional positioning, and solves the problem of panoramic and accurate three-dimensional measurement of the light field camera.

The invention provides a flame three-dimensional multi-physics field reconstruction method based on a light field focus stack, which comprises the following steps of: imaging blackbody radiation sources at different temperatures by using a light field camera, and establishing and outputting a fitting relationship between response values of color channels of a light field refocusing image and corresponding spectral radiation intensities according to a Planck law; acquiring a flame light field focus stack image sequence of the flame light field original image by adopting a super-resolution reconstruction method based on a sub-aperture image; and outputting temperature distribution and soot concentration distribution of the corresponding flame slice image by using the three-dimensional convolutional neural network model constrained by the fused radiation transmission physical information. According to the method, the implicit correlation model between the flame focus slice and the three-dimensional physical field of the flame focus slice is directly constructed by fusing the three-dimensional convolutional neural network model constrained by the forward radiation transmission physical information, and the generalization ability and the interpretability of the model can be improved, so that the joint reconstruction work of the physical field is more effectively realized.