131 results about "Light-field camera" patented technology

The invention discloses an amphibious unmanned aerial vehicle riverway sediment concentration real-time monitoring method based on underwater light field imaging and cross-modal fusion, and relates to the technical field of riverway sediment monitoring. Comprising the following steps: acquiring aerial multispectral data and preprocessing the aerial multispectral data; an integrated polarized light field camera is carried on an amphibious unmanned aerial vehicle to capture underwater light field data, and underwater three-dimensional light field information is reconstructed and sediment scattering characteristics are extracted through an optical model and a lightweight deep learning network; the air multispectral data and sediment scattering characteristics are fused through space-time alignment and an attention mechanism, a sediment concentration prediction model is generated, and then a spatial distribution diagram of the river sediment concentration is output; and carrying out real-time monitoring and early warning on the river sediment concentration according to the spatial distribution diagram of the river sediment concentration through edge calculation and cloud collaboration architecture. The method is combined with an advanced deep learning algorithm, high-resolution silt concentration inversion can be realized, and the method has real-time performance and adapts to a complex water body environment.

The invention provides a palm vein feature extraction and depth adaptive matching method, which relates to the technical field of biological recognition, and is characterized in that a three-dimensional space-time convolutional neural network and a graph neural network are combined to extract space depth features, time blood flow dynamic features and blood vessel topological features of palm veins, and information bottleneck is utilized to optimize and screen a feature subset with the most distinguishing property; the method breaks through the limitation of traditional two-dimensional texture feature extraction, obtains abundant palm vein information through multi-source data acquisition (such as near-infrared imaging, a light field camera and a spectrum sensor), overcomes the interference of environmental factors such as illumination, angle and noise, achieves the high-precision modeling and stable extraction effects of palm vein features, and can achieve the real-time extraction of the palm vein features in a complex scene. A high recognition rate is kept under different illumination conditions or user hand posture changes, the robustness of the palm vein recognition system is remarkably improved, and the method is suitable for scenes with high safety requirements.

The utility model discloses an acousto-optic coupling three-dimensional flame area temperature field monitoring system, which comprises a computer, a power amplifier, a relay, a loudspeaker, a microphone and a signal analyzer which are sequentially connected, and further comprises a three-dimensional rack, a light field camera and a blackbody furnace, a plurality of groups of loudspeakers and microphones are arranged, and each group of loudspeakers and microphones are located at the same position on the three-dimensional rack; the signal input end of the light field camera is respectively connected with the three-dimensional flame and the blackbody furnace in the three-dimensional rack, and the signal output end of the light field camera is connected with the signal input end of the computer; the blackbody furnace is used for the light field camera to calibrate the relationship between the gray level of the two-dimensional layered flame original brightness image and the radiation intensity; measurement targets of the system are gas temperature and three-dimensional flame temperature in a three-dimensional to-be-measured area which is surrounded by a three-dimensional rack and contains three-dimensional flame, the overall temperature of a gas medium in a three-dimensional space is measured by using a non-contact sound velocity method temperature measurement principle, and a more comprehensive contribution is made to temperature monitoring of a key area of a hearth.

The invention provides a molten pool temperature field in-situ monitoring method and system based on multi-view light field imaging, and the method employs a light field camera multi-view system, obtains molten pool images of R, G and B channels through building a light field sub-aperture Bayer model, and calibrates the relative spectral responsivity of a light field camera and the proportion of dual-wavelength intensity under a coaxial temperature measurement system. Molten pool images of two channels are selected from the molten pool images of the R channel, the G channel and the B channel, calibration parameters are combined, and a molten pool temperature field is derived through a dual-wavelength temperature measurement method. A blackbody furnace is adopted to calibrate a multi-view light field temperature measurement system, and the error of a monitored temperature field is kept below 3%. The method simplifies the monitoring process, improves the temperature measurement precision, has a wide temperature range and low complexity, can perform in-situ monitoring on the high-temperature metal molten pool, and has a wide application prospect in the field of additive manufacturing.

The invention relates to the technical field of data processing, and discloses a mercuric chloride-based high-reliability light field super-resolution edge calculation method and a mercuric chloride-based high-reliability light field super-resolution edge calculation system. The method comprises the following steps: collecting RAW data through a light field camera and carrying out DnCNN denoising; using Transform-CNN (Convolutional Neural Network) to calibrate and generate an error coefficient and a correction matrix; the FPGA accelerates the OfficientNet and processes the OfficientNet and the FFT to output a score and a frequency domain result; the mercuration NPU executes ESRGAN to form a deep learning image; performing U-Net fusion to obtain an 8K image; and carrying out quality evaluation and dynamically allocating resources to construct a redundancy mechanism. According to the method, the high-reliability light field super-resolution edge calculation method based on mercuric chloride is constructed, innovative mechanisms such as adaptive quality evaluation, dual-path super-resolution processing and intelligent fusion, heterogeneous resource dynamic scheduling and the like are integrated, and the reliability and adaptability of light field super-resolution processing in the edge environment are effectively improved.

Disclosed in the present invention are a method and a system for monitoring unmanned aerial vehicles based on three-dimensional light field technology. Provided is an unmanned aerial vehicle monitoring method based on three-dimensional light field technology, comprising: beginning unmanned aerial vehicle monitoring; by means of a light field camera, acquiring low resolution video image information; determining whether the acquired video image information is an unmanned aerial vehicle; performing graphic reconstruction on an unmanned aerial vehicle image therein; and acquiring reconstructed light field image depth and position information to monitor the unmanned aerial vehicle and emitting an alert. The method and the system for monitoring in the present invention are able to acquire a clear stereoscopic image, thus raising efficiency and accuracy in the process of unmanned aerial vehicle monitoring or detection.

The invention relates to the field of light field imaging, in particular to a light field camera calibration method based on original light field image line feature detection, which comprises the following steps: acquiring a white background original light field image set of a light field camera under different shooting conditions, and processing to obtain projection center grid data of a microlens array; obtaining a checkerboard original light field image set of the light field camera under different shooting conditions, and loading projection center grid data of the micro-lens array on each checkerboard original light field image; calculating a gradient value of each pixel in each checkerboard original light field image, queuing the gradient values of the pixels, and performing region diffusion and model fitting according to a gradient angle similar direction to obtain line features in each checkerboard original light field image so as to calculate angular point coordinates of each checkerboard original light field image; and constructing a light field camera projection model, and calibrating parameters of the light field camera in combination with the angular point coordinates. According to the method, introduced errors are reduced while time is saved.

The invention discloses a face pose detection method and system based on a light field camera. The method comprises the following steps: acquiring a scene image by using the light field camera to obtain light field image data; acquiring three-dimensional data of a human face region of interest from the light field image data; performing detection processing on the three-dimensional data of the region of interest of the human face to obtain information of a plurality of three-dimensional feature points of the human face; and substituting the information of the plurality of face three-dimensional feature points into the face attitude calculation model for solving to obtain a face attitude matrix. According to the light field camera-based face pose detection method, detection is carried out based on the multi-view image, the influence of shielding, illumination change and pose change on detection can be effectively reduced, high environmental adaptability is achieved, and the accuracy of face pose detection can be effectively improved.

The invention relates to the technical field of image acquisition, in particular to an image acquisition device, an image acquisition method, electronic equipment and a storage medium. According to the embodiment of the invention, camera calibration parameters need to be acquired through the image acquisition calibration module, the trigger signals are generated in the multi-camera control module, and the trigger signals are respectively sent to the light field camera imaging module and the event camera imaging module; the light field camera imaging module and the event camera imaging module carry out data acquisition in the same detection time slot; in a light field camera imaging module, collecting light field data of a target scene in a detection time slot according to the trigger signal; in an event camera imaging module, event data of a target scene are collected in a detection time slot according to the trigger signal; and in the data fusion module, image acquisition fusion data is generated according to the camera calibration parameters, the light field data and the event data. Therefore, high-quality real-time imaging can be realized.

The invention belongs to the technical field of three-dimensional imaging, and particularly relates to an active light field three-dimensional imaging method based on phase guidance. The phase information is introduced into the light field stereo matching process, and the spatial resolution loss of the light field camera is compensated through phase guidance measurement, so that the bottleneck problem that the existing light field camera is limited by the sub-aperture resolution to cause the low resolution of the depth map is solved. Based on effective fusion of the light field and the structured light, three-dimensional reconstruction with high pixel resolution can be realized in single shooting, the robustness and precision of depth estimation can be ensured, and the method has wide application potential in the fields of precision measurement, industrial detection, three-dimensional modeling and the like.

The invention discloses an additive manufacturing process high dynamic range imaging method and system based on a multi-view light field. Firstly, an original light field image in the additive manufacturing process is collected through a light field camera; extracting R, G and B single-wavelength channel images from the original light field image by the established light field Bayer model; then, taking the R-channel image as a reference, and adopting a K-means clustering algorithm for segmentation to obtain a low-light area and a high-light area in the R-channel image; and finally, fusing the pixel value of the G channel image corresponding to the low-light area with the R channel image to enhance weak light details, and fusing the pixel value of the B channel image corresponding to the highlight area with the R channel image to suppress highlight, thereby reconstructing to obtain an HDR image. According to the method, the image quality and the information integrity monitored in the additive manufacturing process are effectively improved, and high-quality visual data are provided for additive manufacturing process optimization and defect recognition.

The invention provides a robot touch and vision fused double-loop coupling self-adaptive grabbing control method, which relates to the technical field of robot grabbing control, and comprises the following steps of: acquiring surface characteristic data through a light field camera array, acquiring dynamic contact information and extracting stability characteristics by a sensing array, and acquiring the surface characteristic data through a touch sensor array; cross-modal coupling is constructed based on Tucker decomposition, tactile and visual information is fused, mapping parameters are established, an adaptive time window is constructed to optimize grabbing parameters, and the grabbing posture and the clamping force are adjusted in combination with a Lyapunov stability criterion and a dynamic compensation mechanism. According to the method, the grabbing stability and robustness of the robot in a complex environment can be improved.

The invention discloses a light field time domain super-resolution method based on an event camera, and belongs to the field of optical engineering. A light field camera and an event camera are used for collecting light field event data, decoding the light field data, denoising the event data and generating an event image, the light field data and an event grayscale image are subjected to joint calibration in a sub-aperture image mode, joint optimization is carried out through calibration results of multiple viewpoints, a checkerboard is shot, and a target object is obtained. And obtaining an affine transformation matrix and a perspective transformation matrix, and obtaining a super-resolution result of projecting the event image onto a light field image plane. Starting from the expression mode of the light field macro pixel, the method has applicability to light field data of any macro pixel size; compared with a traditional time domain super-resolution method, the time domain super-resolution of the light field camera at any frame rate can be realized by segmenting the event stream at different scales.

This invention discloses a method, device, storage medium, and program product for calibrating and aligning a light field camera and an RGB camera. The deviation value is calculated by matching the images from the two cameras and fed back to the engineer, allowing for more precise fine-tuning. The deviation value provided by this invention is accurate and objective, unaffected by human factors. With objective guidance, engineers can significantly improve the efficiency of fine-tuning camera positions, thereby enhancing the efficiency of this task in actual production assembly.

The present invention discloses a simulation system and method for a high-dimensional light field event camera. The simulation method comprises the following steps: (1) constructing a world coordinate system, building a three-dimensional object model to be simulated, and setting the object's motion state to meet the dynamic scene requirements of a dynamic vision sensor; (2) constructing an optical camera model in front of the three-dimensional object model according to a focused Kepler light field structure, so that the light field camera can capture the angle and position of light and obtain high-dimensional light field information including the light position and incident angle; (3) mapping the obtained high-dimensional light field information to a two-dimensional plane to form an image frame, simulating it as an event according to the brightness difference between each frame, and finally outputting a light field event stream. The present invention can accurately simulate the light field event stream, avoid errors caused by processes such as microlens array calibration in actual hardware construction, and provide a simple and feasible method for the production of large-scale light field event data sets.

The invention belongs to the technical field of computer vision and light field imaging, and discloses a large-area highlight detection and repair method and system. The method comprises the following steps: imaging by using a light field camera array to obtain a multi-view-angle light field image, and calculating an initial parallax; the method comprises the following steps: obtaining a multi-view highlight area mask through highlight detection, removing the highlight area mask to obtain a light field image without a highlight area, carrying out light field refocusing to obtain a non-highlight refocusing image stack, applying the highlight area mask obtained through central view angle detection to an initial parallax image, and constructing a reliable parallax set to realize parallax layering; according to parallax layer information of a non-highlight area pixel adjacent to a highlight pixel space in the highlight area, carrying out adaptation on a parallax layer of the highlight pixel; and constructing a hierarchical restoration strategy based on observation conditions of different viewpoints when no-highlight refocusing image of the highlight pixel in the corresponding parallax layer is superposed and averaged at the pixel position, thereby realizing restoration of the highlight area. According to the method, high-quality and large-area highlight repair can be realized.

The invention relates to the technical field of optical lens automatic focusing, in particular to an optical lens automatic focusing system and method. The method comprises the following steps: carrying out polarized light field acquisition through a light field camera to obtain a polarized multi-view image set; performing light field feature extraction on the polarization multi-view image set to obtain a polarization phase gradient spectrum; constructing a polarization sensing depth reconstruction network according to the polarization phase gradient spectrum to obtain a preliminary depth map; performing ray tracing simulation processing according to the initial depth map to obtain an enhanced depth map; performing depth map optimization on the preliminary depth map and the enhanced depth map to obtain an optimized depth map; acquiring optical system parameters; performing scene layering on the optimized depth map to obtain a layered scene; and performing Fresnel diffraction simulation according to the optical system parameters and the layered scene to obtain a Fresnel diffraction image set. According to the invention, the focusing precision, speed and robustness are improved through the automatic focusing technology of the optical lens.

The present application is applicable to the technical field of light field cameras, and particularly relates to a light field camera and its calibration method, device, and medium. The method performs frequency domain conversion on the direction plane coordinates in the acquired initial light field data to obtain frequency domain light field data, screens the frequency domain light field image with a frequency domain threshold to obtain light field component data, pairs all sub-views according to the light field component data to obtain sub-view pairs, generates matching point pairs according to the corner positions of the calibration board collected in the sub-view pairs, respectively forms a fundamental matrix with each matching point pair and the internal and external parameters, constructs an epipolar constraint equation, obtains the initial internal and external parameters with the minimum of the epipolar constraint equation as the objective, and calibrates the light field camera using the internal and external parameters obtained by optimizing the initial internal and external parameters. Through frequency domain analysis, the computational redundancy in light field data processing is effectively reduced. By using high-frequency feature extraction and the establishment of the epipolar constraint equation, the initial value estimation of camera calibration is accelerated, and the efficiency of camera calibration is improved.

The invention provides a light field multispectral temperature measurement method and system based on an infrared sensor. The method comprises the following steps: calculating optical parameters of a microlens array and a filter array according to an optical formula; then, processing and manufacturing a micro-lens array through photoetching, developing, pressing a plate and atomic force scanning; secondly, building an infrared light field multispectral radiation temperature measurement system; calibrating the system by using a blackbody furnace to obtain a corresponding relation between the camera output signal intensity and the standard temperature; decoupling the multispectral light field image according to the imaging characteristics of the light field camera; and finally, the real temperatures of the object points of the high-temperature component are calculated in sequence by using the decoupled data, and a two-dimensional real temperature field of the high-temperature component is obtained. According to the invention, the limitation of multi-spectral temperature measurement based on a visible light field is solved, the multi-spectrum of the light field is combined with an infrared imaging technology, the measurement of a two-dimensional temperature field is realized, and the defects that an existing optical temperature measurement system is complex in structure, and the temperature measurement precision and breadth cannot be considered at the same time are effectively overcome.

Systems, methods, and apparatuses are provided herein for changing the positions and / or shapes of microlenses of a light field camera to generate light field images with enhanced depth of field and / or dynamic range. This may be accomplished by a light field camera determining a plurality of focus measurements for a plurality of microlenses, wherein one or more of the plurality of microlenses vary in distance from a main lens of the light field camera. The light field camera may use the plurality of focus measurements to determine a microlens of the plurality of microlenses that captures information that is the most focused. The light field camera can then determine defocus functions for the microlenses that are not capturing information that is the most focused. The light field camera can then generate a light field image using the determined defocus functions and the information captured by the plurality of microlenses.

The invention provides a rapid automatic focusing method, device and equipment based on a light field camera and a medium, and the method comprises the steps: carrying out the space and angle information reconstruction of a first sub-image according to the arrangement mode of a micro-lens array, and obtaining a first image set; screening out a first to-be-processed image according to the information richness corresponding to each first image in the first image set; determining whether the first distance is an optimal focusing distance or not according to the angle information fusion degree and definition of the first to-be-processed image in a preset focusing window; wherein the optimal focusing distance is used for light field reconstruction. The problems of long time consumption, large calculation amount, high required calculation power and the like in the light field focusing process can be solved.

The present invention relates to the technical field of automatic focusing of optical lenses, and particularly to an automatic focusing system and method for optical lenses. The method includes the following steps: collecting a polarized light field through a light field camera to obtain a polarized multi-view image set; extracting light field features from the polarized multi-view image set to obtain a polarized phase gradient spectrum; constructing a polarized perception depth reconstruction network based on the polarized phase gradient spectrum to obtain a preliminary depth map; performing ray tracing simulation processing on the preliminary depth map to obtain an enhanced depth map; optimizing the preliminary depth map and the enhanced depth map to obtain an optimized depth map; obtaining optical system parameters; performing scene layering on the optimized depth map to obtain a layered scene; and performing Fresnel diffraction simulation based on the optical system parameters and the layered scene to obtain a Fresnel diffraction image set. The present invention improves the focusing accuracy, speed and robustness through the automatic focusing technology of optical lenses.

The invention discloses a single-exposure three-dimensional imaging method combining structured light and light field imaging. The single-exposure three-dimensional imaging method comprises the following steps: S1, projecting a composite fringe pattern formed by orthogonally superposing vertical fringes and horizontal fringes to an object; s2, acquiring a stripe image modulated by the surface of the object through single exposure of a light field camera, and converting the stripe image into a horizontal pole plane graph and a vertical pole plane graph; s3, depth estimation is carried out on the horizontal pole plane graph and the vertical pole plane graph, a fused depth map is obtained, and three-dimensional reconstruction is achieved through the fused depth map. According to the invention, high-precision and high-efficiency three-dimensional reconstruction can be realized.

Provided is an imaging module having a biometric authentication function, which is suitable for a wearable electronic device, is thin, and can be made compact. An imaging module is included in a wearable electronic device, the imaging module including: a light emitting element that is provided to face a measurement target side in a worn state and that emits light to a measurement target; a light field camera that is provided to face the measurement target side in a worn state and that images the light emitted from the light emitting element to the measurement target; and a processor configured to process an image obtained by imaging the light emitted from the light emitting element to the measurement target using the light field camera, in which the light field camera includes a liquid crystal lens array and an image sensor array in order from the measurement target side.

The invention belongs to the technical field of optical camera products, and provides a high-duty-ratio micro-lens array preparation method and a light field camera. The method comprises the following steps: S1, preparing a photoetching mask plate with a high duty ratio, and performing contact photoetching processing to obtain a spin coating chromium plate photoetching mask plate; s2, cleaning the micro lens array substrate; s3, spin-coating a hydrophobic layer; s4, patterning the sacrificial layer; s5, performing plasma ashing and selective surface modification; s6, removing the part, which is not modified, of the photoresist; and S7, performing blade coating of optical cement to form a micro lens, thereby obtaining a wettability micro lens array with a high duty ratio. By adopting the cheap CMOS image sensor and by virtue of the advantages of the wettability micro-lens array with high duty ratio, the light field camera is constructed, so that the brightness, the signal-to-noise ratio, the spatial resolution and the three-dimensional perception capability of a picture shot by the light field camera are remarkably improved, and accurate analysis of the image is facilitated; the method is used in the fields of automatic driving, cultural relic digitization, industrial detection, medical images and the like.

The application provides a robot tactile and visual fusion double-ring coupling adaptive grasping control method, relates to the technical field of robot grasping control, and comprises the following steps: acquiring surface characteristic data through a light field camera array, collecting dynamic contact information and extracting stability features by a sensor array, constructing cross-modal coupling and fusing tactile and visual information based on Tucker decomposition, establishing mapping parameters and constructing an adaptive time window to optimize grasping parameters, and adjusting grasping posture and clamping force in combination with Lyapunov stability criterion and a dynamic compensation mechanism. The method can improve the stability and robustness of robot grasping in a complex environment.

The application discloses a three-dimensional light-thermal parameter reconstruction method and system for a combustion process, and belongs to the field of combustion diagnosis.The application solves the problems of low measurement accuracy in a high-temperature environment and poor accuracy of extracting effective signals in a complex background existing in the prior art light field parameter measurement.The application adopts a polarization spectrum camera and a light field camera to synchronously perform multi-angle data acquisition on a combustion area, and acquires polarization spectrum data and light field data of the combustion area;light field depth information is extracted from the light field data, a three-dimensional structure of the combustion area is constructed by combining the light field depth information with a light field reconstruction algorithm, spectral analysis is performed on the polarization spectrum data, and polarization degree and polarization angle information of the polarization spectrum are acquired;the three-dimensional structure is combined with a CT reconstruction algorithm; a three-dimensional distribution field of particulate matters in the combustion process is acquired; and based on the polarization spectrum information and the preprocessed light field data, the three-dimensional temperature field and the concentration field distribution of gases in the combustion area are calculated by using the optimized three-dimensional structure.The application is mainly used for combustion product parameter acquisition.

The invention discloses a tea bud picking obstacle avoidance path planning method and system based on a multi-modal information fusion diffusion model. A tea tree canopy image is acquired through a light field camera, a dense three-dimensional point cloud is constructed, tea bud target detection and positioning are realized in combination with an improved YOLOV8 and a PointNet + + network, a path generation method based on a U-net architecture condition denoising diffusion model is provided, path planning is modeled as a condition track generation problem, and the problem is solved. According to the method, multi-modal condition information input and a target guiding steering function are combined, multi-step denoising iteration is executed, a continuous, smooth, collision-free, dynamic and feasible mechanical arm picking path is directly generated, the sawtooth-shaped defect and post-optimization dependence of a path generated through traditional sampling algorithms such as RRT are avoided, and the path quality and execution stability are improved. The system comprises a light field acquisition module, a three-dimensional reconstruction module, a visual positioning module, a path planning module, an execution control module and the like, realizes accurate obstacle-avoiding picking of tea buds, is suitable for path planning of picking operation of tea leaves and other fruits and vegetables, and provides technical support for agricultural picking and harvesting.

The invention discloses a single-camera three-dimensional particle tracking velocity measurement method and system based on a double-helix point spread function. The test method comprises the following steps: exciting tracer particles in a flow field to be measured; performing double-helix phase modulation on the particle scattered light; calibrating the angle-depth relation of the double-helix particles; collecting an original image sequence of flow field particles; performing three-dimensional particle field reconstruction on the original image sequence; carrying out three-dimensional particle velocity field solving on the three-dimensional particle field; and outputting a high-precision three-dimensional velocity field. According to the test method, double-helix phase modulation is carried out on particle scattered light, the depth of particles is determined through angle information of two modulated lobes, particle three-dimensional positioning of a single camera is achieved, and then three-dimensional flow field particle tracking speed measurement of the single camera is achieved. Compared with a single-light-field camera three-dimensional particle tracking velocity measurement technology, the spatial resolution is improved. Compared with a three-hole mask three-dimensional particle tracking speed measurement technology, the light utilization rate is improved. The method is suitable for high-precision speed measurement of a three-dimensional flow field.

The invention discloses a naked-eye 3D imaging method, and relates to the technical field of naked-eye 3D, and the method comprises the steps: employing a binocular camera or a light field camera to obtain multi-angle data, enabling the distance between cameras to change according to a target distance through dynamic baseline adjustment, increasing a baseline in a long distance to improve the depth-of-field resolution, and reducing the baseline in a short distance to enhance the local stereoscopic impression; meanwhile, in combination with an AI depth-of-field prediction model, a low-light environment and a high-reflectivity surface are subjected to deep optimization treatment, and errors caused by light interference are reduced; in the depth calculation process, an optical flow analysis method is adopted, a pixel motion vector field is calculated through a Horn-Schunck optical flow, the stability of optical flow calculation is optimized in combination with Kalman filtering, parallax matching or optical field reconstruction is performed based on an optical flow calculation result, for a binocular camera mode, a block matching method is adopted to calculate parallax, and the optical field is reconstructed. And in the light field camera mode, depth reconstruction is carried out by using an EPI polar plane parallax estimation algorithm.