91 results about "Photometric stereo" patented technology

The invention discloses an image recognition system for industrial part defect detection, and particularly relates to the field of part defect detection, which comprises a multi-axis controllable light source array module, a high-speed polarization camera module, an edge computing node module, a double-branch semantic segmentation network module and a physical constraint post-processing module, according to the invention, through combination of time-sharing stroboscopic illumination and polarization image sequence acquisition, multi-dimensional perception of surface topography and material differences is realized; generating an elevation map and a normal map by using photometric stereo solution, constructing a differential rendering layer reverse matching CAD model, and extracting flash sensitive features; a double-branch U-Net network is adopted to fuse geometric and polarization characteristics, the characterization capability is enhanced through a trans-attention mechanism, and a pixel-level mask is output; and finally, mapping a two-dimensional result to a three-dimensional coordinate system by means of calibration parameters, carrying out geometric verification in combination with a tolerance zone and a height threshold value, and automatically generating a structured defect report containing position, size, grade and visual information.

Disclosed in the present invention is a photometric stereo-based device for rapidly detecting surface features of an object, capable of rapidly acquiring a plurality of images meeting the requirements of a photometric stereo algorithm. The device comprises a light emitting assembly, an acquisition assembly located above the light emitting assembly, and a data processing module; the acquisition assembly comprises a light path system and a camera group; the light path system comprises: a beam splitting prism group used for splitting reflected light from the surface of an object into a plurality of beams of light and respectively outputting the plurality of beams of light in different preset directions to form images to be processed, and a plurality of bandpass filters correspondingly arranged on the output sides of the plurality of beams of light used for forming said images, wherein the central wavelengths of the bandpass filters are respectively configured to be different wavebands of the reflected light from the surface of the object; the camera group comprises a plurality of cameras which are arranged behind the bandpass filters in a one-to-one correspondence mode and acquire filtered light beams; and the data processing module receives image information sent by the cameras, and performs compensation and photometric stereo reconstruction on the image information, so as to obtain surface features of the object. Further disclosed is a method implemented on the basis of the device.

Photometric stereo techniques enable using a single camera to perform an enrollment process for creating a user-specific anatomical model of an eye for gaze tracking. The user-specific anatomical model includes information about a user's center of vision at multiple dilation states of the eye, which can be used to enhance the accuracy of gaze tracking techniques. Accurate gaze tracking techniques enable the use of gaze tracking at close range, for example, gaze tracking within a head-mounted display device.

The invention discloses an electronic screen film surface dust detection method based on photometric stereo. The method comprises the following steps: image acquisition: acquiring 4-8 grayscale images in different incident directions; image preprocessing: carrying out edge-preserving noise reduction processing on the original image by adopting a bilateral filter, improving the smoothness by combining a Gaussian filter, and converting the filtered image into a floating point type; normal reconstruction: calculating a surface normal vector corresponding to each pixel point through a least square method or robust estimation according to a photometric stereo principle by utilizing a multi-light source matrix form, and generating a normal distribution diagram; curvature calculation: estimating a local curved surface of each pixel neighborhood based on the reconstructed normal graph; and dust identification: through setting a dynamic threshold and a region growing algorithm, extracting a continuous and significant normal disturbance region, and carrying out classification, marking and visual output in combination with characteristics such as area, length, boundary curvature and the like.

An appearance determination device includes at least one processor. The at least one processor carries out a determination step of determining whether an object of determination is conforming or nonconforming, in accordance with a color image and a shape image, the color image being generated by photometric stereo and representing a color optical image of the object of determination, the shape image being generated by the photometric stereo and representing a shape of the object of determination.

The invention discloses a small-size visual tactile sensor based on a UV mark, and relates to the field of sensors. The visual tactile sensor solves the problem that the existing visual tactile sensor cannot realize high-precision force and texture perception on the same sensor at the same time, independently or in a time-sharing manner. A UV fluorescence labeling layer which is transparent under visible light and develops under ultraviolet light is arranged on the contact surface of the elastomer; the composite light source module comprises a UV light source unit and an RGB light source unit which can be independently controlled; the UV light source or the RGB light source is turned on through time-sharing switching, so that the image acquisition module can acquire mark point displacement images used for calculating contact force distribution or photometric stereo images used for reconstructing contact surface textures in a time-sharing manner. According to the invention, the problem that a traditional sensor cannot realize lossless switching of force and texture sensing functions on a single device is fundamentally solved by using the invisibility characteristic of the UV mark and a dual-light-source time-sharing working mode. The method is used for robot touch sensing.

The invention provides a three-dimensional two-dimensional code surface information reconstruction method, device and equipment and a storage medium, and belongs to the technical field of image processing and three-dimensional reconstruction, and the method comprises the steps: carrying out the contrast enhancement of an initial image set of a three-dimensional two-dimensional code based on an exposure fusion contrast enhancement algorithm, and obtaining a contrast-enhanced image set, the initial image set comprises three-dimensional two-dimensional code images obtained by lighting and shooting the three-dimensional two-dimensional code at four different space angles; normal vector field reconstruction is carried out on the surface of the stereo two-dimensional code based on a four-light-source photometric stereo method and a contrast enhancement image set; converting the established normal vector field into a depth image based on a Poisson solution; and obtaining a reconstructed two-dimensional code image based on the depth image. According to the invention, the technical problem that the error of the surface information of the low-contrast stereo two-dimensional code extracted in a complex environment in the prior art is large is effectively solved.

The application belongs to the technical field of three-dimensional measurement, and particularly discloses a metal surface measurement method and system based on photometric stereo and binocular structured light. The method comprises the following steps: acquiring original depth point cloud of a mirror surface region of a surface to be measured by using a binocular camera; acquiring complete normal vector data of the surface to be measured by using a photometric stereo method; and fusing the original depth point cloud and the normal vector data to calculate surface point cloud of the surface to be measured. The measurement data of the binocular structured light and the near-field photometric stereo are combined, the missing measurement data of the metal surface caused by the mirror reflection is complemented, the mutual fusion of the height data and the normal vector data is utilized, and the accuracy of surface reconstruction is improved.

A multispectral luminosity three-dimensional imaging system based on a nanocrystalline color conversion light source comprises a nanocrystalline light source module, an illumination control module, an image acquisition module and a three-dimensional reconstruction calculation module. The nanocrystalline light source module realizes multi-band accurate illumination; the illumination control module realizes independent lighting of a light source channel by channel based on a shift register and a single chip microcomputer, and outputs a synchronous trigger signal to a camera, so that each image corresponds to the unique light source direction; the image acquisition module acquires a multi-wavelength and multi-direction reflection image sequence, and uses a mask to limit an effective area; and the three-dimensional reconstruction calculation module estimates the light source direction, performs photometric stereo solution on each spectral channel, reconstructs a complete three-dimensional surface topography through cross-channel normal fusion and Poisson depth recovery, and generates a multispectral albedo graph and a pseudo-color normal graph. According to the invention, the stability and precision of normal estimation and depth recovery under a complex material surface are improved, and miniaturized, high-fidelity and multiband photometric three-dimensional imaging is realized.

This application proposes a method, controller, system, medium, and product for constructing a large-scale 3D reconstruction model. The method includes: acquiring training images; rendering the training images based on mesh rasterization to obtain a photometric stereo image; inputting the photometric stereo image into a preset explicit network to obtain a three-dimensional representation of the photometric stereo image; rendering the three-dimensional representation to obtain various rendered images, and predicting the albedo of the three-dimensional representation to obtain an albedo map; calculating the loss based on various rendered images, the albedo map, and the photometric stereo image using a preset loss function to obtain a loss value; and optimizing the explicit network based on the loss value to obtain a constructed large-scale 3D reconstruction model. Therefore, this application calculates the loss using various rendered images, the albedo map, and the photometric stereo image, and then optimizes the explicit network based on the loss value to obtain a constructed large-scale 3D reconstruction model, thereby improving the accuracy and quality of the 3D reconstruction model construction.

The invention discloses an intelligent material scanning device and method based on multi-angle polarized illumination, and the device comprises a box body, and a camera assembly, a polarization filter, an illumination assembly and a rotating table which are sequentially distributed in the box body from top to bottom, and the polarization filter is rotatably connected to the front end of an objective lens of the camera assembly. The lighting assembly is provided with a plurality of light sources distributed on the hemispherical surface in a longitude and latitude mode. The camera assembly obtains image data on the rotating table through the polarization filter. The objective of the invention is to design an intelligent material scanning device which is low in cost, high in precision and ecologically open, and based on the combination of a physical polarization means and a photometric stereo method, the generated material is real and uniform under different environment illumination.

The invention discloses an insulator surface defect detection method and system based on photometric stereo vision. The method comprises the following steps: S1, obtaining a plurality of insulator surface images obtained by a photometric stereo method; s2, restoring a 2.5 D depth map of the surface of the insulator from the surface image of the insulator; s3, denoising the 2.5 D depth map on the surface of the insulator; s4, performing key point extraction and curved surface fitting on the denoised depth map to obtain key control points and corresponding curved surfaces as a sample set A; constructing a random control point coordinate combination and a corresponding curved surface as a sample set B; training the feature extraction network through the sample set A and the sample set B to obtain a control point extractor to obtain control points of the curved surface, and obtaining an insulator surface curved surface model; and S5, based on the insulator surface curved surface model, calculating parameter information of the insulator surface, and performing defect detection according to the parameter information. The method has the advantages of high detection precision and the like.

The application provides an inflatable flexible visual-tactile device and method for measuring surface topography of an object. The device comprises an inflatable flexible visual-tactile sensor, a clamp and a support, the inflatable flexible visual-tactile sensor comprises a tactile perception unit and a visual imaging unit; when the surface of the object contacts the sensor, the flexible film and the external reflective layer in the tactile perception unit are deformed, and are attached to the surface of the object under the action of the internal air pressure of the inflatable flexible visual-tactile sensor; the deformed flexible film is reconstructed in three dimensions by using photometric stereo technology, so that the reconstruction of the whole surface topography of the object and the real-time measurement of the size are realized. The device adopts a sealed inflatable structure design, omits the skeleton support to expand the effective measurement range of the flexible film, and simultaneously makes the flexible film closely attached to the surface of the measured object under the action of the air pressure, so that the accurate surface concave-convex feature information of the object is collected, and the reconstruction accuracy and the measurement accuracy of the complex surface topography of the object are significantly improved.

The present invention discloses a surface defect detection method, device, system, and medium. The detection method includes: controlling each light source to light up and cycle in sequence, using a true color line array camera to shoot a moving object to be tested, so as to obtain the original image information collected by each channel of the true color line array camera; splitting the original image information collected by each channel so that the image information corresponding to the same light source is summarized in sequence, thereby obtaining sub-images corresponding to all light sources in each channel; fusing the sub-images corresponding to all light sources in each channel, calculating the average value of all image data corresponding to the same position in each sub-image in the current channel as the fused data at the corresponding position, thereby obtaining color information of the surface of the object to be tested; extracting the sub-images corresponding to all light sources in any channel, and calculating a photometric stereogram based on the angular information of each light source to obtain concave-convex information. The present invention can improve detection efficiency while reducing costs, and realizes the simultaneous acquisition of color information and concave-convex information.

The application discloses a kind of identification system and method based on photometric stereo, comprising: image acquisition module and data processing device, the image acquisition module is according to predetermined illumination mode and object rotation angle and is photographed image, forms a group of image data including multiple different illumination and angle conditions, and image data is uploaded to data processing device inside, the image acquisition device includes: top shell, controller, bottom shell, workpiece moving mechanism and image acquisition mechanism.The identification system and method based on photometric stereo, introduce more advanced fusion strategy, combine the physical illumination model of photometric stereo with the self-attention mechanism of physical perception, strengthen the physical interpretability of features, and then solve the problem that traditional fusion algorithm ignores the physical law of illumination, and realize the multi-angle image acquisition of different height levels of workpiece, obtain the image information of each side and front and back of workpiece, provide rich data for subsequent comprehensive analysis and three-dimensional reconstruction.

The invention relates to the technical field of image processing and machine vision, and discloses a surface reconstruction method and system based on phase deflection and photometric stereo fusion, a computer readable storage medium and a computer program product. The method comprises the following steps: solving a shape graph of an object by using a phase deflection method, and calculating an average curvature graph of the object according to a photometric three-dimensional normal vector; and fusing the shape graph and the average curvature graph to obtain a surface reconstruction result graph of the object. According to the method, the gray scale difference of the gloss ratio, the diffuse reflection image, the mirror reflection image and the average brightness image obtained through calculation with the phase deflection method in different states is simulated and serves as input of the photometric stereo, the curvature image is calculated, the defects of the photometric stereo during mirror reflection processing are overcome, the finer effect can be achieved, and the image quality is improved. And global high-precision reconstruction can be realized.

The present application belongs to the technical field of image processing, and particularly relates to a non-Lambertian surface photometric stereo model and method based on three-dimensional convolution. The present application discloses a non-Lambertian surface photometric stereo model based on three-dimensional convolution, which comprises an information fusion layer, an inter-frame feature extraction layer, a spatial feature extraction layer, a maximum pooling layer and a regression layer. The information fusion layer is used for fusing each image and corresponding light source information, so as to ensure one-to-one correspondence between the image and the light source in subsequent processing. The inter-frame feature extraction layer is used for extracting inter-frame information, obtaining information between input image frames, and estimating the normal map. The spatial feature extraction layer extracts structural information inside a single image for normal map recovery. The maximum pooling layer is used for dimension reduction, removal of redundant information, compression of features, simplification of network complexity, reduction of calculation amount and reduction of memory consumption.

Embodiments of the present application provide a binocular matching method and device and a computer readable storage medium, and relate to the field of computer vision. In the binocular matching method and device, a normal map is obtained by using a photometric stereo method, and the normal map is fused with an original image in a channel, so that the photometric image originally having only intensity features becomes a first multi-dimensional image and a second multi-dimensional image containing normal vectors, many binocular matching features can be increased, the matching effect is improved, and the situation of few textures or no textures in three-dimensional reconstruction can be solved.

The present invention discloses a method and system for photometric stereo reconstruction of worn surface morphology by fusing full-light source images. The method estimates the normal of the worn surface from the photometric stereo input based on a deep learning algorithm, and uses full-light source images taken with all light sources turned on to provide an estimate of the prior depth and its uncertainty for surface reconstruction. The surface normal and the prior depth are fused pixel by pixel based on the uncertainty and geometric constraints. The fused features and the prior depth are further used to directly generate a surface depth map through a network, thereby obtaining the three-dimensional morphology of the worn surface. The entire network model is trained by a designed total loss function and a constructed synthetic wear surface data set, and is applied to the reconstruction of actual worn surface morphology. The present invention provides prior depth information for reconstructing the worn surface morphology by the photometric stereo method by introducing full-light source images without the need for additional devices, thereby improving the warping and distortion problem of the reconstructed wear morphology caused by the non-Lambertian effect and improving the three-dimensional reconstruction accuracy.

Embodiments of the invention provide an image processing method and apparatus, and an electronic device. The method comprises the steps of obtaining images acquired by an image acquisition device when a lens is located at different optical axis positions in different illumination modes; generating a super depth-of-field depth map based on the obtained image; generating a photometric stereo normal vector diagram based on the obtained image, and generating a photometric stereo depth map based on the photometric stereo normal vector diagram; and correcting depth information in the same depth-of-field region in the super depth-of-field depth map according to a photometric stereo normal vector in the photometric stereo normal vector map, and correcting depth information in a low texture region in the super depth-of-field depth map according to a photometric stereo depth value in the photometric stereo depth map. The detail depth information in the same depth-of-field region can be corrected by the photometric stereo normal vector, and the missing depth information in the low-texture region can be restored by the photometric stereo depth value, so that the super depth-of-field depth map which can well reflect the depth details of the collection view field can be obtained.

The invention relates to a visual tactile sensor and a sensing method thereof. The visual tactile perception method is applied to a visual tactile sensor provided with a soft elastic body, and comprises the following steps: configuring a light source, and enabling the light source to irradiate a reflecting medium which is arranged on one side of a contact object of the soft elastic body and has a non-directional reflection characteristic or a partial non-directional reflection characteristic through the soft elastic body; wherein the reflecting medium comprises a first reflecting medium acting on internal environment reflection of the soft elastomer and a second reflecting medium for marking; the albedo of the first reflecting medium is different from that of the second reflecting medium; collecting a visual change image of a local position and a normal line generated by the reflection medium because the reflection medium contacts the object with the soft elastomer; and performing photometric stereo calculation on the visual change image to perceive the mechanical characteristic data of the soft elastomer in the current state. According to the method, the first reflection medium and the second reflection medium with different reflectivity are arranged, so that a normal resolving blind area caused by the fact that a mark destroys a photometric stereo reflection model is eliminated.

The invention relates to the field of robot tactile perception, in particular to an object surface morphology and hardness identification method and system based on visual tactile perception. Comprising the following steps: S1, pressing a to-be-measured object at a fixed displacement through a visual touch sensor, and obtaining a pressure diagram sequence of the to-be-measured object in the pressing process; s21A: obtaining a magnitude sequence of force corresponding to the pressure graph sequence through a first classification neural network model; s22A: calculating the maximum value of the force and the gradient of the force in the magnitude sequence of the force corresponding to the pressure diagram sequence; s21B, analyzing the last frame of pressure diagram in the pressure diagram sequence through a photometric stereo algorithm, and drawing a three-dimensional point cloud view; s22B, the fitting radius of the three-dimensional point cloud view is calculated; s3, splicing the maximum value of the force, the gradient value of the force and the calculation radius to form a feature matrix; and S4, analyzing the feature matrix through a second classification neural network model to obtain the types of the surface morphology and hardness corresponding to the to-be-detected object.

A housing (12) for a photometric stereo scanner (10), the interior of which is easily accessible and which is also suitable for larger planar objects, comprises: - a base (16) having a first side edge (16a), a second side edge (16b) opposite the first side edge, and a front edge (16d) connecting the first and second side edges; - a ceiling (18) on which at least three lamps (38) can be attached in such a way that they are directed towards the floor; - a perimeter wall (20) that connects the floor to the ceiling and has a first and a second wall section (20a, 20b); - a bracket (36) on which a camera (14) can be attached such that the camera is either mounted under the ceiling and directed towards the floor or mounted on the ceiling and directed towards the floor through an opening (32b) in the ceiling; where - the second wall section is essentially designed in the form of a helmet visor and is movably mounted on the floor, the ceiling and / or the first wall section; - the second wall section can assume a closed position in which, together with the floor, ceiling and the first wall section, it encloses an interior space of the enclosure; - the second wall section can assume an open position in which the interior is accessible from the outside and the first and second side edges and the front edge are exposed.

The present application relates to the field of industrial vision detection and automatic manufacturing of clothes, and particularly to a shirt automatic sewing detection system based on industrial vision; comprising: a motion phase monitoring module, a time sequence synchronization control module, a multi-partition directional lighting module, an image acquisition module, a normal vector calculation module and an abnormality discrimination module; monitoring the phase signal of the mechanical mechanism, extracting the target object's stationary time window, and generating a time-sharing trigger signal in the window to activate the independent lighting sectors of different spatial orientations in turn, synchronously executing continuous exposure to obtain multi-orientation illumination images of the same cloth area; based on the photometric stereo algorithm, calculating the surface normal vector data, extracting the three-dimensional topological features, and outputting the abnormality discrimination results of forms such as broken lines, missing seams, wrinkles, floating lines or bulges; the present application realizes high signal-to-noise ratio three-dimensional form detection of the same color and same quality shirt sewing area under high-speed sewing conditions.

The application discloses a kind of real-time surface defect detection device and method of automobile parts friction stir welding, including friction welding component limiting platform, for fixing the measured friction welding component, the measured surface of measured friction welding component has side-by-side first welding edge and second welding edge;Polarized imaging component and polarized light source are relatively displaced in the length direction of first welding edge and second welding edge;Polarized imaging component includes first polarized imaging component and second polarized imaging component;Polarized light source includes first polarized light source and second polarized light source, which simultaneously irradiate first welding edge and second welding edge, the polarization direction of first polarized light source and first polarized imaging component is perpendicular to the light source polarization direction of second polarized light source and second polarized imaging component.The application effectively solves the shortcoming that simple photometric stereo method cannot realize edge moving while photographing, and also solves the defect that simple binocular camera cannot realize defect information strengthening.

This invention discloses a method and system for constructing the true value of the microscopic morphology of the inner wall of a pipeline based on spatial domain triggering and dynamic shutter control, belonging to the field of pipeline inspection technology. It employs a dynamic time-division stroboscopic acquisition strategy based on spatial sampling consistency to ensure that the four image sequences required for photometric stereo vision overlap in physical space. It introduces dynamic shutter control based on instantaneous velocity to reduce the non-uniformity of imaging quality caused by the variable speed movement of multiple sensor modules. A robust photometric stereo algorithm is used to decouple and separate the pure pipe wall texture and micro-geometry, and reconstruct a high-precision 3D depth map. Finally, standardized true value data integrating position, texture, and geometry is constructed. This invention solves the problem in existing photometric stereo vision methods where multiple frames acquired under different lighting conditions are difficult to keep spatially consistent when the carrier is in continuous motion or experiencing mechanical micro-vibration, leading to significant errors in normal vector calculation and 3D reconstruction results.

The present invention discloses a method and system for detecting insulator surface defects based on photometric stereo vision. The method comprises the following steps: S1. acquiring multiple insulator surface images obtained using photometric stereo; S2. restoring a 2.5D depth map of the insulator surface from the insulator surface images; S3. denoising the 2.5D depth map of the insulator surface; S4. performing key point extraction and surface fitting on the denoised depth map to obtain key control points and corresponding surfaces as sample set A; constructing random control point coordinate combinations and corresponding surfaces as sample set B; training a feature extraction network using sample sets A and B to obtain a control point extractor to obtain control points of the surface and obtain an insulator surface model; and S5. calculating parameter information of the insulator surface based on the insulator surface model, and performing defect detection based on the parameter information. The present invention has the advantages of high detection accuracy.

The application relates to the field of building engineering detection and computer vision technology and discloses a building engineering quality defect visual identification and diagnosis method, which comprises the following steps: collecting a multi-illumination direction image sequence, separating an initial surface normal graph and an albedo graph by using photometric stereo calculation, inferring a normalized stiffness field graph through a deep neural network, constructing a discrete physical grid to apply a virtual load, solving a mechanical equilibrium equation to calculate a displacement field and performing a differentiable reverse rendering, reversely optimizing network parameters by using the difference between a rendered image and an observed image, and judging defects and outputting diagnosis data based on the optimized stiffness field graph. By introducing a self-supervised closed loop of physical mechanics constraints, the application effectively eliminates the interference of surface stains on detection, realizes quantitative inference from visual features to material constitutive properties, and improves the robustness and physical interpretability of defect identification.

A structural scanning and analysis apparatus comprises a solid-state light-based scanner arranged to capture a plurality of images of a target surface that is substantially flat and having an area, A. A depth-map generation module generates a depth map of the target surface using surface normal information derived from the plurality of images, wherein the depth map shows 3D topographical attributes of the target surface to an accuracy that may be related to the area imaged or at least ±0.1mm. A feedback module then provides an output based on a result from the analysis module which may be used to control a manufacturing process. Additional inventions claimed relate to a photometric stereo scanner and a feedback visualisation apparatus.

This invention relates to a photometric stereoscopic 2.5D mold monitoring imaging method, comprising a pre-calibration step and a real-time monitoring step. The real-time monitoring step includes: S1: sequentially triggering multiple light sources from different directions to illuminate individually via pulse control signals, and simultaneously triggering a camera to acquire multiple corresponding mold surface images; S2: performing pixel-level synthesis of the multiple mold surface images to generate a stable image with uniform illumination; S3: calculating the pixel-level surface normal vector distribution based on the photometric stereoscopic method; S4: performing integral reconstruction of the normal vector distribution to generate a 2.5D height map; S5: detecting mold surface anomalies based on the stable image and / or the 2.5D height map. This invention, through multi-directional light source collaborative illumination and image synthesis strategy, accurately solves the problem of uneven illumination in unilateral bright and dark areas, obtaining a bright and clear detection image of the entire surface, while also possessing high stability and real-time performance, meeting the needs of industrial mold monitoring.