118 results about "Photometric stereo" patented technology

The invention provides a machine vision-based heavy-load large holding tool driving wheel multi-mode damage detection method, and relates to the technical field of automatic detection, and the method comprises the steps: collecting the image, vibration and temperature data of a driving wheel, extracting the visual features of the image through a multi-scale pyramid network, and obtaining the multi-mode damage of the driving wheel; inputting a cross-modal feature fusion network in combination with eccentricity and runout features of the vibration signals and stress concentration features of the temperature data to obtain fusion features; inputting the fusion features into a multi-task detection module to identify the damage, and measuring the damage area, depth, crack growth rate and stress concentration degree based on image segmentation, a photometric stereo method, vibration signal analysis and temperature gradient analysis; and finally, in combination with the equipment operation condition and the damage evolution law, an improved evidence theory is adopted to fuse the evaluation indexes, and an evaluation report including the damage state, the development trend and the maintenance suggestion is generated.

The invention discloses an image recognition system for industrial part defect detection, and particularly relates to the field of part defect detection, which comprises a multi-axis controllable light source array module, a high-speed polarization camera module, an edge computing node module, a double-branch semantic segmentation network module and a physical constraint post-processing module, according to the invention, through combination of time-sharing stroboscopic illumination and polarization image sequence acquisition, multi-dimensional perception of surface topography and material differences is realized; generating an elevation map and a normal map by using photometric stereo solution, constructing a differential rendering layer reverse matching CAD model, and extracting flash sensitive features; a double-branch U-Net network is adopted to fuse geometric and polarization characteristics, the characterization capability is enhanced through a trans-attention mechanism, and a pixel-level mask is output; and finally, mapping a two-dimensional result to a three-dimensional coordinate system by means of calibration parameters, carrying out geometric verification in combination with a tolerance zone and a height threshold value, and automatically generating a structured defect report containing position, size, grade and visual information.

Disclosed in the present invention is a photometric stereo-based device for rapidly detecting surface features of an object, capable of rapidly acquiring a plurality of images meeting the requirements of a photometric stereo algorithm. The device comprises a light emitting assembly, an acquisition assembly located above the light emitting assembly, and a data processing module; the acquisition assembly comprises a light path system and a camera group; the light path system comprises: a beam splitting prism group used for splitting reflected light from the surface of an object into a plurality of beams of light and respectively outputting the plurality of beams of light in different preset directions to form images to be processed, and a plurality of bandpass filters correspondingly arranged on the output sides of the plurality of beams of light used for forming said images, wherein the central wavelengths of the bandpass filters are respectively configured to be different wavebands of the reflected light from the surface of the object; the camera group comprises a plurality of cameras which are arranged behind the bandpass filters in a one-to-one correspondence mode and acquire filtered light beams; and the data processing module receives image information sent by the cameras, and performs compensation and photometric stereo reconstruction on the image information, so as to obtain surface features of the object. Further disclosed is a method implemented on the basis of the device.

Photometric stereo techniques enable using a single camera to perform an enrollment process for creating a user-specific anatomical model of an eye for gaze tracking. The user-specific anatomical model includes information about a user's center of vision at multiple dilation states of the eye, which can be used to enhance the accuracy of gaze tracking techniques. Accurate gaze tracking techniques enable the use of gaze tracking at close range, for example, gaze tracking within a head-mounted display device.

The invention discloses an electronic screen film surface dust detection method based on photometric stereo. The method comprises the following steps: image acquisition: acquiring 4-8 grayscale images in different incident directions; image preprocessing: carrying out edge-preserving noise reduction processing on the original image by adopting a bilateral filter, improving the smoothness by combining a Gaussian filter, and converting the filtered image into a floating point type; normal reconstruction: calculating a surface normal vector corresponding to each pixel point through a least square method or robust estimation according to a photometric stereo principle by utilizing a multi-light source matrix form, and generating a normal distribution diagram; curvature calculation: estimating a local curved surface of each pixel neighborhood based on the reconstructed normal graph; and dust identification: through setting a dynamic threshold and a region growing algorithm, extracting a continuous and significant normal disturbance region, and carrying out classification, marking and visual output in combination with characteristics such as area, length, boundary curvature and the like.

The invention discloses a hub defect detection method based on a 2.5 D photometric stereo image. The method comprises the following steps: sending an acquired preprocessed hub image into a pre-trained deep learning model for identifying and positioning coating hub defects so as to output a detection result; wherein the preprocessing of the hub image comprises the following steps: acquiring images of a coated hub under various exposures through a camera, and fusing the images under the various exposures by adopting an exposure fusion algorithm to obtain the preprocessed hub image. The 2.5 D photometric stereo image technology is utilized, and supervised learning and unsupervised learning methods are combined, so that the detection of the defects of the high-reflective coating hubs with different colors is realized.

An appearance determination device includes at least one processor. The at least one processor carries out a determination step of determining whether an object of determination is conforming or nonconforming, in accordance with a color image and a shape image, the color image being generated by photometric stereo and representing a color optical image of the object of determination, the shape image being generated by the photometric stereo and representing a shape of the object of determination.

The invention provides a device for identifying tire sidewall characters and an identification method. The device comprises a conveying unit used for conveying a tire to a to-be-detected area, a lifting platform used for jacking the tire to an identification position, a light source assembly used for illuminating the tire, and a camera used for shooting a top view image of the tire. The data processing module is used for controlling the camera to shoot the top view image of the tire, determining gradient information in the image of the tire according to the top view image of the tire and a photometric stereo algorithm, and identifying characters of the tire according to the gradient information in the image of the tire. According to the technical scheme, the camera and the light source assembly are matched to photograph the tire, and the gradient information in the image of the tire is confirmed through a three-dimensional algorithm, so that more accurate character information on the tire can be obtained. The characters of the tire are recognized through the gradient information in the image of the tire, the character recognition accuracy is improved, the whole recognition process is automatically achieved, and the recognition accuracy and recognition efficiency are improved.

The invention relates to the technical field of image processing, and discloses a phase deflection and photometric stereo fused surface reconstruction method and system, a computer readable storage medium and a computer program product. The method comprises the following steps: respectively acquiring a plurality of fringe patterns with different phases in a first direction and a second direction of an object to be imaged; a reflection image of the object needing to be imaged after the binary images of the object needing to be imaged are projected in the multiple different directions is captured; a phase deflection method is used for solving a shape graph of the object, a captured reflection graph is used as photometric stereo input, an intensity matrix is constructed, and an average curvature graph of the object is calculated; and fusing the shape graph and the average curvature graph to obtain a final result graph. According to the method, the respective advantages of PMD and photometric stereo are comprehensively utilized, the adaptability to a complex surface is enhanced by dynamic weight, a finer effect can be achieved, and the multi-scene application capability of the prior art can also be improved.

The invention discloses a small-size visual tactile sensor based on a UV mark, and relates to the field of sensors. The visual tactile sensor solves the problem that the existing visual tactile sensor cannot realize high-precision force and texture perception on the same sensor at the same time, independently or in a time-sharing manner. A UV fluorescence labeling layer which is transparent under visible light and develops under ultraviolet light is arranged on the contact surface of the elastomer; the composite light source module comprises a UV light source unit and an RGB light source unit which can be independently controlled; the UV light source or the RGB light source is turned on through time-sharing switching, so that the image acquisition module can acquire mark point displacement images used for calculating contact force distribution or photometric stereo images used for reconstructing contact surface textures in a time-sharing manner. According to the invention, the problem that a traditional sensor cannot realize lossless switching of force and texture sensing functions on a single device is fundamentally solved by using the invisibility characteristic of the UV mark and a dual-light-source time-sharing working mode. The method is used for robot touch sensing.

The invention provides a three-dimensional two-dimensional code surface information reconstruction method, device and equipment and a storage medium, and belongs to the technical field of image processing and three-dimensional reconstruction, and the method comprises the steps: carrying out the contrast enhancement of an initial image set of a three-dimensional two-dimensional code based on an exposure fusion contrast enhancement algorithm, and obtaining a contrast-enhanced image set, the initial image set comprises three-dimensional two-dimensional code images obtained by lighting and shooting the three-dimensional two-dimensional code at four different space angles; normal vector field reconstruction is carried out on the surface of the stereo two-dimensional code based on a four-light-source photometric stereo method and a contrast enhancement image set; converting the established normal vector field into a depth image based on a Poisson solution; and obtaining a reconstructed two-dimensional code image based on the depth image. According to the invention, the technical problem that the error of the surface information of the low-contrast stereo two-dimensional code extracted in a complex environment in the prior art is large is effectively solved.

The application belongs to the technical field of three-dimensional measurement, and particularly discloses a metal surface measurement method and system based on photometric stereo and binocular structured light. The method comprises the following steps: acquiring original depth point cloud of a mirror surface region of a surface to be measured by using a binocular camera; acquiring complete normal vector data of the surface to be measured by using a photometric stereo method; and fusing the original depth point cloud and the normal vector data to calculate surface point cloud of the surface to be measured. The measurement data of the binocular structured light and the near-field photometric stereo are combined, the missing measurement data of the metal surface caused by the mirror reflection is complemented, the mutual fusion of the height data and the normal vector data is utilized, and the accuracy of surface reconstruction is improved.

A multispectral luminosity three-dimensional imaging system based on a nanocrystalline color conversion light source comprises a nanocrystalline light source module, an illumination control module, an image acquisition module and a three-dimensional reconstruction calculation module. The nanocrystalline light source module realizes multi-band accurate illumination; the illumination control module realizes independent lighting of a light source channel by channel based on a shift register and a single chip microcomputer, and outputs a synchronous trigger signal to a camera, so that each image corresponds to the unique light source direction; the image acquisition module acquires a multi-wavelength and multi-direction reflection image sequence, and uses a mask to limit an effective area; and the three-dimensional reconstruction calculation module estimates the light source direction, performs photometric stereo solution on each spectral channel, reconstructs a complete three-dimensional surface topography through cross-channel normal fusion and Poisson depth recovery, and generates a multispectral albedo graph and a pseudo-color normal graph. According to the invention, the stability and precision of normal estimation and depth recovery under a complex material surface are improved, and miniaturized, high-fidelity and multiband photometric three-dimensional imaging is realized.

This application proposes a method, controller, system, medium, and product for constructing a large-scale 3D reconstruction model. The method includes: acquiring training images; rendering the training images based on mesh rasterization to obtain a photometric stereo image; inputting the photometric stereo image into a preset explicit network to obtain a three-dimensional representation of the photometric stereo image; rendering the three-dimensional representation to obtain various rendered images, and predicting the albedo of the three-dimensional representation to obtain an albedo map; calculating the loss based on various rendered images, the albedo map, and the photometric stereo image using a preset loss function to obtain a loss value; and optimizing the explicit network based on the loss value to obtain a constructed large-scale 3D reconstruction model. Therefore, this application calculates the loss using various rendered images, the albedo map, and the photometric stereo image, and then optimizes the explicit network based on the loss value to obtain a constructed large-scale 3D reconstruction model, thereby improving the accuracy and quality of the 3D reconstruction model construction.

The invention discloses an intelligent material scanning device and method based on multi-angle polarized illumination, and the device comprises a box body, and a camera assembly, a polarization filter, an illumination assembly and a rotating table which are sequentially distributed in the box body from top to bottom, and the polarization filter is rotatably connected to the front end of an objective lens of the camera assembly. The lighting assembly is provided with a plurality of light sources distributed on the hemispherical surface in a longitude and latitude mode. The camera assembly obtains image data on the rotating table through the polarization filter. The objective of the invention is to design an intelligent material scanning device which is low in cost, high in precision and ecologically open, and based on the combination of a physical polarization means and a photometric stereo method, the generated material is real and uniform under different environment illumination.

A method includes capturing a first image of a surface illuminated from a first direction along an axis by a first light at a first elevation angle with other lights of the illumination system off, and capturing a second image of the surface illuminated from a second direction along the axis by a second light at a second elevation angle with the other lights off, the second direction being opposite the first direction, the second elevation angle being substantially identical to the first elevation angle. The images are processed by taking a derivative of a reflected light intensity at each of a plurality of pixel locations in the images, respectively. The respective derivatives are employed to identify a set of invalid pixels where a photometric stereo condition is not satisfied. A photometric stereo reconstruction is generated based on a set of images and the identified set of invalid pixels.

The invention discloses an insulator surface defect detection method and system based on photometric stereo vision. The method comprises the following steps: S1, obtaining a plurality of insulator surface images obtained by a photometric stereo method; s2, restoring a 2.5 D depth map of the surface of the insulator from the surface image of the insulator; s3, denoising the 2.5 D depth map on the surface of the insulator; s4, performing key point extraction and curved surface fitting on the denoised depth map to obtain key control points and corresponding curved surfaces as a sample set A; constructing a random control point coordinate combination and a corresponding curved surface as a sample set B; training the feature extraction network through the sample set A and the sample set B to obtain a control point extractor to obtain control points of the curved surface, and obtaining an insulator surface curved surface model; and S5, based on the insulator surface curved surface model, calculating parameter information of the insulator surface, and performing defect detection according to the parameter information. The method has the advantages of high detection precision and the like.

The present invention relates to an industrial appearance defect detection method based on photometric stereo and diffusion model. First, a multi-intensity fusion photometric stereo method is used to construct a defect-free normal sample data set, and a condition-guided diffusion model deep learning network is designed and constructed. In the deep learning network, the input data of the noise module are defect-free normal samples, random Gaussian noise and random mask images, and the input of the denoising module is the noisy samples, time control tensors and comprehensive features; the encoder and decoder include a residual module and time embedding, and the residual module is responsible for introducing the information of the time embedding. At the same time, the attention mechanism of the residual module can make the network training faster; the trained model is used to perform defect detection and threshold segmentation on the input image. The present invention combines the photometric stereo with the diffusion model, and at the same time uses a large number of defect-free samples in industrial production, combined with the training strategy of unsupervised learning, to study and explore a method suitable for industrial appearance defect detection.

The application provides an inflatable flexible visual-tactile device and method for measuring surface topography of an object. The device comprises an inflatable flexible visual-tactile sensor, a clamp and a support, the inflatable flexible visual-tactile sensor comprises a tactile perception unit and a visual imaging unit; when the surface of the object contacts the sensor, the flexible film and the external reflective layer in the tactile perception unit are deformed, and are attached to the surface of the object under the action of the internal air pressure of the inflatable flexible visual-tactile sensor; the deformed flexible film is reconstructed in three dimensions by using photometric stereo technology, so that the reconstruction of the whole surface topography of the object and the real-time measurement of the size are realized. The device adopts a sealed inflatable structure design, omits the skeleton support to expand the effective measurement range of the flexible film, and simultaneously makes the flexible film closely attached to the surface of the measured object under the action of the air pressure, so that the accurate surface concave-convex feature information of the object is collected, and the reconstruction accuracy and the measurement accuracy of the complex surface topography of the object are significantly improved.

An imaging device includes an imaging unit, a signal separating unit, a normal line calculating unit, and a distance estimation unit. The imaging unit includes a plurality of polarization lights having different polarization directions of light emitted to a subject and a polarization sensor that captures an image of the subject that is simultaneously irradiated with the light from the plurality of polarization lights. The signal separating unit separates pixel signals corresponding to each of the polarization directions from the image captured by the imaging unit and generates an image for every polarization direction. The normal line calculating unit calculates a normal line on a surface of the subject from the image for each of the polarization directions by photometric stereo. The distance estimation unit estimates the shape of the subject based on the normal line calculated by the normal line calculating unit.

The present invention discloses a surface defect detection method, device, system, and medium. The detection method includes: controlling each light source to light up and cycle in sequence, using a true color line array camera to shoot a moving object to be tested, so as to obtain the original image information collected by each channel of the true color line array camera; splitting the original image information collected by each channel so that the image information corresponding to the same light source is summarized in sequence, thereby obtaining sub-images corresponding to all light sources in each channel; fusing the sub-images corresponding to all light sources in each channel, calculating the average value of all image data corresponding to the same position in each sub-image in the current channel as the fused data at the corresponding position, thereby obtaining color information of the surface of the object to be tested; extracting the sub-images corresponding to all light sources in any channel, and calculating a photometric stereogram based on the angular information of each light source to obtain concave-convex information. The present invention can improve detection efficiency while reducing costs, and realizes the simultaneous acquisition of color information and concave-convex information.

The application discloses a kind of identification system and method based on photometric stereo, comprising: image acquisition module and data processing device, the image acquisition module is according to predetermined illumination mode and object rotation angle and is photographed image, forms a group of image data including multiple different illumination and angle conditions, and image data is uploaded to data processing device inside, the image acquisition device includes: top shell, controller, bottom shell, workpiece moving mechanism and image acquisition mechanism.The identification system and method based on photometric stereo, introduce more advanced fusion strategy, combine the physical illumination model of photometric stereo with the self-attention mechanism of physical perception, strengthen the physical interpretability of features, and then solve the problem that traditional fusion algorithm ignores the physical law of illumination, and realize the multi-angle image acquisition of different height levels of workpiece, obtain the image information of each side and front and back of workpiece, provide rich data for subsequent comprehensive analysis and three-dimensional reconstruction.

The invention relates to the technical field of image processing and machine vision, and discloses a surface reconstruction method and system based on phase deflection and photometric stereo fusion, a computer readable storage medium and a computer program product. The method comprises the following steps: solving a shape graph of an object by using a phase deflection method, and calculating an average curvature graph of the object according to a photometric three-dimensional normal vector; and fusing the shape graph and the average curvature graph to obtain a surface reconstruction result graph of the object. According to the method, the gray scale difference of the gloss ratio, the diffuse reflection image, the mirror reflection image and the average brightness image obtained through calculation with the phase deflection method in different states is simulated and serves as input of the photometric stereo, the curvature image is calculated, the defects of the photometric stereo during mirror reflection processing are overcome, the finer effect can be achieved, and the image quality is improved. And global high-precision reconstruction can be realized.

The present application belongs to the technical field of image processing, and particularly relates to a non-Lambertian surface photometric stereo model and method based on three-dimensional convolution. The present application discloses a non-Lambertian surface photometric stereo model based on three-dimensional convolution, which comprises an information fusion layer, an inter-frame feature extraction layer, a spatial feature extraction layer, a maximum pooling layer and a regression layer. The information fusion layer is used for fusing each image and corresponding light source information, so as to ensure one-to-one correspondence between the image and the light source in subsequent processing. The inter-frame feature extraction layer is used for extracting inter-frame information, obtaining information between input image frames, and estimating the normal map. The spatial feature extraction layer extracts structural information inside a single image for normal map recovery. The maximum pooling layer is used for dimension reduction, removal of redundant information, compression of features, simplification of network complexity, reduction of calculation amount and reduction of memory consumption.

Embodiments of the present application provide a binocular matching method and device and a computer readable storage medium, and relate to the field of computer vision. In the binocular matching method and device, a normal map is obtained by using a photometric stereo method, and the normal map is fused with an original image in a channel, so that the photometric image originally having only intensity features becomes a first multi-dimensional image and a second multi-dimensional image containing normal vectors, many binocular matching features can be increased, the matching effect is improved, and the situation of few textures or no textures in three-dimensional reconstruction can be solved.

The present invention discloses a method and system for photometric stereo reconstruction of worn surface morphology by fusing full-light source images. The method estimates the normal of the worn surface from the photometric stereo input based on a deep learning algorithm, and uses full-light source images taken with all light sources turned on to provide an estimate of the prior depth and its uncertainty for surface reconstruction. The surface normal and the prior depth are fused pixel by pixel based on the uncertainty and geometric constraints. The fused features and the prior depth are further used to directly generate a surface depth map through a network, thereby obtaining the three-dimensional morphology of the worn surface. The entire network model is trained by a designed total loss function and a constructed synthetic wear surface data set, and is applied to the reconstruction of actual worn surface morphology. The present invention provides prior depth information for reconstructing the worn surface morphology by the photometric stereo method by introducing full-light source images without the need for additional devices, thereby improving the warping and distortion problem of the reconstructed wear morphology caused by the non-Lambertian effect and improving the three-dimensional reconstruction accuracy.

Embodiments of the invention provide an image processing method and apparatus, and an electronic device. The method comprises the steps of obtaining images acquired by an image acquisition device when a lens is located at different optical axis positions in different illumination modes; generating a super depth-of-field depth map based on the obtained image; generating a photometric stereo normal vector diagram based on the obtained image, and generating a photometric stereo depth map based on the photometric stereo normal vector diagram; and correcting depth information in the same depth-of-field region in the super depth-of-field depth map according to a photometric stereo normal vector in the photometric stereo normal vector map, and correcting depth information in a low texture region in the super depth-of-field depth map according to a photometric stereo depth value in the photometric stereo depth map. The detail depth information in the same depth-of-field region can be corrected by the photometric stereo normal vector, and the missing depth information in the low-texture region can be restored by the photometric stereo depth value, so that the super depth-of-field depth map which can well reflect the depth details of the collection view field can be obtained.

The invention discloses a visual detection method and device for a bending area of an automobile guide arm. The method comprises the following steps: acquiring point cloud data of a to-be-detected guide arm; point cloud matching is conducted on the point cloud data of the to-be-detected guide arm and preset reference point cloud data, the spatial position of the to-be-detected guide arm relative to a visual detection mechanism is obtained, the conversion relation between the position of the to-be-detected guide arm and a coordinate system of the visual detection mechanism is obtained, and the visual detection mechanism comprises a machine visual camera; moving the machine vision camera to an image acquisition position of an upper bending area of the to-be-detected guide arm according to the conversion relation; illuminating the upper bending area of the to-be-detected guide arm from a plurality of different angles and synchronously acquiring images to generate a photometric stereo image of the upper bending area; and detecting whether cracks exist in the upper bending area based on the photometric stereo image of the upper bending area. According to the invention, the problems of inaccurate and unstable image acquisition, low detection reliability and low detection efficiency in the prior art are solved.

The invention relates to a visual tactile sensor and a sensing method thereof. The visual tactile perception method is applied to a visual tactile sensor provided with a soft elastic body, and comprises the following steps: configuring a light source, and enabling the light source to irradiate a reflecting medium which is arranged on one side of a contact object of the soft elastic body and has a non-directional reflection characteristic or a partial non-directional reflection characteristic through the soft elastic body; wherein the reflecting medium comprises a first reflecting medium acting on internal environment reflection of the soft elastomer and a second reflecting medium for marking; the albedo of the first reflecting medium is different from that of the second reflecting medium; collecting a visual change image of a local position and a normal line generated by the reflection medium because the reflection medium contacts the object with the soft elastomer; and performing photometric stereo calculation on the visual change image to perceive the mechanical characteristic data of the soft elastomer in the current state. According to the method, the first reflection medium and the second reflection medium with different reflectivity are arranged, so that a normal resolving blind area caused by the fact that a mark destroys a photometric stereo reflection model is eliminated.

The embodiment of the invention provides a training method of a three-dimensional reconstruction prediction model and a three-dimensional reconstruction method, and relates to the technical field of image processing, and the training method of the three-dimensional reconstruction prediction model comprises the steps: obtaining a plurality of image groups obtained by photographing an object through a camera, each image group comprises a plurality of images with different relative angles between the object and the light source and the same shooting view angle, different image groups have different shooting view angles, generating a first feature map by using a pre-trained photometric stereo prediction model according to the plurality of image groups, and obtaining a second feature map according to a first loss function and a second loss function; model parameters in the three-dimensional reconstruction prediction model are trained, the first loss function represents the difference between the first feature map and the second feature map, the second feature map is determined according to the middle layer output of the three-dimensional reconstruction prediction model, and the second loss function represents the difference between the three-dimensional object information and the three-dimensional object reference information; and the reconstruction result of the three-dimensional reconstruction prediction model for the object carries more surface details.