A method of photometric stereo 2.5D mold monitoring imaging

Abstract

This invention relates to a photometric stereoscopic 2.5D mold monitoring imaging method, comprising a pre-calibration step and a real-time monitoring step. The real-time monitoring step includes: S1: sequentially triggering multiple light sources from different directions to illuminate individually via pulse control signals, and simultaneously triggering a camera to acquire multiple corresponding mold surface images; S2: performing pixel-level synthesis of the multiple mold surface images to generate a stable image with uniform illumination; S3: calculating the pixel-level surface normal vector distribution based on the photometric stereoscopic method; S4: performing integral reconstruction of the normal vector distribution to generate a 2.5D height map; S5: detecting mold surface anomalies based on the stable image and / or the 2.5D height map. This invention, through multi-directional light source collaborative illumination and image synthesis strategy, accurately solves the problem of uneven illumination in unilateral bright and dark areas, obtaining a bright and clear detection image of the entire surface, while also possessing high stability and real-time performance, meeting the needs of industrial mold monitoring.

Description

Technical Field

[0001] This invention relates to the field of industrial mold monitoring technology, specifically to a method for photometric 2.5D mold monitoring imaging. Background Technology

[0002] Industrial molds (such as injection molds and stamping molds) are prone to abnormalities during production, such as foreign object adhesion, surface wear, cracks, or deformation. Real-time monitoring is necessary to prevent defective products or equipment damage. Existing mold monitoring technologies mainly employ single-light source 2D imaging or traditional photometric stereoscopic methods, but these have the following drawbacks: 1) Illumination by a single light source easily produces bright and shadow areas, resulting in the loss of image details; 2) Strong ambient light interference in industrial environments leads to poor imaging stability; 4) The multi-source solution lacks an effective strategy for simultaneous image acquisition and synthesis, making it difficult to obtain uniform 2D images and high-precision 3D information at the same time, and thus failing to meet the online monitoring requirements during the mold opening and closing cycle.

[0003] In view of this, we developed and designed a mold monitoring imaging method that can resist ambient light interference, has stable imaging, strong real-time performance, and can simultaneously provide 2D texture and 2.5D height information. Summary of the Invention

[0004] The purpose of this invention is to provide a method for photometric stereoscopic 2.5D mold monitoring and imaging. By using multi-directional light source pulse synchronous control, pixel-level maximum value synthesis, and photometric stereoscopic reconstruction combined with ambient light compensation, a highly stable and interference-resistant 2.5D imaging and anomaly detection of the mold surface is achieved, solving the problems of uneven imaging, large ambient light interference, and poor real-time performance in existing technologies.

[0005] To achieve the above technical solution, the technical solution of the present invention is as follows: A method for photometric stereoscopic 2.5D mold monitoring and imaging, comprising a pre-calibration step and a real-time monitoring step, wherein the real-time monitoring step adopts a coordinated control method of light source control pulse triggering camera to take pictures, specifically including the following steps: S1: A pulse control signal is generated by the light source controller to trigger the individual lighting of four light sources in different directions in sequence. At the same time, when each light source is lit, the pulse control signal synchronously triggers the camera to take pictures, and the corresponding four mold surface images are collected in sequence. S2: Perform pixel-level synthesis on the four mold surface images to generate a stable image that removes external light interference; S3: Calculate the pixel-level surface normal vector distribution from the four mold surface images based on the photometric stereo method; S4: Integrate and reconstruct the surface normal vector distribution to generate a 2.5D height map of the mold surface; S5: Monitor the surface condition of the mold based on the stable image and / or 2.5D height map, and detect abnormalities.

[0006] Furthermore, in step S1, the four light sources are located in the upper, lower, left, and right directions, or in four directions evenly distributed around the mold. LED surface light sources are used. The light source controller uses pulse control signals to sequentially illuminate each light source. The time interval between illuminating two adjacent light sources is 50-200ms to ensure that there is no light superposition interference when a single light source illuminates independently. The frequency of the pulse control signal is matched with the camera's frame rate to ensure that the camera completes one image acquisition at the same time as each light source is illuminated, and the resolution of the acquired image is not less than 1280×720.

[0007] Furthermore, the pixel-level synthesis in step S2 adopts a maximum value synthesis algorithm, that is, the brightness values ​​of the same pixel position in the four images are compared, and the maximum brightness value is taken as the brightness value of the pixel position in the stable image. In this way, the interference of dark pixels caused by external ambient light and stray light is effectively filtered, and the imaging uniformity is improved.

[0008] Furthermore, in step S3, the relationship between illumination and surface reflectance of the mold is established based on the Lambertian reflection model. The surface normal vector distribution of each pixel is calculated using at least four images under illumination from different directions. During the calculation process, an ambient light compensation coefficient is introduced to remove the influence of ambient light on reflectance and improve the accuracy of normal vector calculation.

[0009] Furthermore, in step S4, the Poisson integral algorithm is used to reconstruct the surface normal vector distribution by integration, generating a 2.5D height map of the mold surface. This height map can characterize the height information of each point on the mold surface relative to the reference plane, and the resolution is consistent with the resolution of the acquired image.

[0010] Furthermore, the anomaly detection in step S5 specifically includes: calculating the pixel-by-pixel deviation between the real-time generated 2.5D height map and the standard 2.5D height map stored in the pre-calibration step; when the deviation value exceeds a preset threshold, it is determined that there is an anomaly; at the same time, performing texture uniformity analysis on the stable image; when local texture abrupt changes or abnormal grayscale values ​​appear in the image, it is determined that there is an anomaly; the anomalies include foreign object adhesion, surface wear, cracks, and deformation.

[0011] Furthermore, the pre-calibration step includes: when the mold is in normal working condition, acquiring and generating a standard stable image and a standard 2.5D height map according to the method of steps S1-S4, and storing them in the database of the industrial control terminal as a comparison benchmark for subsequent real-time monitoring; the pre-calibration process can be re-executed according to the mold model change, maintenance, etc., to update the standard image and standard 2.5D height map.

[0012] Furthermore, the entire imaging cycle (from triggering the first light source to completing the anomaly detection) is less than 1 second, with the image acquisition time of step S1 being less than 300ms, the image processing and reconstruction time of steps S2-S4 being less than 500ms, and the anomaly detection time of step S5 being less than 200ms, supporting real-time online mold monitoring in industrial settings.

[0013] Furthermore, the light source controller communicates with the camera via an industrial Ethernet or RS485 bus to ensure the synchronization of the pulse control signal; the image data acquired by the camera is transmitted to the industrial control terminal through a high-speed data transmission interface for subsequent image synthesis, normal vector calculation, 2.5D reconstruction and anomaly detection processing.

[0014] Compared with the prior art, the present invention has the following beneficial effects: 1) This invention uses pulse control signals to sequentially trigger four light sources in different directions to light up and simultaneously trigger the camera to take pictures, ensuring that the image is only illuminated by a single designated light source during the image acquisition process. At the same time, it combines a pixel-level maximum value synthesis algorithm to generate a stable image, which can effectively filter the interference of external ambient light and stray light, improve imaging stability and uniformity, and solve the problem of poor imaging quality in complex lighting environments in industrial sites. 2) This invention combines photometric stereo method to calculate the surface normal vector distribution from multi-light images and integrates it to generate a 2.5D height map, which can accurately characterize the three-dimensional height information of the mold surface, while retaining the two-dimensional texture information of the stable image. Through the synergistic analysis of two-dimensional and three-dimensional information, the detection accuracy of anomalies such as foreign objects, wear, cracks, and deformation can be greatly improved, overcoming the defect of traditional 2D imaging that cannot identify three-dimensional subtle anomalies. 3) This invention shortens the image acquisition time by optimizing the collaborative control mechanism of the light source and camera. At the same time, it adopts efficient image synthesis, normal vector calculation and integral reconstruction algorithms to ensure that the entire imaging cycle is less than 1 second, which can meet the requirements of real-time online monitoring of industrial molds, realize timely detection and alarm of anomalies, and reduce economic losses. 4) This invention adopts a combination of four LED light sources and a common industrial camera, and realizes pulse collaborative control through the light source controller, eliminating the need for complex 3D scanning equipment and reducing equipment costs; at the same time, the algorithm is simple and efficient, easy to implement on existing industrial control terminals, and facilitates industrial promotion and application. Attached Figure Description

[0015] To further illustrate the various embodiments, the present invention provides accompanying drawings. These drawings are part of the disclosure of the present invention, primarily used to illustrate the embodiments and to explain the operating principles of the embodiments in conjunction with the relevant descriptions in the specification. With reference to these drawings, those skilled in the art should be able to understand other possible implementations and the advantages of the present invention. Components in the drawings are not drawn to scale, and similar component symbols are generally used to represent similar components.

[0016] Figure 1 A flowchart of a method for photometric stereoscopic 2.5D mold monitoring imaging; Figure 2 Images were captured under light sources from four directions; Figure 3 This is a diagram showing the effect of the mold monitoring surface after synthesis. Figure 4 This is the vector graph during synthesis. Detailed Implementation

[0017] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0018] To enable those skilled in the art to better understand the present invention, the present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments.

[0019] Please see the appendix Figures 1 to 4 The diagram illustrates a photometric 2.5D mold monitoring imaging method applied to real-time online monitoring of injection molds. The specific implementation process includes a pre-calibration stage and a real-time monitoring stage. The hardware used includes: four LED surface light sources, a light source controller, an industrial camera, an industrial control terminal, and a data transmission module. The four LED surface light sources are mounted on supports around the mold, located at the top, bottom, left, and right, respectively, at a distance of 5-10 cm from the mold surface. The light source controller is an SMT32 programmable logic controller capable of generating pulse control signals with a pulse width of 1-10 ms and a frequency of 1 kHz. The industrial camera is a 5-megapixel resolution camera with a frame rate of 30 fps. The industrial control terminal is equipped with an Intel CPU, FPFA, or ARM processor for image data processing, 2.5D reconstruction, and anomaly detection.

[0020] Once the injection mold is installed and in normal working condition (no foreign objects, no misalignment, no wear, no cracks, no deformation), perform the pre-calibration steps as follows: The light source controller generates a pulse control signal to sequentially trigger the four LED surface light sources (top, bottom, left, and right) to light up individually. The lighting time interval between two adjacent light sources is set to 1-10ms. At the moment each light source lights up, the pulse control signal synchronously triggers the industrial camera via I / O to take a picture, sequentially acquiring four images of the mold surface in BMP format, which are then directly input into the CPU for image algorithm processing.

[0021] The industrial control terminal performs pixel-level maximum value synthesis processing on the four acquired images: it iterates through each pixel position (x,y) of each image, extracts the brightness values ​​L1(x,y), L2(x,y), L3(x,y), and L4(x,y) of the corresponding positions in the four images, and takes the maximum value Lmax(x,y)=max[L1(x,y),L2(x,y),L3(x,y),L4(x,y)] as the brightness value of the pixel in the stable image, and finally generates a standard stable image and completes storage.

[0022] The pixel-level surface normal vector distribution is calculated based on the photometric stereo method: First, a Lambertian reflection model is established, assuming that the mold surface is an ideal Lambertian body. The relationship between its reflected brightness L and the surface normal vector n and the light source direction vector s is L=ρ(n·s)+Le, where ρ is the surface reflection coefficient and Le is the ambient light brightness. By using the direction vectors s1, s2, s3, and s4 of four different light sources (obtained in advance through calibration), combined with the brightness values ​​of four acquired images, a system of equations is established to solve for the surface normal vector n(x,y) and ambient light brightness Le of each pixel. Then, the influence of ambient light is removed to obtain an accurate surface normal vector distribution.

[0023] The surface normal vector distribution is reconstructed by integrating the Poisson integral algorithm: taking the lower left corner vertex of the mold surface as the reference point and setting its height value to 0, the normal vector of each pixel is integrated by the Poisson integral formula to obtain the height value H(x,y) of each pixel relative to the reference point, generating a standard 2.5D height map, which is stored in the database of the industrial control terminal as a comparison reference for subsequent real-time monitoring.

[0024] During normal operation of the injection mold, a real-time monitoring process is executed every 0.8 seconds. The specific steps are as follows: S1: The light source controller generates pulse control signals according to the parameters in the pre-calibration stage, which sequentially triggers the four LED surface light sources (upper, lower, left, and right) to light up individually, and simultaneously triggers the industrial camera to take pictures, sequentially acquiring four real-time mold surface images, which are then transmitted to the industrial control terminal via high-speed Ethernet.

[0025] S2: The industrial control terminal performs pixel-level maximum value synthesis on the four real-time images to generate a real-time stable image. The specific process is the same as step 2 in the pre-calibration stage.

[0026] S3: Calculate the pixel-level normal vector distribution of the real-time mold surface based on the photometric stereo method. Use the same Lambertian reflection model and solution method as the pre-calibration stage to remove ambient light interference and obtain the real-time surface normal vector distribution.

[0027] S4: The Poisson integral algorithm is used to reconstruct the real-time surface normal vector distribution, generating a real-time 2.5D height map, with the reference point remaining consistent with the pre-calibration stage.

[0028] S5: Anomaly Detection and Alarm: (1) 2.5D height map deviation detection: The real-time 2.5D height map and the standard 2.5D height map stored in the pre-calibration stage are calculated pixel by pixel to obtain the deviation map D(x,y)=|Hreal-time(x,y)-Hstandard(x,y)|; the deviation threshold is set to 0.05mm. When there are 5 or more consecutive pixels in the deviation map with deviation values ​​greater than the threshold, it is determined that there is wear, deformation or foreign matter adhesion abnormality.

[0029] (2) Stable image texture uniformity detection: perform gray-level co-occurrence matrix analysis on the real-time stable image and calculate the texture uniformity index; compare the index with the texture uniformity index of the standard stable image, and when the difference exceeds the preset threshold (0.1), it is determined that there is a crack or surface contamination abnormality.

[0030] (3) If any of the above abnormalities are detected, the industrial control terminal will immediately issue an audible and visual alarm signal and store the abnormal image, 2.5D height map and detection timestamp in the database. At the same time, it will send a control signal to the injection molding machine control system to prompt the operator to check and handle the problem.

[0031] In this embodiment, the imaging cycle of the entire real-time monitoring process is 0.7 seconds, including an image acquisition time of 240ms, an image synthesis time of 80ms, a normal vector calculation time of 150ms, a 2.5D reconstruction time of 130ms, and an anomaly detection time of 100ms, which can meet the requirements of real-time online monitoring of injection molds. Testing showed that this method can accurately detect wear of 0.02mm, cracks of 0.1mm, and foreign matter attachment of 0.05mm on the mold surface, with a detection accuracy of 99.2%. Furthermore, the imaging stability is good under ambient light interference from industrial site natural light and workshop lighting, with no false alarms or missed alarms.

[0032] The above description is merely a preferred embodiment of the present invention and is not intended to limit the present invention in any way. Although the present invention has been disclosed above with reference to preferred embodiments, it is not intended to limit the present invention. Any person skilled in the art should be able to make equivalent embodiments by making some changes or modifications to the above-disclosed technical content without departing from the scope of the present invention. Any simple modifications, equivalent changes, and modifications made to the above embodiments based on the technical essence of the present invention without departing from the scope of the present invention shall still fall within the scope of the present invention.

Claims

1. A method for photometric stereoscopic 2.5D mold monitoring and imaging, characterized in that, It includes a pre-calibration step and a real-time monitoring step, wherein the real-time monitoring step includes: S1: Multiple light sources in different directions are individually lit up sequentially by pulse control signals, and the camera is simultaneously triggered to capture multiple images of the mold surface. S2: Perform pixel-level synthesis on the multiple mold surface images to generate a stable image with uniform illumination; S3: Calculate pixel-level surface normal vector distribution based on photometric stereo method; S4: Perform integral reconstruction on the normal vector distribution to generate a 2.5D height map; S5: Detect mold surface anomalies based on the stabilized image and / or 2.5D height map.

2. The method for photometric stereoscopic 2.5D mold monitoring and imaging as described in claim 1, characterized in that: In step S1, the multiple light sources in different directions are four-directional light sources; the four-directional light sources are LED surface light sources in four directions: up, down, left, and right, or four directions evenly distributed around the monitoring surface of the mold; the LED surface light sources are lit sequentially by the light source controller through pulse control signals, and the time interval between the lighting of two adjacent light sources is 50-200ms; the frequency of the pulse control signal is matched with the camera's shooting frame rate to ensure that the camera completes one image acquisition at the same time as each light source is lit.

3. The method for photometric stereoscopic 2.5D mold monitoring and imaging as described in claim 1, characterized in that: The pixel-level synthesis in step S2 adopts the maximum value synthesis algorithm, that is, the brightness value of the same pixel position in the four images is compared, and the maximum brightness value is taken as the brightness value of the pixel position in the stable image.

4. The method for photometric stereoscopic 2.5D mold monitoring and imaging as described in claim 1, characterized in that: Step S3 uses the Lambertian reflection model to establish the relationship between illumination and the reflectance of the mold surface. It uses images under illumination from four different directions to calculate the surface normal vector distribution of each pixel. During the calculation process, an ambient light compensation coefficient is introduced to remove the influence of ambient light on the reflectance.

5. The method for photometric stereoscopic 2.5D mold monitoring and imaging as described in claim 1, characterized in that: In step S4, the Poisson integral algorithm is used to reconstruct the surface normal vector distribution. The generated 2.5D height map represents the height information of each point on the mold surface relative to the reference plane, and the resolution is consistent with the resolution of the acquired image.

6. The method for photometric stereoscopic 2.5D mold monitoring and imaging as described in claim 1, characterized in that: S5 includes: comparing the real-time 2.5D height map with the standard 2.5D height map stored in the pre-calibration step on a pixel-by-pixel basis, and / or performing texture uniformity analysis on the stable image; the anomaly types include foreign object adhesion, surface wear, cracks and deformation.

7. The method for photometric stereoscopic 2.5D mold monitoring and imaging as described in claim 1, characterized in that: The pre-calibration step includes: performing steps S1-S4 under normal mold conditions to generate and store a standard stable image and a standard 2.5D height map as a comparison benchmark.

Images