A photometric stereo imaging device based on sensor triggering and motion timing compensation

Abstract

The utility model relates to machine vision detection technical field, concretely is a kind of photometric stereo imaging device based on sensor trigger and motion timing compensation, including sensor detection unit, optical flow calculation module, synchronous control board card, trigger advance compensation module and imaging system;Sensor detection unit and imaging system are located above detection area, the output of sensor detection unit is connected with synchronous control board card, the output of synchronous control board card is connected with imaging system, and imaging system includes industrial camera and multi-partition annular light source assembly, and industrial camera output is connected with optical flow calculation module, and the output of optical flow calculation module is connected with trigger advance compensation module and synchronous control board card, and trigger advance compensation module input is also connected with pipeline encoder, and the output of trigger advance compensation module is connected with synchronous control board card. Through the optimization hardware structure and connection relation, realize high-precision detection under dynamic scene.

Description

Technical Field

[0001] This utility model relates to the field of machine vision inspection technology, and in particular to a photometric stereo imaging device based on sensor triggering and motion timing compensation. Background Technology

[0002] In the field of industrial automation inspection, photometric stereo imaging technology acquires images through multi-angle illumination to achieve high-precision 3D reconstruction of object surfaces, and is widely used in product defect detection and dimensional measurement. However, when this technology is applied to dynamic production line scenarios, existing devices have the following technical shortcomings: the hardware structure related to motion timing compensation is imperfect, resulting in untimely motion blur correction and imaging misalignment errors exceeding 0.5mm in high-speed scenarios; the signal transmission delay between timing control-related hardware is large, and the timing error often exceeds 1ms, making it difficult to adapt to production line speed fluctuations; the light source trigger control structure is fixed and cannot be flexibly adjusted according to the movement direction of the parts, resulting in the accumulation of positional deviations across multiple frames of images, affecting the accuracy of 3D reconstruction.

[0003] The aforementioned problems result in traditional photometric stereoscopic devices being inefficient and lacking in accuracy during automated testing. Utility Model Content

[0004] In view of the above-mentioned shortcomings and deficiencies of the prior art, this utility model provides a photometric stereo imaging device based on sensor triggering and motion timing compensation, which achieves high-precision detection in dynamic scenes by optimizing the hardware structure and connection relationship.

[0005] To achieve the above objectives, the main technical solutions adopted by this utility model include:

[0006] A photometric stereo imaging device based on sensor triggering and motion timing compensation includes a sensor detection unit, an optical flow calculation module, a synchronization control board, a trigger lead compensation module, and an imaging system.

[0007] The sensor detection unit and the imaging system are located above the detection area. The output of the sensor detection unit is connected to the input of the synchronization control board, and the output of the synchronization control board is connected to the input of the imaging system. The imaging system includes an industrial camera and a multi-zone ring light source assembly. The output of the industrial camera is connected to the input of the optical flow calculation module. The output of the optical flow calculation module is connected to both the trigger advance compensation module and the input of the synchronization control board. The input of the trigger advance compensation module is also connected to the output of the production line encoder. The output of the trigger advance compensation module is connected to the input of the synchronization control board.

[0008] Furthermore, the optical flow computing module includes an FPGA hardware acceleration module.

[0009] Furthermore, the output of the synchronization control board is connected to the imaging system via an optical fiber link.

[0010] Furthermore, the synchronization control board includes a clock chip module, a timing calibration circuit module, an FPGA core control module, and an optical fiber transceiver module. The output of the clock chip module is connected to the input of the timing calibration circuit module, the output of the timing calibration circuit module is connected to the input of the FPGA core control module, the input of the FPGA core control module is also connected to the outputs of the optical flow calculation module and the trigger advance compensation module, the output of the FPGA core control module is connected to the input of the optical fiber transceiver module and the multi-zone ring light source assembly, the input of the optical fiber transceiver module is also connected to the output of the sensor detection unit, and the output of the optical fiber transceiver module is connected to the input of the industrial camera.

[0011] Furthermore, the multi-zone ring light source assembly is an integrated ring structure, which is divided into 4 independent and controllable light source zones along the circumference, with an azimuth angle interval of 90° between adjacent zones; each zone is equipped with an LED light source array, and the line connecting the center of each zone and the imaging center point is the light source center line, with the angle between the light source center line and the normal of the imaging plane being 45°±2°, and the light intensity of each zone can be adjusted independently.

[0012] Furthermore, the industrial camera is connected to four light source zones with linkage control circuits, and each light source zone triggers a single exposure synchronously when it flashes, with a single exposure time ≤50μs.

[0013] Furthermore, it also includes a support structure, which is n-shaped. The bottom ends of the support structure are connected to the two ends of the pipeline width direction. The multi-zone ring light source assembly is rigidly connected to the support structure. The flatness of the connection surface is ≤0.02mm, and the spatial position accuracy of the four light source zones is ≤0.1mm.

[0014] Furthermore, the industrial camera is fixed at the center of the support structure, and the optical axis of the industrial camera coincides with the geometric center of the multi-zone ring light source assembly and is perpendicular to the conveyor plane of the production line.

[0015] Furthermore, the bottom of the support structure is provided with a leveling mechanism, and the leveling mechanism is provided with a shock-absorbing pad.

[0016] The beneficial effects of this utility model are:

[0017] This invention relates to a photometric stereoscopic imaging device based on sensor triggering and motion timing compensation. By optimizing the hardware structure and connection relationships, the imaging misalignment error is controlled within 0.1mm at a pipeline speed of 800mm / s, which significantly improves the dynamic detection accuracy compared to traditional devices.

[0018] By using FPGA hardware acceleration, the optical flow calculation latency is reduced to ≤100ns, which is 80% less than that of traditional software algorithms. It is adapted to a pipeline speed of 800mm / s and improves the accuracy of motion timing compensation.

[0019] The hardware design of the fiber optic link (delay < 45ns) and the phase-locked loop calibration circuit (phase jitter < 4ns) ensures that the trigger interval of multiple modules is ≤ 50ns, the dynamic detection misalignment error is ≤ 0.1mm, and the timing synchronization error is reduced. The phase-locked loop calibration circuit is the sequential connection of the clock chip module, the timing calibration circuit module, the FPGA core control module, and the multi-zone ring light source assembly.

[0020] The independent and controllable structure of the multi-zone ring light source component enables flexible illumination from multiple angles. When inspecting reflective metal parts, the image clarity is greatly improved. At the same time, each zone supports independent light intensity adjustment (duty cycle 5%-95%), adapting to defect detection of parts made of different materials (such as reflective metals), improving contrast by 40%, and enhancing the flexibility of light source triggering. Attached Figure Description

[0021] Figure 1 This is a schematic diagram of the overall structure of a photometric stereo imaging device based on sensor triggering and motion timing compensation according to this utility model;

[0022] Figure 2 This is a schematic diagram of the synchronous control board circuit structure of a photometric stereo imaging device based on sensor triggering and motion timing compensation according to this utility model;

[0023] Figure 3 This is a schematic diagram of the imaging system structure of a photometric stereoscopic imaging device based on sensor triggering and motion timing compensation according to this utility model.

[0024] In the diagram: 1. Sensor detection unit; 2. Optical flow calculation module; 3. Imaging system; 3-1. Multi-zone ring light source assembly; 3-2. Industrial camera; 4. Synchronization control board; 4-1. FPGA core control module; 4-2. Fiber optic transceiver module; 4-3. Clock chip module; 4-4. Timing calibration circuit module; 5. Trigger advance compensation module; 6. Support structure; 7. Leveling mechanism; 8. Production line encoder. Detailed Implementation

[0025] To better explain and facilitate understanding of this utility model, it will be described in detail below with reference to the accompanying drawings and specific embodiments. To better understand the above technical solutions, exemplary embodiments of this utility model will be described in more detail below with reference to the accompanying drawings. Although exemplary embodiments of this utility model are shown in the drawings, it should be understood that this utility model can be implemented in various forms and should not be limited to the embodiments set forth herein. Rather, these embodiments are provided to enable a clearer and more thorough understanding of this utility model and to fully convey the scope of this utility model to those skilled in the art.

[0026] like Figure 1 As shown, a photometric stereo imaging device based on sensor triggering and motion timing compensation includes a sensor detection unit 1, an optical flow calculation module 2, a synchronization control board 4, a trigger advance compensation module 5, and an imaging system 3.

[0027] The sensor detection unit 1 and the imaging system 3 are located above the detection area. The output of the sensor detection unit 1 is connected to the input of the synchronization control board 4. The output of the synchronization control board 4 is connected to the input of the imaging system 3. The imaging system 3 includes an industrial camera 3-2 and a multi-zone ring light source assembly 3-1. The output of the industrial camera 3-2 is connected to the input of the optical flow calculation module 2. The output of the optical flow calculation module 2 is simultaneously connected to the input of the trigger advance compensation module 5 and the synchronization control board 4. The input of the trigger advance compensation module 5 is also connected to the output of the production line encoder 8. The output of the trigger advance compensation module 5 is connected to the input of the synchronization control board 4.

[0028] Motion timing compensation refers to the following: the trigger advance compensation module 5 uses the speed-advance correspondence stored in the hardware ROM and combines it with the real-time motion data output by the optical flow calculation module 2 to generate a trigger offset command. The FPGA of the synchronization control board 4 adjusts the triggering time of the light source and the camera to offset the imaging deviation caused by the movement of the parts.

[0029] Specifically, sensor detection unit 1 is installed 100mm upstream of the conveyor in the production line direction. It uses a Keyence LV-H41 laser displacement sensor with a detection accuracy of ±0.01mm and a detection range diameter of 50mm. The output is connected to the GPIO interface (Pin1) of the synchronization control board 4 via an M12×5-PUR shielded cable, outputting a 10μs wide high-level trigger pulse to trigger the system to start. The signal transmission delay is ≤1μs, and it is resistant to industrial electromagnetic interference (EMC level 3). The position trigger signal of sensor detection unit 1 is only transmitted to the synchronization control board 4. The optical flow calculation module 2 automatically starts calculation by receiving the image sequence from the industrial camera 3-2.

[0030] Specifically, the optical flow calculation module 2 is equipped with an FPGA hardware acceleration module.

[0031] More specifically, the optical flow calculation module 2 integrates a Xilinx Spartan-7 series FPGA as a hardware acceleration module. The FPGA includes: an 8×8 pixel cache array (composed of 32 D flip-flops, 256-bit width, physical address 0x0000-0x001F); a parallel comparison circuit (128 74HC86 XOR gates to realize hardware calculation of pixel grayscale difference between adjacent frames); and 16 displacement vector registers (74HC573 chip, latching motion data in the X / Y directions, outputting electrical signals corresponding to a speed range of 0-1000mm / s). The input end receives real-time image sequences (resolution 2048×2048, frame rate 90fps) from the industrial camera 3-2 via the CameraLink interface; the output end is connected to the input end of the trigger advance compensation module 5 and the synchronous control board 4 via RS-422 bus, respectively, and outputs real-time motion data such as part movement speed (e.g., 800mm / s) and direction (e.g., 0°). The bus transmission rate is 1Mbps, the signal transmission delay is ≤200ns, and the overall calculation to output delay is ≤100ns.

[0032] Specifically, the trigger advance compensation module 5 has a built-in CPLD timing logic chip. The CPLD of the trigger advance compensation module 5 has a built-in 16Kbit ROM circuit. The address line is connected to the speed input register (8 bits), and the data line outputs the advance signal (16 bits). The speed-advance correspondence is fixed by hardware programming, and there is no software modification interface. The production line encoder 8 is installed at the end of the drive wheel shaft of the conveyor belt. It transmits the production line speed signal to the trigger advance compensation module 5 through the SPI interface (resolution 0.1mm / s). The output of the trigger advance compensation module 5 sends the trigger advance command (such as advance of 500μs) to the synchronization control board 4 through the SPI bus.

[0033] Specifically, the output of the synchronization control board 4 is connected to the imaging system 3 via an optical fiber link.

[0034] Specifically, such as Figure 2As shown, the synchronization control board 4 includes a clock chip module 4-3, a timing calibration circuit module 4-4, an FPGA core control module 4-1, and an optical fiber transceiver module 4-2. The output of the clock chip module 4-3 is connected to the input of the timing calibration circuit module 4-4, and the output of the timing calibration circuit module 4-4 is connected to the input of the FPGA core control module 4-1. The input of the FPGA core control module 4-1 is connected to the output of the optical flow calculation module 2 via an RS-422 bus interface, and also to the output of the trigger advance compensation module 5 via an SPI interface. The output of the FPGA core control module 4-1 is connected to the input of the optical fiber transceiver module 4-2 and the multi-zone ring light source assembly 3-1. The input of the optical fiber transceiver module 4-2 is also connected to the output of the sensor detection unit 1, and the output of the optical fiber transceiver module 4-2 is connected to the input of the industrial camera 3-2.

[0035] More specifically, the FPGA core control module 4-1 uses a Xilinx Spartan-7 series FPGA, and based on this, it integrates an HFBR-1521 fiber optic transceiver module 4-2 (with built-in delay compensation circuit, using an adjustable capacitor (10-100pF) to calibrate the transmission delay and ensure trigger signal synchronization error ≤5ns), and a clock chip module 4-3 with a Si5341 clock chip. The Si5341 clock chip outputs a differential clock signal (jitter ±50ns), which is transmitted to the timing calibration circuit module 4-4 via a differential signal line. The timing calibration circuit module 4-4 uses a CD4046 phase-locked loop chip, and its input is connected to the Si5341 clock chip. The differential signal (Pin3-Pin4) and output (Pin6) are connected to the MRCC clock pin (Y12) of the FPGA via a 100Ω resistor. Phase jitter is calibrated to ≤4ns by the internal VCO circuit. The FPGA core control module receives motion data from the optical flow calculation module 2, which is transmitted via an RS-422 bus at a rate of 1Mbps. It also receives trigger advance commands from the trigger advance compensation module 5, which are received via the SPI interface. These commands, combined with the synchronization clock signal, generate PWM signals used to set the basic light intensity parameters for each light source zone. The source zone trigger signal is a pulse signal (pulse width 20-100μs) that has undergone strict timing calibration. Its rising edge is strictly synchronized with the camera exposure trigger signal (interval ≤50ns), used to trigger the corresponding light source zone to light up instantaneously at a precise moment. At this time, the brightness of the light source zone is controlled by the aforementioned directly transmitted PWM signal. The PWM signal zone interval is ≤50ns, and the duty cycle is 5%-95%, providing basic timing control (duty cycle, frequency) for the multi-zone ring light source component 3-1, adapting to the light intensity requirements of components made of different materials; at the same time, it also generates the camera exposure trigger signal and the zone trigger signal, which are transmitted to the fiber optic transceiver module. 4-2. After receiving the arrival trigger signal transmitted by the sensor detection unit 1 through the shielded cable, the fiber optic transceiver module 4-2 outputs one camera exposure trigger signal and four zone trigger signals to the industrial camera 3-2 and the multi-zone ring light source assembly 3-1, respectively. The rising edges of the zone trigger signals are synchronized, and the pulse width is 20-100μs. These light source zone trigger signals are pulse signals that have undergone strict timing calibration, and their rising edges are strictly synchronized with the camera exposure trigger signal (interval <50ns). They are used to trigger the corresponding light source zone to light up instantaneously at a precise moment. At this time, the brightness of the light source zone is controlled by the aforementioned directly transmitted PWM signal.

[0036] Specifically, such as Figure 3As shown, the multi-zone ring light source component 3-1 is an integrated ring structure, which is divided into 4 independent and controllable light source zones along the circumference, with an azimuth angle interval of 90° between adjacent zones; each zone is equipped with an LED light source array, and the line connecting the center of each zone and the imaging center point is the light source center line. The angle β between the light source center line and the normal of the imaging plane is 45°±2°, and the light intensity of each zone can be adjusted independently.

[0037] More specifically, the multi-zone ring light source assembly 3-1 adopts an integrated aluminum alloy shell with an inner diameter of 150mm and an outer diameter of 200mm. Each zone has 12 CREE Xlamp XP-E2 LED beads built in, which are independently powered by TPS92662 drivers with a driving current of 350mA and support dynamic adjustment of light intensity (duty cycle 5%-95%).

[0038] Specifically, the industrial camera 3-2 is connected to four light source zones via linkage control circuits. Each light source zone triggers a single exposure synchronously when it flashes, with a single exposure time ≤50μs. The trigger interval between adjacent light source zones is ≥20μs, ensuring that a single exposure corresponds to only one zone flashing.

[0039] More specifically, the industrial camera 3-2 uses a Basler ace acA2040-90um (global shutter CMOS sensor), equipped with a Computar M0814-MP2 lens, and supports time-division trigger exposure mode. Each light source zone flashes simultaneously, triggering a single exposure with a single exposure time of 20-100μs. It receives the camera exposure trigger signal and outputs a real-time image sequence to the optical flow calculation module 2.

[0040] Specifically, it also includes a support structure 6, which is n-shaped. The bottom ends of the support structure 6 are connected to the two ends of the pipeline width direction. The multi-zone ring light source assembly 3-1 is rigidly connected to the support structure 6. The flatness of the connection surface is ≤0.02mm, and the spatial position accuracy of the four light source zones is ≤0.1mm.

[0041] Specifically, the industrial camera 3-2 is fixed at the center of the support structure 6, and the optical axis of the industrial camera 3-2 coincides with the geometric center of the multi-zone ring light source assembly 3-1 and is perpendicular to the conveyor plane of the production line.

[0042] Specifically, the bottom of the support structure 6 is provided with a leveling mechanism 7, and the leveling mechanism 7 is provided with a shock-absorbing pad.

[0043] More specifically, the support structure 6 is made of 6061-T6 aluminum alloy profile. The bottom fixed end mates with the two ends of the assembly line in the width direction using a stepped positioning method, with a flatness ≤0.02mm. The bottom of the support structure 6 is equipped with an M12 anchor bolt leveling mechanism 7, rigidly connected to the assembly line workbench. An 8mm thick nitrile rubber shock-absorbing pad with a Shore hardness of 60±5 is added at the connection point. This ensures that the perpendicularity of the optical axis of the industrial camera 3-2 to the assembly line plane is ⊥≤0.03°, and the positional drift is ≤0.01mm within 30 days under a 5-50Hz vibration environment.

[0044] The working process of a photometric stereo imaging device based on sensor triggering and motion timing compensation is as follows:

[0045] When the part to be tested enters the detection area of ​​sensor detection unit 1, sensor detection unit 1 outputs a position trigger signal to synchronous control board 4, activating industrial camera 3-2. Industrial camera 3-2 acquires image sequences and transmits them to optical flow calculation module 2. Optical flow calculation module 2 calculates the part motion data through FPGA hardware logic and outputs it to trigger advance compensation module 5. Trigger advance compensation module 5 combines the speed signal and motion data of production line encoder 8 to calculate the time difference between the part reaching the imaging center and generates a trigger advance command. Synchronous control board 4 sends partition trigger signals to each partition of the light source through fiber optic links according to the command, and simultaneously controls industrial camera 3-2 to expose synchronously. Support structure 6 ensures the stability of imaging system 3 through leveling mechanism 7 and shock-absorbing pads. Finally, the image sequence is processed through a three-dimensional reconstruction algorithm to realize the detection of surface defects of the part.

[0046] Although embodiments of the present invention have been shown and described above, it is understood that the above embodiments are exemplary and should not be construed as limiting the present invention. Those skilled in the art can make modifications, alterations, substitutions and variations to the above embodiments within the scope of the present invention.

Claims

1. A photometric stereo imaging device based on sensor triggering and motion timing compensation, characterized in that, It includes a sensor detection unit (1), an optical flow calculation module (2), a synchronization control board (4), a trigger advance compensation module (5), and an imaging system (3); The sensor detection unit (1) and the imaging system (3) are located above the detection area. The output of the sensor detection unit (1) is connected to the input of the synchronization control board (4). The output of the synchronization control board (4) is connected to the input of the imaging system (3). The imaging system (3) includes an industrial camera (3-2) and a multi-zone ring light source assembly (3-1). The output of the industrial camera (3-2) is connected to the input of the optical flow calculation module (2). The output of the optical flow calculation module (2) is connected to both the trigger advance compensation module (5) and the input of the synchronization control board (4). The input of the trigger advance compensation module (5) is also connected to the output of the production line encoder (8). The output of the trigger advance compensation module (5) is connected to the input of the synchronization control board (4).

2. The photometric stereo imaging device based on sensor triggering and motion timing compensation as described in claim 1, characterized in that: The optical flow calculation module (2) is equipped with an FPGA hardware acceleration module.

3. The photometric stereo imaging device based on sensor triggering and motion timing compensation as described in claim 1, characterized in that: The output of the synchronization control board (4) is connected to the imaging system (3) via an optical fiber link.

4. The photometric stereo imaging device based on sensor triggering and motion timing compensation as described in claim 1, characterized in that: The synchronization control board (4) includes a clock chip module (4-3), a timing calibration circuit module (4-4), an FPGA core control module, and an optical fiber transceiver module (4-2). The output of the clock chip module (4-3) is connected to the input of the timing calibration circuit module (4-4). The output of the timing calibration circuit module (4-4) is connected to the input of the FPGA core control module. The input of the FPGA core control module is also connected to the output of the optical flow calculation module (2) and the trigger advance compensation module (5). The output of the FPGA core control module is connected to the input of the optical fiber transceiver module (4-2) and the multi-zone ring light source assembly (3-1). The input of the optical fiber transceiver module (4-2) is also connected to the output of the sensor detection unit (1). The output of the optical fiber transceiver module (4-2) is connected to the input of the industrial camera (3-2).

5. The photometric stereo imaging device based on sensor triggering and motion timing compensation as described in claim 1, characterized in that: The multi-zone ring light source component (3-1) is an integrated ring structure, which is divided into 4 independent and controllable light source zones along the circumference, with an azimuth angle interval of 90° between adjacent zones; each zone is equipped with an LED light source array, and the line connecting the center of each zone and the imaging center point is the light source center line, with the angle between the light source center line and the normal of the imaging plane being 45°±2°, and the light intensity of each zone can be adjusted independently.

6. The photometric stereo imaging device based on sensor triggering and motion timing compensation as described in claim 5, characterized in that: The industrial camera (3-2) is connected to the four light source zones with linkage control circuits. When each light source zone flashes, a single exposure is triggered synchronously, and the single exposure time is ≤50μs.

7. The photometric stereo imaging device based on sensor triggering and motion timing compensation as described in claim 5, characterized in that: It also includes a support structure (6), which is n-shaped. The bottom ends of the support structure (6) are connected to the two ends of the width direction of the production line. The multi-zone ring light source assembly (3-1) is rigidly connected to the support structure (6). The flatness of the connection surface is ≤0.02mm, and the spatial position accuracy of the four light source zones is ≤0.1mm.

8. The photometric stereo imaging device based on sensor triggering and motion timing compensation as described in claim 7, characterized in that: The industrial camera (3-2) is fixed at the center of the support structure (6). The optical axis of the industrial camera (3-2) coincides with the geometric center of the multi-zone ring light source assembly (3-1) and is perpendicular to the conveyor plane.

9. A photometric stereo imaging device based on sensor triggering and motion timing compensation as described in claim 7, characterized in that: The bottom of the support structure (6) is provided with a leveling mechanism (7), and the leveling mechanism (7) is provided with a shock-absorbing pad.

Images