Cooperative control method and cooperative control system for projection exposure
By using programmable gate arrays (FPGAs) for hardware parallel processing and precise timing, the problem of synchronous timing control between the optomechanical system and the sensor was solved, achieving strict alignment and precise parameter matching between optomechanical projection and sensor exposure, thus improving the quality and reliability of image acquisition.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-12-04
- Publication Date
- 2026-03-10
AI Technical Summary
In existing technologies, the synchronization timing control between optomechanics and sensors suffers from low synchronization accuracy, insufficient coordination, and poor reliability, resulting in blurred image acquisition and large measurement errors.
The use of a programmable gate array (FPGA) for hardware parallel processing and precise timing ensures that the optomechanical projection and sensor exposure are strictly aligned in time. A mode and exposure time binding mechanism is used to achieve precise matching, thereby improving synchronization accuracy and reliability.
It improves the quality and reliability of image acquisition, ensures that exposure and acquisition are performed only after the pattern projection is stable, reduces image blur, improves the dynamic range and signal-to-noise ratio of the image, and enhances the adaptability and reliability of the system.
Smart Images
Figure CN121645003A_ABST
Abstract
Description
Technical Field
[0001] This application relates to the field of image processing technology, and more specifically, to a collaborative control method and a collaborative control system for projection exposure. Background Technology
[0002] In structured light-based 3D measurement or high dynamic range imaging, to overcome overexposure or underexposure caused by large differences in the reflectivity of the light pattern surfaces, it is usually necessary to project a series of specific light patterns (such as Gray code or sinusoidal fringes) and simultaneously acquire and fuse multiple sets of images with different exposure times. This process places extremely high demands on the synchronous timing control between the optomechanical system and the sensor.
[0003] In related technologies, a serial instruction mode dominated by host computer (such as industrial control computer) software is used to perform synchronous timing control between the optomechanical system and the sensor. The process usually involves the software sending instructions to control the optomechanical system to switch patterns, waiting for the switching to complete, sending instructions, setting camera exposure parameters and triggering exposure, waiting for image acquisition to complete, and then looping to the next pattern.
[0004] However, using the above method may cause exposure to begin before the pattern has stabilized, resulting in blurry images and poor reliability. Summary of the Invention
[0005] In view of this, embodiments of this application provide a collaborative control method and a collaborative control system for projection exposure, in order to solve the problem that existing methods start exposure before the pattern has stabilized, resulting in blurry images and poor reliability.
[0006] In a first aspect, embodiments of this application provide a collaborative control method for projection exposure, applied to a programmable gate array in a collaborative control system. The collaborative control system further includes: an optomechanical system and an image sensor communicatively connected to the programmable gate array. The method includes: Obtain the collaborative parameter configuration table, which includes multiple sets of collaborative parameters. One set of collaborative parameters includes: an optical-mechanical mode identifier, the sensor exposure time corresponding to the optical-mechanical mode identifier, and a delay time. The optical-mechanical mode identifier is used to indicate the pattern to be projected by the optical engine, and the delay time is the stable display time of the pattern to be projected after projection. Determine the current group's collaboration parameters from the collaboration parameter configuration table; Based on the current optical engine mode identifier in the current group coordination parameters, control the optical engine to project the target pattern indicated by the current optical engine mode identifier; After the current delay time in the current group coordination parameters, the image sensor is controlled to expose within the current sensor exposure time according to the current sensor exposure time in the current group coordination parameters, and the target pattern is acquired to obtain the acquired image corresponding to the current group coordination parameters.
[0007] In an optional implementation, controlling the optical engine to project the target pattern indicated by the current optical engine mode identifier based on the current optical engine mode identifier in the current group coordination parameters includes: Send a pattern switching trigger signal to the optical engine to control the optical engine to project the target pattern; After sending the pattern switching trigger signal to the optomechanism, the method further includes: The system receives a synchronization trigger signal sent by the optical engine, the synchronization trigger signal being used to indicate that the optical engine has received the current optical engine mode identifier.
[0008] In an optional implementation, the step of controlling the image sensor to expose within the current sensor exposure time according to the current sensor exposure time in the current group coordination parameters, and acquiring the target pattern to obtain the acquired image corresponding to the current group coordination parameters, includes: Load the current sensor exposure time into a preset exposure timer and start the exposure timer; Send an exposure enable signal to the image sensor to control the image sensor to perform exposure; After the exposure timer expires, an exposure stop signal is sent to the image sensor to control the image sensor to stop exposure and acquire the target pattern to obtain the acquired image corresponding to the current group coordination parameters.
[0009] In an optional implementation, the step of controlling the image sensor to expose within the current sensor exposure time according to the current sensor exposure time in the current group coordination parameters, and acquiring the target pattern to obtain the acquired image corresponding to the current group coordination parameters, includes: An exposure trigger signal is sent to the image sensor, the exposure trigger signal including the current sensor exposure time, so as to control the image sensor to expose within the current sensor exposure time and acquire the target pattern to obtain the acquired image corresponding to the current group of cooperative parameters.
[0010] In an optional implementation, after controlling the image sensor to expose within the current sensor exposure time according to the current group coordination parameters, and acquiring the target pattern to obtain the acquired image corresponding to the current group coordination parameters, the method further includes: The next set of collaborative parameters is determined from the collaborative parameter configuration table, and the acquired image corresponding to the next set of collaborative parameters is obtained, until the acquired images corresponding to the multiple sets of collaborative parameters are obtained; Image fusion is performed on the acquired images corresponding to the multiple sets of collaborative parameters to obtain the target fused image.
[0011] In an optional implementation, controlling the optical engine to project the target pattern indicated by the current optical engine mode identifier based on the current optical engine mode identifier in the current group coordination parameters includes: Update the preset configuration register based on the current group collaboration parameters; The current group coordination parameters are read from the configuration register, and the current optical engine mode identifier in the current group coordination parameters is used to control the optical engine to project the target pattern.
[0012] In an optional implementation, before controlling the image sensor to expose within the current sensor exposure time according to the current sensor exposure time in the current group coordination parameters after the current delay time in the current group coordination parameters, and before acquiring the target pattern to obtain the acquired image corresponding to the current group coordination parameters, the method further includes: The current delay time is loaded into a preset delay compensation timer, and the delay compensation timer is started; If the delay compensation timer times out, then the current delay time is determined to have arrived.
[0013] Secondly, embodiments of this application also provide a collaborative control system, including: a programmable gate array, an optomechanical system, and an image sensor, wherein the optomechanical system and the image sensor are respectively communicatively connected to the programmable gate array; The programmable gate array is used to perform the method described in any of the first aspects.
[0014] In an optional implementation, the collaborative control system further includes a host computer and a memory that are communicatively connected to the programmable gate array.
[0015] In an optional implementation, the collaborative control system further includes a central processing unit and a graphics processor connected in communication, the central processing unit also being connected in communication with the programmable gate array.
[0016] This application provides a method and system for coordinated control of projection exposure. The method includes: acquiring a coordinated parameter configuration table, which includes multiple sets of coordinated parameters; determining the current set of coordinated parameters from the table; controlling the optomechanical system to project the target pattern indicated by the current optomechanical mode identifier based on the current optomechanical mode identifier in the current set of coordinated parameters; and controlling the image sensor to expose and acquire images within the current sensor exposure time based on the current sensor exposure time in the current set of coordinated parameters after the current delay time in the current set of coordinated parameters, thereby obtaining the acquired image corresponding to the current set of coordinated parameters. This ensures that exposure and acquisition only occur after the pattern projection is stable, guaranteeing strict timing alignment between optomechanical projection and sensor exposure. Furthermore, a mode and exposure time binding mechanism achieves precise parameter matching, improving image quality. Attached Figure Description
[0017] To more clearly illustrate the technical solutions of the embodiments of this application, the accompanying drawings used in the embodiments will be briefly introduced below. It should be understood that the following drawings only show some embodiments of this application and should not be regarded as a limitation of the scope. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort.
[0018] Figure 1 A schematic diagram of the architecture of the collaborative control system provided in the embodiments of this application; Figure 2 A flowchart illustrating the collaborative control method for projection exposure provided in this application embodiment. Figure 1 ; Figure 3 A flowchart illustrating the collaborative control method for projection exposure provided in this application embodiment. Figure 2 ; Figure 4 A flowchart illustrating the collaborative control method for projection exposure provided in this application embodiment. Figure 3 ; Figure 5 A flowchart illustrating the collaborative control method for projection exposure provided in this application embodiment. Figure 4 ; Figure 6 A flowchart illustrating the collaborative control method for projection exposure provided in this application embodiment. Figure 5 ; Figure 7 A specific delay compensation flowchart is provided for an embodiment of this application; Figure 8 This application provides a specific flowchart for starting the exposure process. Figure 9 This is a timing diagram for signal synchronization triggering provided in an embodiment of this application; Figure 10 A schematic diagram of the structure of the collaborative control device for projection exposure provided in the embodiments of this application. Detailed Implementation
[0019] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. The components of the embodiments of this application described and shown in the accompanying drawings can generally be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of this application provided in the accompanying drawings is not intended to limit the scope of the claimed application, but merely represents selected embodiments of this application. All other embodiments obtained by those skilled in the art based on the embodiments of this application without inventive effort are within the scope of protection of this application.
[0020] The use of a serial instruction mode dominated by host computer (such as an industrial control computer) software for synchronous timing control between optomechanics and sensors has the following drawbacks: Low synchronization accuracy: The execution of software instructions, the task scheduling of the operating system, and the latency of communication buses (such as USB and Ethernet) are all uncertain and random, making it impossible to establish and maintain a precise and repeatable fixed phase relationship between the moment when the pattern is stably displayed and the moment of sensor exposure. This timing jitter (usually on the order of milliseconds) can lead to severe image misalignment or incomplete pattern acquisition in high-speed applications with microsecond-level exposure, introducing huge measurement errors.
[0021] Insufficient coordination: It is difficult to achieve rapid and accurate matching between pattern and exposure time. For example, for fine patterns with high spatial frequencies, a longer exposure time is needed to ensure the signal-to-noise ratio, while for globally bright patterns, an extremely short exposure is required to prevent sensor saturation. Software looping methods struggle to complete this adaptive switching and synchronization of parameters in a very short time.
[0022] Poor reliability: Due to the uncertainty of timing, in some extreme cases, the sensor may start to expose before the optomechanical pattern has fully stabilized, resulting in blurred images and thus the failure of the entire measurement sequence.
[0023] Based on this, this application leverages the hardware parallel processing and precise timing capabilities of a Field Programmable Gate Array (FPGA) to improve synchronization accuracy. Exposure and acquisition only occur after pattern projection has stabilized, ensuring strict timing alignment between optomechanical projection and sensor exposure, thus enhancing reliability. Furthermore, a mode and exposure time binding mechanism achieves precise parameter matching, improving image quality and enhancing coordination. This constructs a collaborative control mechanism that ensures strict timing alignment and precise parameter matching between optomechanical projection and sensor exposure, enabling the acquisition of high-quality, highly reliable multi-exposure image sequences.
[0024] Figure 1 This is a schematic diagram of the architecture of the collaborative control system provided in the embodiments of this application, as shown below. Figure 1 As shown, it includes: a programmable gate array 10, an optomechanical system 20, and an image sensor 30. The optomechanical system 20 and the image sensor 30 are respectively communicatively connected to the programmable gate array 10.
[0025] In an optional implementation, the collaborative control system further includes a host computer 40 and a memory 50 that are communicatively connected to the programmable gate array 10.
[0026] The host computer 40 is used to send the configured coordination parameter configuration table to the programmable gate array 10 through the communication interface during the initialization configuration phase. The coordination parameter configuration table includes multiple sets of coordination parameters. One set of coordination parameters includes: an optomechanical mode identifier, the sensor exposure time corresponding to the optomechanical mode identifier, and a delay time. The optomechanical mode identifier is used to indicate the pattern to be projected by the optomechanical system. The delay time is the stable display time of the pattern to be projected after projection. The exposure time refers to the sensor exposure time corresponding to the pattern to be projected indicated by the optomechanical mode identifier. Different patterns to be projected correspond to different exposure times. For example, for fine patterns with high spatial frequency, a longer exposure time is required to ensure the signal-to-noise ratio. For globally high-brightness patterns, an extremely short exposure time is required to prevent sensor saturation. The specific settings can be set according to the actual situation. This embodiment does not make any special limitations on this.
[0027] The programmable gate array 10, also known as the FPGA, is used to determine a set of cooperative parameters from the cooperative parameter configuration table. The optomechanical system 20 is used to project the pattern corresponding to the set of cooperative parameters. The image sensor is used to expose the pattern projected by the optomechanical system 20 within the sensor exposure time corresponding to the set of cooperative parameters and to acquire the image corresponding to the set of cooperative parameters.
[0028] It should be noted that exposure is the process by which the image sensor 30 senses light, thereby accumulating charge on each pixel. It determines the basic image quality such as brightness and dynamic range. Image acquisition refers to the process of reading out the charge signal accumulated on each pixel and converting it into digital image data.
[0029] The memory 50 can be, for example, a Double Data Rate Synchronous Dynamic Random Access Memory (DDR SDRAM) for storing acquired images corresponding to multiple sets of coordinated parameters.
[0030] In an optional implementation, the collaborative control system further includes a central processing unit (CPU) 60 and a graphics processing unit (GPU) 70, which are communicatively connected. The CPU 60 is also communicatively connected to the programmable gate array 10.
[0031] It should be noted that the programmable gate array 10 is the control center of the entire system, integrating a collaborative control unit, a multi-channel acquisition and memory management unit, an image fusion unit, and a back-end interface unit. The collaborative control unit communicates with the host computer 40 through a communication interface to receive the collaborative parameter configuration table sent by the host computer 40 and trigger synchronously to control the optical engine 20 to execute the projection pattern and the image sensor 30 to perform exposure and acquisition. The multi-channel acquisition and memory management unit is used to store the acquired images corresponding to each set of collaborative parameters in the memory 50. The image fusion unit is used to read the acquired images in segments from the memory 50 and perform image fusion to obtain the target fused image, which is then cached in the memory 50 and sent to the central processing unit 50 through the back-end interface unit, so that the central processing unit 60 controls the graphics processor 70 to render and display the target fused image. In addition, the collaborative control module can also send the number of patterns to the multi-channel acquisition and memory management module.
[0032] The communication interface can be, for example, a Universal Asynchronous Receiver / Transmitter (UART), an Inter-Integrated Circuit (IIC), or a Serial Peripheral Interface (SPI).
[0033] The core of this solution lies in the original design of the collaborative control unit, which is not a simple pulse generator, but a complex state machine system that integrates parameter set binding, programmable delay compensation and hard synchronization timing generation.
[0034] Figure 2 A flowchart illustrating the collaborative control method for projection exposure provided in this application embodiment. Figure 1 In this embodiment, the execution entity can be a programmable gate array in a collaborative control system. The collaborative control system also includes an optomechanical system and an image sensor that are communicatively connected to the programmable gate array.
[0035] like Figure 2 As shown, the method may include: S101. Obtain the collaborative parameter configuration table.
[0036] The collaborative parameter configuration table includes multiple sets of collaborative parameters. One set of collaborative parameters includes: an optical-mechanical mode identifier, the sensor exposure time corresponding to the optical-mechanical mode identifier, and a delay time. The optical-mechanical mode identifier is used to indicate the pattern to be projected by the optical engine. For example, the optical-mechanical mode identifier can be the pattern number or address of the pattern to be projected. The delay time is the stable display time of the pattern to be projected after projection, that is, the time required for the compensation optical engine to achieve a fully stable display from receiving the pattern switching trigger signal.
[0037] The sensor exposure time corresponding to the optomechanical mode identifier is the sensor exposure time corresponding to the pattern indicated by the optomechanical mode identifier. The sensor exposure time corresponding to different patterns can be different. It can be configured according to the actual situation to avoid overexposure or underexposure of the image sensor. When the exposure is performed within the configured sensor exposure time, the image sensor will not be overexposed or underexposed. The sensor exposure time is in the number of clock cycles of the programmable logic gate array.
[0038] In some embodiments, the delay time (T_delay) can be set by optical-mechanical intrinsic parameters, precisely measured, or set according to the optical-mechanical datasheet to ensure that T_delay ≥ T_stable (the preset optical-mechanical pattern stabilization time).
[0039] Table 1 is the collaborative parameter configuration table. As shown in Table 1, the collaborative parameter configuration table contains n entries. Each entry represents a set of independent collaborative parameters. Entry 1 contains the optical-mechanical identifier (i.e., the optical-mechanical mode identifier) 1, the exposure parameter (exposure time) 1, the delay parameter (delay time) 1, and the reserved parameter 1. Entry 2 contains the optical-mechanical identifier (i.e., the optical-mechanical mode identifier) 2, the exposure parameter (exposure time) 2, the delay parameter (delay time) 2, and the reserved parameter 2. Similarly, entry n contains the optical-mechanical identifier (i.e., the optical-mechanical mode identifier) n, the exposure parameter (exposure time) n, the delay parameter (delay time) n, and the reserved parameter n.
[0040] Table 1
[0041] In some embodiments, the host computer receives and sends a cooperative parameter configuration table to the programmable gate array through a communication interface. During device operation, the global or local parameters in the cooperative parameter configuration table can also be dynamically updated to ensure parameter flexibility.
[0042] In this embodiment, by binding each set of collaborative parameters, the optimal exposure time and precise synchronization delay are preset for each pattern, realizing a refined collaborative strategy of "one case at a time".
[0043] S102. Determine the current group's collaboration parameters from the collaboration parameter configuration table.
[0044] The current group of collaborative parameters can be the group of collaborative parameters to be processed among the multiple groups of collaborative parameters in the collaborative parameter configuration table, such as the first group of collaborative parameters, which is the collaborative parameter corresponding to item 1.
[0045] S103. Based on the current optical engine mode identifier in the current group coordination parameters, control the optical engine to project the target pattern indicated by the current optical engine mode identifier.
[0046] S104. After the current delay time in the current group coordination parameters, according to the current sensor exposure time in the current group coordination parameters, control the image sensor to expose within the current sensor exposure time and acquire the target pattern to obtain the acquired image corresponding to the current group coordination parameters.
[0047] The current group coordination parameters include: the current optical-mechanical mode identifier, the current delay time, and the current sensor exposure time.
[0048] Based on the current optical engine mode identifier, the optical engine is controlled to project the target pattern indicated by the current optical engine mode identifier. After the current delay time, it is indicated that the target pattern to be projected is completely stable and displayed. Then, based on the current sensor exposure time, the image sensor is controlled to expose within the current sensor exposure time. In this case, the image sensor will not be overexposed or underexposed, and the target pattern projected by the optical engine is acquired to obtain the image corresponding to the current group of cooperative parameters.
[0049] In an optional implementation, before step S104, after the current delay time in the current group coordination parameters and before controlling the image sensor to expose within the current sensor exposure time according to the current sensor exposure time in the current group coordination parameters, and before acquiring the target pattern to obtain the acquired image corresponding to the current group coordination parameters, the method may further include: Load the current delay time into the preset delay compensation timer and start the delay compensation timer; If the delay compensation timer times out, the current delay time is determined to have elapsed.
[0050] After controlling the optical engine to project the target pattern indicated by the current optical engine mode identifier, the current delay time is loaded into the preset delay compensation timer and the delay compensation timer is started. If the delay compensation timer times out, the current delay time is determined to have arrived, and after the current delay time, the image sensor is controlled to expose and acquire data.
[0051] In this embodiment, by starting a delay compensation timer, the target pattern projected by the optical engine is stabilized during the delay compensation timer counting period. If the delay compensation timer times out, it means that the target pattern has been completely stabilized and the exposure delay compensation period of the sensor ends.
[0052] In this embodiment, by introducing a programmable delay compensation mechanism, the image sensor is ensured to expose only after the pattern projected by the optical engine has completely stabilized. This fundamentally avoids the image quality degradation caused by blurring during pattern switching, ensuring that clear and effective pattern information is obtained in each acquisition, thus guaranteeing the quality of the acquired images. Furthermore, the mode and exposure time binding mechanism enables the system to dynamically match the optimal exposure time for the characteristics (brightness, frequency) of different patterns, achieving adaptive optimization of the acquisition process. This significantly improves the dynamic range and signal-to-noise ratio of the overall imaging, enhancing the system's adaptability and intelligence.
[0053] Meanwhile, the entire timing control process is executed automatically and continuously by hardware logic. The process is completely deterministic and repeatable, eliminating the risk of random failures caused by software scheduling or external interference, greatly improving the reliability of industrial applications, and enhancing the reliability and stability of the system.
[0054] Figure 3 A flowchart illustrating the collaborative control method for projection exposure provided in this application embodiment. Figure 2 ,like Figure 3 As shown, in an optional implementation, step S103, which controls the optical engine to project the target pattern indicated by the current optical engine mode identifier based on the current optical engine mode identifier in the current group coordination parameters, may include: S201. Send a pattern switching trigger signal to the optical engine to control the optical engine to project the target pattern.
[0055] The programmable gate array sends a pattern switching trigger signal to the optomechanical system. The pattern switching trigger signal carries the target pattern. The optomechanical system receives the pattern switching trigger signal and projects the target pattern.
[0056] After step S201 above, which involves sending a pattern switching trigger signal to the optomechanism, the method may further include: S202, Receive the synchronization trigger signal sent by the optical engine.
[0057] After sending the pattern switching trigger signal, the optomechanical system can send back a synchronization trigger signal to indicate that it has received the current optomechanical mode identifier, thereby forming a handshake operation. The programmable gate array detects the synchronization trigger signal to determine that the optomechanical system configuration is successful, thus avoiding subsequent empty exposure of the image sensor. In other words, the programmable gate array only executes the operation of controlling the image sensor to perform exposure and acquisition in step S104 after receiving the synchronization trigger signal. If the synchronization trigger signal is not received, the operation of controlling the image sensor to perform exposure and acquisition in step S104 will not be executed.
[0058] Figure 4 A flowchart illustrating the collaborative control method for projection exposure provided in this application embodiment. Figure 3 ,like Figure 4 As shown, in an optional embodiment, step S104, which controls the image sensor to expose within the current sensor exposure time according to the current sensor exposure time in the current group coordination parameters, and acquires the target pattern to obtain the acquired image corresponding to the current group coordination parameters, may include: S301. Load the current sensor exposure time into the preset exposure timer and start the exposure timer.
[0059] S302. Send an exposure enable signal to the image sensor to control the image sensor to perform exposure.
[0060] When exposure is initiated, the current sensor exposure time is loaded into the preset exposure timer and the exposure timer is started. Then, an exposure enable signal is sent to the image sensor. The exposure enable signal is used to control the image sensor to perform exposure. The image sensor starts exposure based on the exposure enable signal.
[0061] The exposure enable signal can be a high-level active global exposure signal.
[0062] S303. After the exposure timer expires, an exposure stop signal is sent to the image sensor to control the image sensor to stop exposure and acquire the target pattern to obtain the acquired image corresponding to the current group coordination parameters.
[0063] After the exposure timer expires, an exposure stop signal is sent to the image sensor. This signal controls the image sensor to stop exposure. Based on the exposure stop signal, the image sensor stops exposure and acquires the target pattern to obtain the acquired image corresponding to the current group's coordination parameters. Throughout the exposure period, the target pattern projected by the optomechanical system remains unchanged.
[0064] In other words, when the exposure timer count reaches the set current sensor exposure time, the exposure enable signal is pulled down to obtain the exposure stop signal and sent to the image sensor, thus ending the current acquisition.
[0065] In an optional implementation, based on the current sensor exposure time in the current group coordination parameters, the image sensor is controlled to expose within the current sensor exposure time, and the target pattern is acquired to obtain the acquired image corresponding to the current group coordination parameters, including: An exposure trigger signal is sent to the image sensor. The exposure trigger signal includes the current sensor exposure time, so as to control the image sensor to expose within the current sensor exposure time and acquire the target pattern to obtain the acquired image corresponding to the current group of cooperative parameters.
[0066] The exposure trigger signal is used to control the image sensor to expose within the current sensor exposure time. After receiving the exposure trigger signal sent by the programmable gate array, the image sensor exposes within the current sensor exposure time and acquires the target pattern to obtain the acquired image corresponding to the current group of cooperative parameters.
[0067] In other words, the image sensor keeps track of the exposure time, and the exposure stops when the current sensor exposure time is reached.
[0068] Figure 5 A flowchart illustrating the collaborative control method for projection exposure provided in this application embodiment. Figure 4 ,like Figure 5 As shown, in an optional embodiment, after step S104, which involves controlling the image sensor to expose within the current sensor exposure time based on the current sensor exposure time in the current group coordination parameters, and acquiring the target pattern to obtain the acquired image corresponding to the current group coordination parameters, the method may further include: S401. Determine the next set of collaborative parameters from the collaborative parameter configuration table, and obtain the acquisition image corresponding to the next set of collaborative parameters, until multiple sets of collaborative parameters are obtained.
[0069] The next set of collaborative parameters is determined from the collaborative parameter configuration table in a preset order, following the next set of collaborative parameters in the current set, for example, item 2. The next set of collaborative parameters includes: next optomechanical mode identifier, next sensor exposure time, and next delay time.
[0070] Based on the next optomechanical mode identifier in the next set of coordination parameters, the optomechanical system is controlled to project the next pattern indicated by the next optomechanical mode identifier. After the next delay time, based on the next sensor exposure time, the image sensor is controlled to expose within the next sensor exposure time and acquire the next pattern to obtain the acquired image corresponding to the next set of coordination parameters.
[0071] In other words, after obtaining the acquisition image corresponding to a set of collaborative parameters, the acquisition image corresponding to the next set of collaborative parameters can be obtained in a loop until multiple sets of collaborative parameters are obtained. That is, all sets of collaborative parameters in the collaborative parameter configuration table are traversed.
[0072] S402. Perform image fusion based on the acquired images corresponding to multiple sets of collaborative parameters to obtain the target fused image.
[0073] Image fusion is performed on the acquired images corresponding to multiple sets of collaborative parameters to obtain the target fused image. This method can fuse images from multiple exposure times to solve the problems of overexposure or underexposure.
[0074] It should be noted that target fusion images can be applied to scenarios such as 3D reconstruction and defect recognition. Taking 3D reconstruction as an example, the target fusion image is used as the original projection image, and an optical engine projects multiple patterns corresponding to different sets of cooperative parameters onto the object surface. When these patterns are irradiated onto the irregular object surface, they will be distorted and deformed, resulting in deformed patterns. An image sensor is used to expose the object within the exposure time corresponding to the different sets of cooperative parameters, and the deformed patterns are collected separately to obtain multiple deformed images. These multiple deformed images are then fused into a deformed image. Finally, based on the original projection image and the deformed fusion image, the 3D model of the object is reconstructed.
[0075] Figure 6 A flowchart illustrating the collaborative control method for projection exposure provided in this application embodiment. Figure 5 ,like Figure 6 As shown, in an optional implementation, step S103, which controls the optical engine to project the target pattern indicated by the current optical engine mode identifier based on the current optical engine mode identifier in the current group coordination parameters, may include: S501. Update the preset configuration register according to the current group coordination parameters.
[0076] S502: Read the current group coordination parameters from the configuration register, and control the optical engine to project the target pattern according to the current optical engine mode identifier in the current group coordination parameters.
[0077] Because the coordination parameter configuration table is dynamically updated, to avoid execution malfunctions caused by the acquisition of current group coordination parameters followed by an update, the acquired current group coordination parameters are updated in the configuration register. Then, the current group coordination parameters are read from the configuration register, and the optical engine is controlled to project the target pattern based on the current optical engine mode identifier within the current group coordination parameters. In other words, when the current group system parameters are obtained, they are not immediately given to the optical engine; instead, they are first placed in the configuration register and then retrieved. This avoids execution malfunctions caused by changes in the current group coordination parameters.
[0078] In some embodiments, the cooperative control unit of the programmable gate array can use a state machine and a timer to execute the above embodiments. By executing a cyclic process through the state machine, the timing uncertainty of the software and operating system is avoided, and the synchronization jitter between the optomechanical system and the image sensor is controlled within a single clock cycle of the programmable gate array, providing a fundamental guarantee for high-precision measurement.
[0079] When the state machine enters the Nth loop state (corresponding to the Nth entry in the parameter table), the following operations are performed: Obtain the cooperative parameter configuration table and update the parameters in the configuration register; send the optomechanical mode identifier to the optomechanical initiation pattern switching process; load the delay time into the delay timer to prepare for subsequent delay compensation timing; receive the synchronization trigger signal; during the delay waiting period, start the delay compensation timer and wait for the pattern to stabilize; after the delay compensation timer expires, the sensor's exposure delay compensation period ends, and then exposure and acquisition are performed. After this acquisition is completed, the state machine automatically jumps to the next loop state to determine the number of loops. If the loop process has not yet ended, the next entry is executed. This cycle continues until the entire entry has been traversed.
[0080] In summary, the rising edge of the exposure trigger signal (start of exposure) and the effective edge of the pattern switching trigger signal, as well as during the exposure period, are precisely timed by the hardware clock of the programmable gate array (PGA). This ensures that the pattern projected by the optomechanical system is completely stable and predictable within each exposure cycle of the image sensor. This hardware-level synchronization reduces the synchronization error between the optomechanical system and the sensor from milliseconds in software-based methods to nanoseconds or microseconds in the PGA clock cycle, fundamentally guaranteeing strict spatial alignment of multiple images.
[0081] Figure 7 A specific delay compensation flowchart is provided for an embodiment of this application, such as... Figure 7 As shown, it includes the following steps: Step 1: Obtain the collaborative parameter configuration table.
[0082] Step 2: Start the optical engine.
[0083] The optical engine is started by sending a pattern switching trigger signal to it.
[0084] Step 3: Determine whether a synchronization trigger signal has been detected.
[0085] If yes, proceed to step 4; otherwise, return to step 3.
[0086] Step 4: Delay compensation.
[0087] Step 5: Determine if the delay time has been reached.
[0088] If yes, the delay compensation ends; otherwise, return to step 5.
[0089] Figure 8 A specific flowchart for starting the exposure process is provided in this application embodiment, such as... Figure 8 As shown, it includes the following steps: Step 1: Load the exposure time.
[0090] Step 2: Start the exposure.
[0091] The image sensor is activated by sending an exposure enable signal.
[0092] Step 3: Determine if the exposure time has been reached.
[0093] If yes, the current exposure ends, the pattern is acquired to obtain the acquired image, and step 4 is executed; otherwise, return to step 2.
[0094] Step 4: Determine if all entries have ended.
[0095] If yes, the loop ends; otherwise, proceed to step 5.
[0096] Step 5: Start the next timing control process.
[0097] Figure 9 The signal synchronization triggering timing diagram provided in the embodiments of this application is as follows: Figure 9 As shown, a high level for the optomechanical start signal indicates that the optomechanical system is starting up; a high level for the synchronization trigger signal indicates that the synchronization trigger signal returned by the optomechanical system has been received; a high level for the optomechanical stabilization signal indicates that the pattern projected by the optomechanical system is stably displayed; a high level for the sensor exposure signal indicates that the image sensor is performing exposure; and a high level for image acquisition indicates that an image has been acquired.
[0098] Wherein, Exposure 1 represents the first exposure of the image sensor, Exposure 2 represents the second exposure of the image sensor, Exposure n represents the nth exposure of the image sensor, Image 1 represents the acquired image corresponding to the first set of collaborative parameters, Image 2 represents the acquired image corresponding to the second set of collaborative parameters, Image n represents the acquired image corresponding to the nth set of collaborative parameters, Delay Compensation 1 represents the delay compensation corresponding to the first set of collaborative parameters, Delay Compensation 2 represents the delay compensation corresponding to the second set of collaborative parameters, and Delay Compensation n represents the delay compensation corresponding to the nth set of collaborative parameters.
[0099] Figure 10 This is a schematic diagram of the structure of the collaborative control device for projection exposure provided in the embodiments of this application. The device can be integrated into a programmable gate array, such as a collaborative control unit of a programmable gate array.
[0100] like Figure 10 As shown, the device may include: The acquisition module 601 is used to acquire the collaborative parameter configuration table, which includes multiple sets of collaborative parameters. One set of collaborative parameters includes: an optical-mechanical mode identifier, the sensor exposure time corresponding to the optical-mechanical mode identifier, and a delay time. The optical-mechanical mode identifier is used to indicate the pattern to be projected by the optical engine, and the delay time is the stable display time of the pattern to be projected after projection. Processing module 602 is used to determine the current group's collaboration parameters from the collaboration parameter configuration table; The processing module 602 is also used to control the optical engine to project the target pattern indicated by the current optical engine mode identifier according to the current optical engine mode identifier in the current group coordination parameters; The processing module 602 is further configured to, after the current delay time in the current group coordination parameters, control the image sensor to expose within the current sensor exposure time according to the current sensor exposure time in the current group coordination parameters, and acquire the target pattern to obtain the acquired image corresponding to the current group coordination parameters.
[0101] In an optional implementation, the processing module 602 is specifically used for: Send a pattern switching trigger signal to the optical engine to control the optical engine to project the target pattern; The device also includes: The receiving module 603 is used to receive the synchronization trigger signal sent by the optical engine. The synchronization trigger signal is used to indicate that the optical engine has received the current optical engine mode identifier.
[0102] In an optional implementation, the processing module 602 is specifically used for: Load the current sensor exposure time into the preset exposure timer and start the exposure timer; Send an exposure enable signal to the image sensor to control the image sensor to perform exposure; After the exposure timer expires, an exposure stop signal is sent to the image sensor to control the image sensor to stop exposure and acquire the target pattern to obtain the acquired image corresponding to the current group's cooperative parameters.
[0103] In an optional implementation, the processing module 602 is specifically used for: An exposure trigger signal is sent to the image sensor. The exposure trigger signal includes the current sensor exposure time, so as to control the image sensor to expose within the current sensor exposure time and acquire the target pattern to obtain the acquired image corresponding to the current group of cooperative parameters.
[0104] In an optional implementation, the processing module 602 is further configured to: Determine the next set of collaborative parameters from the collaborative parameter configuration table, and obtain the acquisition image corresponding to the next set of collaborative parameters, until multiple sets of collaborative parameters are obtained; The target fused image is obtained by fusing the acquired images corresponding to multiple sets of collaborative parameters.
[0105] In an optional implementation, the processing module 602 is further configured to: Update the preset configuration registers based on the current group collaboration parameters; Read the current group coordination parameters from the configuration register, and control the optical engine to project the target pattern according to the current optical engine mode identifier in the current group coordination parameters.
[0106] In an optional implementation, the processing module 602 is further configured to: Load the current delay time into the preset delay compensation timer and start the delay compensation timer; If the delay compensation timer times out, the current delay time is determined to have elapsed.
[0107] The processing flow of each module in the device and the interaction flow between each module can be referred to the relevant descriptions in the above method embodiments, and will not be detailed here.
[0108] This application also provides a computer-readable storage medium storing a computer program, which is executed by a processor to perform the above-described method.
[0109] In this embodiment, the computer program, when run by the processor, can also execute other machine-readable instructions to perform other methods as described in the embodiments. For details on the specific execution steps and principles, please refer to the description of the embodiments, which will not be repeated here.
[0110] In the embodiments provided in this application, it should be understood that the disclosed apparatus and methods can be implemented in other ways. The apparatus embodiments described above are merely illustrative. For example, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. Furthermore, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Additionally, the displayed or discussed mutual couplings, direct couplings, or communication connections may be through some communication interfaces; indirect couplings or communication connections between devices or units may be electrical, mechanical, or other forms.
[0111] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.
[0112] In addition, the functional units in the embodiments provided in this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit.
[0113] If the aforementioned functions are implemented as software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or a portion of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.
[0114] It should be noted that similar labels and letters in the following figures indicate similar items. Therefore, once an item is defined in one figure, it does not need to be further defined and explained in subsequent figures. In addition, the terms "first", "second", "third", etc. are used only to distinguish descriptions and should not be construed as indicating or implying relative importance.
[0115] Finally, it should be noted that the above-described embodiments are merely specific implementations of this application, used to illustrate the technical solutions of this application, and not to limit them. The protection scope of this application is not limited thereto. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that any person skilled in the art can still modify or easily conceive of changes to the technical solutions described in the foregoing embodiments, or make equivalent substitutions for some of the technical features, within the scope of the technology disclosed in this application; and these modifications, changes, or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application. All should be covered within the protection scope of this application. Therefore, the protection scope of this application should be determined by the protection scope of the claims.
Claims
1. A method of cooperative control of a projection exposure, characterized by The application relates to a programmable logic gate array applied to a cooperative control system, wherein the cooperative control system further comprises a light machine and an image sensor in communication connection with the programmable logic gate array, and the method comprises the following steps: obtaining a cooperative parameter configuration table, wherein the cooperative parameter configuration table comprises a plurality of groups of cooperative parameters, one group of cooperative parameters comprises a light machine mode identifier, a sensor exposure time corresponding to the light machine mode identifier and a delay time, the light machine mode identifier is used for indicating a to-be-projected pattern of the light machine, and the delay time is a stable display time after the to-be-projected pattern is projected; determining a current group of cooperative parameters from the cooperative parameter configuration table; controlling the light machine to project a target pattern indicated by a current light machine mode identifier in the current group of cooperative parameters according to the current light machine mode identifier; after the current delay time in the current group of cooperative parameters, controlling the image sensor to be exposed for a current sensor exposure time in the current group of cooperative parameters, and collecting the target pattern to obtain a collection image corresponding to the current group of cooperative parameters.
2. The method of claim 1, wherein, The method further comprises the following steps after the step of sending the pattern switching trigger signal to the light machine: receiving a synchronization trigger signal sent by the light machine, wherein the synchronization trigger signal is used for indicating that the light machine receives the current light machine mode identifier. The method further comprises the following steps after the step of sending the pattern switching trigger signal to the light machine: receiving a synchronization trigger signal sent by the light machine, wherein the synchronization trigger signal is used for indicating that the light machine receives the current light machine mode identifier.
3. The method of claim 1, wherein, The method further comprises the following steps after the step of sending the pattern switching trigger signal to the light machine: receiving a synchronization trigger signal sent by the light machine, wherein the synchronization trigger signal is used for indicating that the light machine receives the current light machine mode identifier. The method further comprises the following steps after the step of sending the pattern switching trigger signal to the light machine: receiving a synchronization trigger signal sent by the light machine, wherein the synchronization trigger signal is used for indicating that the light machine receives the current light machine mode identifier.
4. The method of claim 1, wherein, The method further comprises the following steps after the step of sending the pattern switching trigger signal to the light machine: receiving a synchronization trigger signal sent by the light machine, wherein the synchronization trigger signal is used for indicating that the light machine receives the current light machine mode identifier. The method further comprises the following steps after the step of sending the pattern switching trigger signal to the light machine: receiving a synchronization trigger signal sent by the light machine, wherein the synchronization trigger signal is used for indicating that the light machine receives the current light machine mode identifier. The method further comprises the following steps after the step of sending the pattern switching trigger signal to the light machine: receiving a synchronization trigger signal sent by the light machine, wherein the synchronization trigger signal is used for indicating that the light machine receives the current light machine mode identifier. The method further comprises the following steps after the step of sending the pattern switching trigger signal to the light machine: receiving a synchronization trigger signal sent by the light machine, wherein the synchronization trigger signal is used for indicating that the light machine receives the current light machine mode identifier. The method further comprises the following steps after the step of sending the pattern switching trigger signal to the light machine: receiving a synchronization trigger signal sent by the light machine, wherein the synchronization trigger signal is used for indicating that the light machine receives the current light machine mode identifier. The method further comprises the following steps after the step of sending the pattern switching trigger signal to the light machine: receiving a synchronization trigger signal sent by the light machine, wherein the synchronization trigger signal is used for indicating that the light machine receives the current light machine mode identifier. The method further comprises the following steps after the step of sending the pattern switching trigger signal to the light machine: receiving a synchronization trigger signal sent by the light machine, wherein the synchronization trigger signal is used for indicating that the light machine receives the current light machine mode identifier. The method further comprises the following steps after the step of sending the pattern switching trigger signal to the light machine: receiving a synchronization trigger signal sent by the light machine, wherein the synchronization trigger signal is used for indicating that the light machine receives the current light machine mode identifier. The method further comprises the following steps after the step of sending the pattern switching trigger signal to the light machine: receiving a synchronization trigger signal sent by the light machine, wherein the synchronization trigger signal is used for indicating that the light machine receives the current light machine mode identifier. The method further comprises the following steps after the step of sending the pattern switching trigger signal to the light machine: receiving a synchronization trigger signal sent by the light machine, wherein the synchronization trigger signal is used for indicating that the light machine receives the current light machine mode identifier. The method further comprises the following steps after the step of sending the pattern switching trigger signal to the light machine: receiving a synchronization trigger signal sent by the light machine, wherein the synchronization trigger signal is used for indicating that the light machine receives the current light machine mode identifier. The method further comprises the following steps after the step of sending the pattern switching trigger signal to the light machine: receiving a synchronization trigger signal sent by the light machine, wherein the synchronization trigger signal is used for indicating that the light machine receives the current light machine mode identifier. The method further comprises the following steps after the step of sending the pattern switching trigger signal to the light machine: receiving a synchronization trigger signal sent by the light machine, wherein the synchronization trigger signal is used for indicating that the light machine receives the current light machine mode identifier. The method further comprises the following steps after the step of sending the pattern switching trigger signal to the light machine: receiving a synchronization trigger signal sent by the light machine, wherein the synchronization trigger signal is used for indicating that the light machine receives the current light machine mode identifier. The method further comprises the following steps after the step of sending the pattern switching trigger signal to the light machine: receiving a synchronization trigger signal sent by the light machine, wherein the synchronization trigger signal is used for indicating that the light machine receives the current light machine mode identifier. The method further comprises the following steps after the step of sending the pattern switching trigger signal to the light machine: receiving a synchronization trigger signal sent by the light machine, wherein the synchronization trigger signal is used for indicating that the light machine receives the current light machine mode identifier. The method further comprises the following steps after the step of sending the pattern switching trigger signal to the light machine: receiving a synchronization trigger signal sent by the light machine, wherein the synchronization trigger signal is used for indicating that the light machine receives the current light machine mode identifier. The method further comprises the following steps after the step of sending the pattern switching trigger signal to the light machine: receiving a synchronization trigger signal sent by the light machine, wherein the synchronization trigger signal is used for indicating that the light machine receives the current light machine mode identifier. The method further comprises the following steps after the step of sending the pattern switching trigger signal to the light machine: receiving a synchronization trigger signal sent by the light machine, wherein the synchronization trigger signal is used for indicating that the light machine receives the current light machine mode identifier. The method further comprises the following steps after the step of sending the pattern switching trigger signal to the light machine: receiving a synchronization trigger signal sent by the light machine, wherein the synchronization trigger signal is used for indicating that the light machine receives the current light machine mode identifier. The method further comprises the following steps after the step of sending the pattern switching trigger signal to the light machine: receiving a synchronization trigger signal sent by the light machine, wherein the synchronization trigger signal is used for indicating that the light machine receives the current light machine mode identifier. The method further comprises the following steps after the step of sending the pattern switching trigger signal to the light machine: receiving a synchronization trigger signal sent by the light machine, wherein the synchronization trigger signal is used for indicating that the light machine receives the current light machine mode identifier. The method further comprises the following steps after the step of sending the pattern switching trigger signal to the light machine: receiving a synchronization trigger signal sent by the light machine, wherein the synchronization trigger signal is used for indicating that the light machine receives the current light machine mode identifier. The method further comprises the following steps after the step of sending the pattern switching trigger signal to the light machine: receiving a synchronization trigger signal sent by the light machine, wherein the synchronization trigger signal is used for indicating that the light machine receives the current light machine mode identifier. The method further comprises the following steps after the step of sending the pattern switching trigger signal to the light machine: receiving a synchronization trigger signal sent by the light machine, wherein the synchronization trigger signal is used for indicating that the light machine receives the current light machine mode identifier. The method further comprises the following steps after the step of sending the pattern switching trigger signal to the light machine: receiving a synchronization trigger signal sent by the light machine, wherein the synchronization trigger signal is used for indicating that the light machine receives the current light machine mode identifier. The method further comprises the following steps after the step of sending the pattern switching trigger signal to the light machine: receiving a synchronization trigger signal sent by the light machine, wherein the synchronization trigger signal is used for indicating that the light machine receives the current light machine mode identifier. The method further comprises the following steps after the step of sending the pattern switching trigger signal to the light machine: receiving a synchronization trigger signal sent by the light machine, wherein the synchronization trigger signal is used for indicating that the light machine receives the current light machine mode identifier. The method further comprises the following steps after the step of sending the pattern switching trigger signal to the light machine: receiving a synchronization trigger signal sent by the light machine, wherein the synchronization trigger signal is used for indicating that 5. The method according to any one of claims 1 to 4, characterized in that, After the exposure of the image sensor according to the current sensor exposure time in the current set of cooperative parameters and the acquisition of the target pattern to obtain the acquisition image corresponding to the current set of cooperative parameters, the method further comprises: determining a next set of cooperative parameters from the cooperative parameter configuration table and acquiring the acquisition image corresponding to the next set of cooperative parameters until the acquisition images corresponding to the multiple sets of cooperative parameters are obtained; performing image fusion according to the acquisition images corresponding to the multiple sets of cooperative parameters to obtain a target fusion image.
6. The method of claim 1, wherein, According to the current light machine mode identifier in the current set of cooperative parameters, the light machine is controlled to project the target pattern indicated by the current light machine mode identifier, comprising: updating a preset configuration register according to the current set of cooperative parameters; reading the current set of cooperative parameters from the configuration register and controlling the light machine to project the target pattern according to the current light machine mode identifier in the current set of cooperative parameters.
7. The method of claim 1, wherein, Before the exposure of the image sensor according to the current sensor exposure time in the current set of cooperative parameters after the current delay time in the current set of cooperative parameters and the acquisition of the target pattern to obtain the acquisition image corresponding to the current set of cooperative parameters, the method further comprises: loading the current delay time to a preset delay compensation timer and starting the delay compensation timer; if the delay compensation timer is timed out, it is determined that the current delay time is reached.
8. A coordinated control system, characterized by comprise: a programmable logic gate array and a light machine and an image sensor, the light machine and the image sensor are respectively in communication connection with the programmable logic gate array; the programmable logic gate array is used for executing the method of any one of claims 1-7.
9. The system of claim 8, wherein, The cooperative control system further comprises: a host computer in communication connection with the programmable logic gate array and a memory.
10. The system of claim 8, wherein, The cooperative control system further comprises: a central processing unit and a graphics processing unit in communication connection, and the central processing unit is further in communication connection with the programmable logic gate array.