Image data and laser information synchronous matching method and three-dimensional scanning device

Abstract

The application relates to the technical field of laser three-dimensional scanning, and discloses a method image data and laser information synchronous matching method and a three-dimensional scanning device, which solves the demand that a general SoC cannot match high-precision real-time processing, high-frequency signal generation and low-delay feedback of a laser module by adopting a combination of an independently-operated SoC control unit and a CPLD control unit, and further confirms execution of laser switching through the CPLD control unit, continuously outputs a laser type code after switching, and simultaneously sends a signal, that is, a laser synchronization code to trigger a hard interrupt, so that image acquisition and laser type code acquisition are synchronized, and finally, image information and laser type information are integrated and associated in the SoC, thereby realizing acquisition of image data associated with the laser type code based on the independently-operated SoC control unit and the CPLD control unit.

Description

Technical Field

[0001] This invention relates to the field of laser 3D scanning technology, specifically to a method for synchronously matching image data and laser information, and a 3D scanning device. Background Technology

[0002] In the field of laser 3D scanning, laser 3D scanning equipment is equipped with a camera module and a laser module. The camera module is generally composed of an image sensor (CIS, CMOS Image Sensor) and a lens, while the laser module is used to project laser features, such as a line laser module or a speckle laser module. The equipment needs to ensure that the laser can be lit during the camera exposure so that the camera can capture the corresponding laser pattern during exposure.

[0003] In existing technologies, scanners typically connect both the CIS (CMOS Image Sensor) and laser module to an FPGA (FPGA-based FPGA) in their hardware design. The FPGA controls both the CIS and laser module, allowing it to naturally know the laser type corresponding to the current frame and easily associate the laser type with image information after the current frame's exposure, ultimately transmitting the image carrying the current laser type information to the algorithm. However, while FPGAs are complex programmable logic devices with powerful functions, they are also expensive. Their use in high-precision industrial-grade laser 3D scanners is understandable, but as laser 3D scanning equipment gradually moves towards the consumer market, the cost-effectiveness demands of this market make the high cost of FPGAs particularly prominent. However, eliminating the FPGA and relying solely on a general-purpose SoC (System-on-a-Chip) to control the laser module presents difficulties in timing accuracy and signal generation capabilities. The high-precision real-time processing, high-frequency signal generation, and low-latency feedback requirements of the laser module are difficult to meet on a general-purpose SoC, necessitating compromises in either cost or performance.

[0004] There is currently no effective solution to the problem of laser module control in laser 3D scanning equipment, which cannot simultaneously achieve high cost-effectiveness and high performance. Summary of the Invention

[0005] The purpose of this invention is to provide a method for synchronously matching image data and laser information in laser 3D scanning equipment, as well as 3D scanning, to solve the problem that existing laser 3D scanning equipment cannot simultaneously achieve high cost-effectiveness and high performance in laser module control.

[0006] In a first aspect, to achieve the aforementioned objective, the present invention provides a method for synchronously matching image data and laser information, applicable to laser 3D scanning equipment. The equipment includes a SoC control unit, a camera module, a laser module, and a CPLD control unit. The CPLD control unit controls the laser module, and the camera module is controlled by the SoC control unit. The method includes: In response to the laser switching command, the CPLD control unit generates a frame synchronization signal and synchronizes it to the CIS of the camera module. At the same time, based on the laser switching command, the CPLD control unit switches to the corresponding laser type and continuously outputs the corresponding laser type code to the first output terminal. The camera module's CIS transmits an imaging data packet containing a laser synchronization code to the SoC control unit based on the frame synchronization signal. The SoC control unit triggers an interrupt service routine based on the laser synchronization code. At the same time, the SoC control unit obtains the laser type code output by the first output terminal of the CPLD control unit and associates the image data in the imaging data packet with the obtained laser type code.

[0007] In the above scheme, after receiving the laser switching command, the CPLD control unit simultaneously generates a frame synchronization signal and synchronizes it to the CIS of the camera module, performs laser type switching, and continuously outputs the corresponding laser type code to the first output terminal. At the same time, it uses the interrupt service routine to construct a synchronization identifier between the CPLD and the SoC, synchronizing the image acquisition and laser type output during the interrupt service routine. This ensures that the laser type code is consistent with the actual laser type switching and the laser type during image acquisition, thus ensuring that laser type data can be acquired synchronously when acquiring images.

[0008] According to the above technical solution, the CIS of the camera module transmits an imaging data packet containing a laser synchronization code to the main control unit based on the frame synchronization signal. The main control unit triggers an interrupt service routine based on the laser synchronization code. Simultaneously, the main control unit outputs the laser type code from the first output terminal of the auxiliary control unit and associates the image data in the imaging data packet with the acquired laser type code. The specific steps include: The camera module's CIS receives the frame synchronization signal, and the camera module's CIS starts exposure and transmits an imaging data packet to the SoC control unit. The imaging data packet includes a laser synchronization code, which is used to trigger the interrupt service routine of the SoC control unit. The SoC control unit receives the imaging data packet and identifies the laser synchronization code in the imaging data packet. Based on the laser synchronization code, it executes an interrupt service routine. The interrupt time of the SoC control unit is consistent with the CIS start exposure time of the camera module. The SoC control unit obtains and caches the laser type code output from the first output terminal of the CPLD control unit based on the interrupt service routine; The SoC control unit executes the interrupt service routine, the CIS of the camera module completes image acquisition, the SoC control unit acquires the image data and associates the laser type code with the image data.

[0009] The above solution employs a combination of a SoC control unit and a CPLD control unit. The SoC controls the camera module's CIS, while the CPLD controls the laser module, perfectly addressing the limitations of a general-purpose SoC in meeting the high-precision real-time processing, high-frequency signal generation, and low-latency feedback requirements of a laser module. However, this also introduces a synchronization problem: the CPLD control unit and the SoC control unit operate independently, with laser control independent of the SoC. There is no direct channel for acquiring laser type data during image acquisition. Therefore, a laser synchronization code is introduced in conjunction with the SoC's hard interrupt service routine. This code triggers the SoC's interrupt service routine. When the laser type changes, the CPLD sends a frame synchronization signal and performs laser switching, continuously outputting a laser type code representing the laser type at the first output terminal. The CIS begins exposure based on the frame synchronization signal and transmits data to the SoC. This includes the laser synchronization code. After the SOC parses the laser synchronization code, it executes an interrupt service routine. At this time, the CIS begins exposure, and the exposure start and interrupt service routine execution are synchronized. This requires system preset to align the two. This is because the laser type information (i.e., laser type code) is obtained by the SoC from the first output of the CPLD during the interrupt service routine execution. Therefore, it is necessary to ensure that the actual laser type in the image data acquired during exposure is consistent with the laser type information obtained by the SoC during the interrupt service routine execution. The best method is to ensure that the interrupt service routine and the CIS start exposure synchronously. This achieves the reading of laser type information and the acquisition of image data as soon as an interrupt occurs. The laser type information is continuously output when the device uses this type of laser, thus ensuring the consistency between the laser type information and the actual laser type captured in the image. At this point, this solution has synchronized the acquisition of image data and laser type information controlled by different control units. Subsequently, the SoC associates the acquired image data and laser type information, thus achieving synchronization between the two.

[0010] According to the above technical solution, the SoC control unit controls the CIS of the camera module through the image acquisition driving framework, and the CIS of the camera module transmits the imaging data packet to the SoC control unit through the corresponding image transmission channel under the image acquisition driving framework.

[0011] According to the above technical solution, the laser synchronization code is a frame start synchronization code, which triggers a hard interrupt of the SoC control unit.

[0012] According to the above technical solution, the SoC control unit acquires image data and associates the laser type encoding with the image data, including: After the camera module's CIS completes image acquisition, the SoC control unit obtains the imaging data packet output by the camera module's CIS to form image data, and stores the image data in the image buffer of the image acquisition driving framework. When the SoC control unit processes the image buffer, it associates the cached laser type code with the image data.

[0013] The above scheme specifically adopts an image acquisition driving framework and its corresponding image transmission channel, and performs image data association between image data and laser-coded data in the buffer.

[0014] According to the above technical solution, the first output terminal is the GPIO pin that connects the CPLD control unit and the SoC control unit.

[0015] According to the above technical solution, the GPIO pins connected to the CPLD control unit are mapped to the image signal processor descriptor in the SoC control unit's device tree.

[0016] According to the above technical solution, the laser type encoding is represented by N-bit binary encoding.

[0017] According to the above technical solution, the number of bits in the binary code used for laser type encoding is consistent with and corresponds one-to-one with the number of GPIO pins connected to the CPLD control unit SoC control unit.

[0018] According to the above technical solution, the number N of binary encoding bits used in the laser type encoding is determined based on the number of laser types, and N binary bits can represent at most 2^N laser types.

[0019] In the above-mentioned schemes, the GPIO pins connected to the CPLD control unit and the SoC control unit are mapped to the binary encoding bits of the laser type code. By using the high and low levels of the pins to map the binary 0 and 1 values, the state of the pins can be reflected in the hardware to show the laser type code. The SoC then obtains the code and associates it with the image. Furthermore, only a small number of pins are needed to represent a certain number of laser types. For example, 3 pins can represent 8 types, namely 000: type 0; 001: type 1; 010: type 2, and so on.

[0020] In a second aspect, a laser 3D scanning device includes a SoC control unit, a camera module, a laser module, and a CPLD control unit. The CPLD control unit is used to control the laser module. The SoC control unit and the CPLD control unit are electrically connected through several GPIO pins. The laser module is connected to the CPLD control unit, and the CIS of the camera module is connected to the SoC control unit. In the device tree of the SoC control unit, the GPIO pins connected to the CPLD control unit are mapped to the descriptors of the image signal processor in the SoC control unit. The device runs the image data and laser information synchronization matching method described in the first aspect above.

[0021] According to the above technical solution, the CIS of the camera module communicates with the SoC control unit through the MIPI CSI-2 interface.

[0022] Thirdly, a computer device, characterized in that it includes a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the steps of the image data and laser information synchronization matching method described in any of the preceding claims.

[0023] Fourthly, a computer-readable storage medium having a computer program stored thereon, characterized in that: when the computer program is executed by a processor, it implements the steps of the image data and laser information synchronization matching method described in any of the preceding claims.

[0024] The present invention has the following beneficial effects: 1. This invention controls the CIS of the camera module and the laser module respectively through the SoC control unit and the CPLD control unit, ensuring that the control requirements of high-precision real-time processing, high-frequency signal generation and low-latency feedback of the laser module are met. By introducing laser synchronization code in combination with the hard interrupt service routine of the SoC, it ensures that the independently operating SoC control unit and CPLD control unit can associate image data with laser type information (i.e. laser type encoding), so as to obtain image data associated with real-time laser type information. Finally, the control method of SoC combined with CPLD solves the problem of controlling laser modules in consumer-grade laser 3D scanning that is difficult to balance high cost performance and high performance.

[0025] 2. This invention maps the GPIO pins connected to the CPLD control unit and the SoC control unit on the hardware to the binary encoding bits of the laser type code. By using the high and low levels of the pins to map the binary 0 and 1 values, the state of the pins can be reflected in the hardware to show the laser type code. The SoC then obtains the code and associates it with the image. Furthermore, only a small number of pins are needed to represent a certain number of laser types. For example, 3 pins can represent 8 types, namely 000: type 0; 001: type 1; 010: type 2, and so on.

[0026] The above description is merely an overview of the technical solution of the present invention. In order to better understand the technical means of the present invention and to implement it according to the contents of the specification, the preferred embodiments of the present invention are described in detail below with reference to the accompanying drawings. Specific embodiments of the present invention are given in detail below with reference to the accompanying drawings. Attached Figure Description

[0027] The accompanying drawings, which are included to provide a further understanding of the invention and form part of this application, illustrate exemplary embodiments of the invention and are used to explain the invention, but do not constitute an undue limitation of the invention.

[0028] Figure 1 This is a schematic diagram of a device frame according to an embodiment of the present invention; Figure 2 This is a schematic flowchart of an embodiment provided by the present invention; Detailed Implementation To make the objectives, technical solutions, and advantages of this application clearer, the application is described and illustrated below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the application. All other embodiments obtained by those skilled in the art based on the embodiments provided in this application without inventive effort are within the scope of protection of this application. Furthermore, it is understood that although the efforts made in such a development process may be complex and lengthy, for those skilled in the art related to the content disclosed in this application, modifications to design, manufacturing, or production based on the technical content disclosed in this application are merely conventional technical means and should not be construed as insufficient disclosure of the content of this application.

[0029] In this application, the reference to "embodiment" means that a specific feature, structure, or characteristic described in connection with an embodiment may be included in at least one embodiment of this application. The appearance of this phrase in various places in the specification does not necessarily refer to the same embodiment, nor is it a separate or alternative embodiment that is mutually exclusive with other embodiments. It will be explicitly and implicitly understood by those skilled in the art that the embodiments described in this application may be combined with other embodiments without conflict.

[0030] Unless otherwise defined, the technical or scientific terms used in this application shall have the ordinary meaning understood by one of ordinary skill in the art to which this application pertains. The terms “a,” “an,” “an,” “the,” and similar words used in this application do not indicate quantity limitation and may indicate singular or plural. The terms “comprising,” “including,” “having,” and any variations thereof used in this application are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or device that includes a series of steps or modules (units) is not limited to the listed steps or units, but may also include steps or units not listed, or may include other steps or units inherent to these processes, methods, products, or devices. The terms “connected,” “linked,” “coupled,” and similar words used in this application are not limited to physical or mechanical connections, but may include electrical connections, whether direct or indirect. “Multiple” used in this application means two or more. “And / or” describes the relationship between related objects, indicating that three relationships may exist; for example, “A and / or B” can represent: A alone, A and B simultaneously, and B alone. The terms “first,” “second,” “third,” etc., used in this application are merely to distinguish similar objects and do not represent a specific ordering of the objects.

[0031] Example 1 Reference Figures 1-2As shown, this invention provides a method for synchronizing and matching image data and laser information, applicable to laser 3D scanning devices. This embodiment uses a handheld laser 3D scanner as an example to illustrate the overall implementation steps of the method. First, it is necessary to clarify the pre-hardware state of the solution provided by this invention. The laser 3D scanning device includes a SoC control unit, a camera module, a laser module, and a CPLD control unit. The CPLD control unit is used to control the laser module. The SoC control unit and the CPLD control unit are electrically connected through several GPIO pins. The laser module is connected to the CPLD control unit, and the CIS of the camera module is connected to the SoC control unit. In the device tree of the SoC control unit, the GPIO pins connected to the CPLD control unit are mapped to the descriptors of the image signal processor in the SoC control unit. The device then runs the image data and laser information synchronization and matching method. The specific method steps are as follows: Figure 2 As shown: In response to the laser switching command, the CPLD control unit generates a frame synchronization signal and synchronizes it to the CIS of the camera module. At the same time, based on the laser switching command, the CPLD control unit switches to the corresponding laser type and continuously outputs the corresponding laser type code to the first output terminal. The camera module's CIS transmits an imaging data packet containing a laser synchronization code to the SoC control unit based on the frame synchronization signal. The SoC control unit triggers an interrupt service routine based on the laser synchronization code. At the same time, the SoC control unit obtains the laser type code output by the first output terminal of the CPLD control unit and associates the image data in the imaging data packet with the obtained laser type code.

[0032] In the above scheme, after receiving the laser switching command, the CPLD control unit simultaneously generates a frame synchronization signal and synchronizes it to the CIS of the camera module, performs laser type switching, and continuously outputs the corresponding laser type code to the first output terminal. At the same time, it uses the interrupt service routine to construct a synchronization identifier between the CPLD and the SoC, synchronizing the image acquisition and laser type output during the interrupt service routine. This ensures that the laser type code is consistent with the actual laser type switching and the laser type during image acquisition, thus ensuring that laser type data can be acquired synchronously when acquiring images.

[0033] Furthermore, the CIS of the camera module transmits an imaging data packet containing a laser synchronization code to the main control unit based on the frame synchronization signal. The main control unit triggers an interrupt service routine based on the laser synchronization code. Simultaneously, the main control unit outputs the laser type code from the first output terminal of the auxiliary control unit and associates the image data in the imaging data packet with the acquired laser type code. The specific steps include: The camera module's CIS receives the frame synchronization signal, and the camera module's CIS starts exposure and transmits an imaging data packet to the SoC control unit. The imaging data packet includes a laser synchronization code, which is used to trigger the interrupt service routine of the SoC control unit. The SoC control unit receives the imaging data packet and identifies the laser synchronization code in the imaging data packet. Based on the laser synchronization code, it executes an interrupt service routine. The interrupt time of the SoC control unit is consistent with the CIS start exposure time of the camera module. The SoC control unit obtains and caches the laser type code output from the first output terminal of the CPLD control unit based on the interrupt service routine; The SoC control unit executes the interrupt service routine, the CIS of the camera module completes image acquisition, the SoC control unit acquires the image data and associates the laser type code with the image data.

[0034] The above solution employs a combination of a SoC control unit and a CPLD control unit. The SoC controls the camera module's CIS, while the CPLD controls the laser module, perfectly addressing the limitations of a general-purpose SoC in meeting the high-precision real-time processing, high-frequency signal generation, and low-latency feedback requirements of a laser module. However, this also introduces a synchronization problem: the CPLD control unit and the SoC control unit operate independently, with laser control independent of the SoC. There is no direct channel for acquiring laser type data during image acquisition. Therefore, a laser synchronization code is introduced in conjunction with the SoC's hard interrupt service routine. This code triggers the SoC's interrupt service routine. When the laser type changes, the CPLD sends a frame synchronization signal and performs laser switching, continuously outputting a laser type code representing the laser type at the first output terminal. The CIS begins exposure based on the frame synchronization signal and transmits data to the SoC. This includes the laser synchronization code. After the SoC parses the laser synchronization code, it executes an interrupt service routine. At this time, the CIS begins exposure. The exposure start and the interrupt service routine execution are synchronized. This requires system preset to align the two. This is because the laser type information (i.e., laser type code) is obtained by the SoC from the first output of the CPLD during the interrupt service routine execution. Therefore, it is necessary to ensure that the actual laser type in the image data acquired during exposure is consistent with the laser type information obtained by the SoC during the interrupt service routine execution. The best method is to ensure that the interrupt service routine and the CIS start exposure synchronously. This achieves the reading of laser type information and the acquisition of image data as soon as an interrupt occurs. The laser type information is continuously output when the device uses this type of laser, thus ensuring the consistency between the laser type information and the actual laser type captured in the image. At this point, this solution has synchronized the acquisition of image data and laser type information controlled by different control units. Subsequently, the SoC associates the acquired image data and laser type information, thus achieving synchronization between the two.

[0035] Furthermore, the SoC control unit controls the CIS of the camera module through the image acquisition driving framework, and the CIS of the camera module transmits the imaging data packet to the SoC control unit through the corresponding image transmission channel under the image acquisition driving framework.

[0036] Furthermore, the laser synchronization code is a frame start synchronization code, which triggers a hard interrupt in the SoC control unit.

[0037] Furthermore, the SoC control unit acquires image data and associates the laser type encoding with the image data, including: After the camera module's CIS completes image acquisition, the SoC control unit obtains the imaging data packet output by the camera module's CIS to form image data, and stores the image data in the image buffer of the image acquisition driving framework. When the SoC control unit processes the image buffer, it associates the cached laser type code with the image data.

[0038] Furthermore, the first output terminal is a GPIO pin that connects the CPLD control unit and the SoC control unit.

[0039] Furthermore, in the device tree of the SoC control unit, the GPIO pins connected to the CPLD control unit are mapped to the descriptors of the image signal processor in the SoC control unit.

[0040] Furthermore, the laser type encoding is represented using N-bit binary encoding.

[0041] Furthermore, the number of bits in the binary code used for the laser type encoding is consistent with and corresponds one-to-one with the number of GPIO pins connected to the CPLD control unit SoC control unit.

[0042] Furthermore, the number of binary encoding bits N used in the laser type encoding is determined based on the number of laser types, and N binary bits can represent at most 2^N laser types.

[0043] In summary, the specific solution provided by this invention can be a low-cost architecture for a handheld laser 3D scanner, using a SoC as the main control unit to control the CIS image sensor and a CPLD as an auxiliary control unit to control multiple lasers. The CPLD and SoC are connected via multiple GPIO pins. The CPLD converts the currently polled laser type into a binary level code and continuously outputs it to the GPIO pins. The number of pins is configured according to requirements. Specifically, the GPIO pins connected to the CPLD control unit and the SoC control unit on the hardware are mapped to the binary code bits of the laser type code. By using the high and low levels of the pins to map the binary 0 and 1 values, the state of the pins can be reflected in the hardware to show the laser type code. The SoC then obtains this code and associates it with the image. Furthermore, only a small number of pins are needed to represent a certain number of laser types. For example, 3 pins can represent 8 types (i.e., 000: type 0; 001: type 1; 010: type 2, and so on). Currently, few laser 3D scanners on the market have more than 8 laser types, and 3 pins are sufficient for the existing scenarios.

[0044] More specifically, in this embodiment, the CIS and SoC communicate through the MIPI CSI-2 interface. The GPIO pins in the device tree are mapped to the ISP module. The system runs based on the standard Linux V4L2 video framework (image acquisition driver framework). The laser type code is written into the reserved2 standard reserved field of the V4L2_buffer structure, which realizes the binding of image data and laser type at the kernel layer without modifying the V4L2 standard framework. Finally, in user space, the upper layer application can obtain the relationship between the acquired image and the laser type without non-standard modifications.

[0045] Combining the specific laser 3D scanner and image data with laser information synchronization matching methods described above, such as... Figure 1 As shown, the solution provided by this invention satisfies the following conditions: Pre-hardware status configuration The device employs a low-cost architecture where the SoC controls the CIS image sensor and the CPLD controls multiple lasers. The CPLD and SoC are connected via multiple GPIO pins, and the CPLD continuously outputs the currently polled laser type as a binary level code to the GPIO pins. The CIS and SoC communicate via the MIPI CSI-2 interface, and the device tree has completed the mapping of GPIO pins to the ISP module. The system runs on the standard Linux V4L2 video framework.

[0046] Frame synchronization triggers, laser type switches synchronously. The CPLD generates a frame synchronization trigger signal (FSYNC) and sends it to the CIS. At the same time, it completes the laser switching for this round, updates the binary code on the GPIO pin, and outputs the laser type code corresponding to the current frame.

[0047] CIS initiates exposure, transmitting image data stream with synchronization code. The SoC issues a capture command, and the CIS begins to expose the capture screen. The captured raw image data is transmitted to the SoC through the MIPI CSI-2 channel, and the SOF (Start of Frame) synchronization code is embedded in the data stream.

[0048] Hardware recognition of synchronization code triggers CPU hardware interrupt. The SoC's internal MIPI CSI-2 protocol layer hardware parses the SOF frame start code and sends a hardware interrupt request to the CPU via the GIC general interrupt controller. At this time, the CIS has just entered the exposure stage and the timing is completely synchronized.

[0049] Reading laser type code in interrupt service The CPU enters the hard interrupt service routine, reads the level signal output by the CPLD in real time through the pre-mapped GPIO pins, converts the binary level into the corresponding laser type number, and temporarily caches the value.

[0050] The interrupt ends, and the image data is stored in the video buffer. The interrupt task is completed and exits; after CIS acquisition is completed, the raw image data is transferred to memory via DMA and stored in the Videoobuf2 (VB2) video buffer of the V4L2 framework.

[0051] Kernel layer binding: Laser type injection image buffer field When the kernel processes the video buffer, the previously cached laser type value is written into the reserved2 standard reserved field of the v4l2_buffer structure. This achieves the binding of image data and laser type at the kernel level without modifying the V4L2 standard framework.

[0052] Standard interface reporting, application layer obtains related data The upper-layer application can retrieve video frames by calling the standard V4L2 interface VIDIOC_DQBUF and reading the contents of the v4l2_buffer structure to obtain complete image data and corresponding laser type information at the same time, without the need for customized modifications to the application.

[0053] At this point, a single image acquisition is complete, and the laser type is accurately correlated with the image frame throughout the entire process.

[0054] It's important to note that the CIS (Image Sensor) hardware is connected to the SoC (System-on-Chips), which handles initialization, register configuration, and operating mode settings. The laser hardware is connected to the CPLD (Content-Based Logic Controller), which is responsible for laser polling and timing control. The SoC is the sole master controller of the CIS; its power-on initialization, exposure parameters, image format, data stream enablement, and MIPI link management are all configured and issued by the SoC via I2C / MIPI buses. Without SoC configuration, the CIS cannot function properly. The CPLD does not have the ability to configure or modify the parameters of the CIS; it only outputs a pure hardware trigger pulse, FSYNC, which is the frame synchronization signal.

[0055] The above scheme enables precise end-to-end association between laser type and the image frame after a single image acquisition. The key is that the CPLD independently controls the laser component, satisfying the control requirements of the laser component while transforming the problem into a synchronization issue between the independently operating SoC and the CPLD. Then, by adding a laser synchronization code, i.e., the SOF frame start synchronization code, a synchronization moment between the SoC and the CPLD is artificially set. At this time, the CPLD ensures that it outputs a laser synchronization code after laser switching to notify the SoC to synchronize. It also ensures that its first output terminal, i.e., the GPIO pin connected to the SoC, continuously outputs the laser type code corresponding to the switched laser. Then, the CIS exposure begins synchronously with the SoC reading the laser type code, ensuring that the laser in the image captured by the CIS is definitely the switched laser. The code of the switched laser type is continuously output through the GPIO pin, so that the SoC's pre-mapped GPIO pin can read the level signal output by the CPLD in real time, convert the binary level into the corresponding laser type number, and temporarily cache the value. Later, the laser type number is associated with the acquired image data. In other words, when the original technical problem is transformed into the synchronization problem between the independently operating SoC and CPLD, the main idea of ​​this invention is to continuously output the type information after the CPLD confirms the laser switching and simultaneously send a signal (laser synchronization code) to trigger a hard interrupt, thereby synchronizing the acquisition of image and laser type information. Finally, the image information and laser type information are integrated and associated in the SoC, thus solving the problem.

[0056] Example 2 This invention provides a schematic diagram of the structure of a computer device, such as a smartphone, tablet computer, laptop computer, desktop computer, rack server, blade server, tower server, or cabinet server (including standalone servers or server clusters composed of multiple servers) capable of executing programs. The computer device in this embodiment includes, but is not limited to, a memory and a processor that can be interconnected via a system bus. It should be noted that only a computer device with components is shown; however, it should be understood that it is not required to implement all the components shown, and more or fewer components may be implemented alternatively.

[0057] In this embodiment, the memory (i.e., the readable storage medium) includes flash memory, hard disk, multimedia card, card-type memory (e.g., SD or DX memory), random access memory (RAM), static random access memory (SRAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), and programmable read-only memory (PROM). The memory can also be an external storage device of the computer device, such as a plug-in hard disk, smart media card (SMC), secure digital card (SD), flash card, etc., equipped on the computer device. Of course, the memory can also include both internal storage units and external storage devices of the computer device. In this embodiment, the memory is typically used to store the operating system and various application software installed on the computer device, such as the program code for the steps of the image data and laser information synchronization matching method in the method embodiment. Furthermore, the memory can also be used to temporarily store various types of data that have been output or will be output.

[0058] In some embodiments, the processor may be a central processing unit (CPU), a controller, a microcontroller, a microprocessor, or other data processing chip. The processor is typically used to control the overall operation of a computer device. In this embodiment, the processor is used to run program code stored in memory or process data, for example, to run program code or process data for a method for synchronizing and matching image data and laser information, in order to achieve the synchronization and matching of image data and laser information in the method embodiment.

[0059] Example 3 A computer-readable storage medium is provided, such as flash memory, hard disk, multimedia card, card-type memory (e.g., SD or DX memory), random access memory (RAM), static random access memory (SRAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), programmable read-only memory (PROM), magnetic memory, disk, optical disk, server, app store, etc., storing a computer program thereon. When the program is executed by a processor, it performs corresponding functions. The program code or processing data of the image data and laser information synchronization matching method of the computer-readable storage medium in this embodiment, when executed by a processor, implements the image data and laser information synchronization matching of the method embodiment.

[0060] It should be noted that, depending on the implementation needs, the various steps / components described in this application can be broken down into more steps / components, or two or more steps / components or parts of the operation of steps / components can be combined into new steps / components to achieve the purpose of this invention.

[0061] Those skilled in the art will readily understand that the above description is merely a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A method for synchronously matching image data and laser information, characterized in that, A method applicable to laser 3D scanning equipment, the equipment including a SoC control unit, a camera module, a laser module, and a CPLD control unit, wherein the CPLD control unit controls the laser module, and the camera module is controlled by the SoC control unit, the method comprising: In response to the laser switching command, the CPLD control unit generates a frame synchronization signal and synchronizes it to the CIS of the camera module. At the same time, based on the laser switching command, the CPLD control unit switches to the corresponding laser type and continuously outputs the corresponding laser type code to the first output terminal. The camera module's CIS transmits an imaging data packet containing a laser synchronization code to the SoC control unit based on the frame synchronization signal. The SoC control unit triggers an interrupt service routine based on the laser synchronization code. At the same time, the SoC control unit obtains the laser type code output by the first output terminal of the CPLD control unit and associates the image data in the imaging data packet with the obtained laser type code.

2. The method for synchronously matching image data and laser information according to claim 1, characterized in that, The camera module's CIS transmits an imaging data packet containing a laser synchronization code to the main control unit based on the frame synchronization signal. The main control unit triggers an interrupt service routine based on the laser synchronization code. Simultaneously, the main control unit outputs the laser type code from the first output terminal of the auxiliary control unit and associates the image data in the imaging data packet with the acquired laser type code. The specific steps include: The camera module's CIS receives the frame synchronization signal, and the camera module's CIS starts exposure and transmits an imaging data packet to the SoC control unit. The imaging data packet includes a laser synchronization code, which is used to trigger the interrupt service routine of the SoC control unit. The SoC control unit receives the imaging data packet and identifies the laser synchronization code in the imaging data packet. Based on the laser synchronization code, it executes an interrupt service routine. The interrupt time of the SoC control unit is consistent with the CIS start exposure time of the camera module. The SoC control unit obtains and caches the laser type code output from the first output terminal of the CPLD control unit based on the interrupt service routine; The SoC control unit executes the interrupt service routine, the CIS of the camera module completes image acquisition, the SoC control unit acquires the image data and associates the laser type code with the image data.

3. The method for synchronously matching image data and laser information according to claim 2, characterized in that, The SoC control unit controls the CIS of the camera module through the image acquisition driving framework, and the CIS of the camera module transmits the imaging data packet to the SoC control unit through the corresponding image transmission channel under the image acquisition driving framework.

4. The method for synchronously matching image data and laser information according to claim 3, characterized in that, The laser synchronization code is a frame start synchronization code, which triggers a hard interrupt in the SoC control unit.

5. The method for synchronously matching image data and laser information according to claim 3 or 4, characterized in that, The SoC control unit acquires image data and associates the laser type encoding with the image data, including: After the camera module's CIS completes image acquisition, the SoC control unit obtains the imaging data packet output by the camera module's CIS to form image data, and stores the image data in the image buffer of the image acquisition driving framework. When the SoC control unit processes the image buffer, it associates the cached laser type code with the image data.

6. The method for synchronously matching image data and laser information according to claim 5, characterized in that, The first output terminal is a GPIO pin that connects the CPLD control unit and the SoC control unit.

7. The method for synchronously matching image data and laser information according to claim 6, characterized in that, In the device tree of the SoC control unit, the GPIO pins connected to the CPLD control unit are mapped to the descriptors of the image signal processor in the SoC control unit.

8. The method for synchronously matching image data and laser information according to claim 7, characterized in that, The laser type is represented by an N-bit binary code.

9. The method for synchronously matching image data and laser information according to claim 8, characterized in that, The number of bits in the binary code used for the laser type encoding is consistent with and corresponds one-to-one with the number of GPIO pins connected to the CPLD control unit SoC control unit.

10. The method for synchronously matching image data and laser information according to claim 9, characterized in that, The number of binary code bits N used in the laser type encoding is determined based on the number of laser types. N binary code bits can represent at most 2^N laser types.

11. A laser 3D scanning device, comprising a SoC control unit, a camera module, a laser module, and a CPLD control unit, wherein the CPLD control unit is used to control the laser module, the SoC control unit and the CPLD control unit are electrically connected via a plurality of GPIO pins, the laser module is connected to the CPLD control unit, the CIS of the camera module is connected to the SoC control unit, and the GPIO pins connected to the CPLD control unit are mapped to the descriptors of the image signal processor in the SoC control unit in the device tree of the SoC control unit, and the device runs the image data and laser information synchronization matching method of any one of claims 1 to 10.

12. The laser three-dimensional scanning device according to claim 11, characterized in that, The camera module's CIS communicates with the SoC control unit via an image acquisition interface.

13. A computer device, characterized in that: The method includes a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, it implements the steps of the image data and laser information synchronization matching method according to any one of claims 1 to 10.

14. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by the processor, it implements the steps of the image data and laser information synchronization matching method according to any one of claims 1 to 10.

Images