FPGA-based hardware and software collaborative acceleration system for hyperspectral data dimension reduction

Abstract

This invention relates to the field of hardware acceleration technology and discloses a hardware and software collaborative acceleration system for hyperspectral data dimensionality reduction based on FPGA. The system includes: a hardware platform for receiving frames of hyperspectral data sent by a hyperspectral camera, and performing dimensionality reduction processing on each frame of hyperspectral data according to band selection information, obtaining dimensionality-reduced hyperspectral data frames which are then sent to a host computer; a host computer for sending band selection information to the hardware platform; the host computer is also used to send camera configuration information to the hardware platform; and the hardware platform is further used to forward the camera configuration information to the hyperspectral camera, so that the hyperspectral camera can acquire each frame of hyperspectral data according to the camera configuration information. This solution can improve data transmission efficiency and achieve real-time transmission of hyperspectral data.

Description

Technical Field

[0001] This invention relates to the field of data dimensionality reduction technology, specifically to a hardware and software collaborative acceleration system for hyperspectral data dimensionality reduction based on FPGA. Background Technology

[0002] In hyperspectral imaging system architecture, the Camera Link interface, due to its ultra-high theoretical transmission bandwidth, has become one of the high-speed data transmission interfaces used in current digital industrial cameras and hyperspectral cameras. Hyperspectral data typically contains a large amount of band information, and directly storing data from all bands would result in enormous storage costs.

[0003] In related technologies, software computation methods are used to process hyperspectral data in order to reduce the burden of data storage and transmission. However, in the case of high-speed data streams, software computation often cannot meet the real-time requirements. In addition, the image acquisition cards in related technologies have interface protocol incompatibility issues and cannot directly parse the physical layer encoding of Camera Link. The system architecture is also difficult to meet the real-time throughput requirements of hyperspectral data. Summary of the Invention

[0004] In view of this, the purpose of this invention is to provide a hardware and software collaborative acceleration system for hyperspectral data dimensionality reduction based on FPGA, so as to solve the problem of poor real-time performance when acquiring hyperspectral data in related technologies.

[0005] In a first aspect, the present invention provides a hardware and software collaborative acceleration system for dimensionality reduction of hyperspectral data based on FPGA, the system comprising:

[0006] The hardware platform is used to receive each frame of hyperspectral data sent by the hyperspectral camera, and to perform dimensionality reduction processing on each frame of hyperspectral data according to the band selection information, so as to obtain each frame of hyperspectral dimensionality reduction data and send it to the host computer.

[0007] The host computer is used to send band selection information to the hardware platform; the host computer is also used to send camera configuration information to the hardware platform; the hardware platform is also used to forward the camera configuration information to the hyperspectral camera, so that the hyperspectral camera can collect each frame of hyperspectral data according to the camera configuration information.

[0008] In one optional implementation, the hardware platform is further configured to receive a low-speed mode switching command sent by a host computer and send target frame hyperspectral data to the host computer.

[0009] The host computer is also used to receive the target frame hyperspectral data; the host computer is also used to extract features from the target frame hyperspectral data through a band selection algorithm to obtain band selection information, and send the band selection information to the hardware platform;

[0010] The host computer is also used to send a high-speed mode switching command to the hardware platform; the hardware platform is also used to receive each frame of hyperspectral data sent by the hyperspectral camera after receiving the high-speed mode switching command.

[0011] In one alternative implementation, the system further includes a Camera Link interface; the hyperspectral camera sends frames of hyperspectral data to the hardware platform via the Camera Link interface.

[0012] In one optional implementation, the system further includes a first UART serial port; the hardware platform forwards the camera configuration information to the hyperspectral camera through the first UART serial port.

[0013] In one alternative implementation, the camera configuration information includes at least one of camera frame rate, pixel cropping, camera gain, and camera model.

[0014] In one optional implementation, the system further includes a UDP gigabit Ethernet interface; the hardware platform sends each frame of hyperspectral dimensionality reduction data to the host computer via the UDP gigabit Ethernet interface.

[0015] In one optional implementation, the system further includes a second UART serial port; the host computer sends band selection information to the hardware platform through the second UART serial port.

[0016] In one alternative implementation, the hardware platform includes an image storage module; the image storage module is used to cache the received frames of hyperspectral data.

[0017] Secondly, this invention provides a hardware and software collaborative acceleration method for hyperspectral data dimensionality reduction based on FPGA, executed by a host computer in the FPGA-based hyperspectral data dimensionality reduction hardware and software collaborative acceleration system described in the first aspect or any corresponding embodiment; the method includes:

[0018] Receive target frame hyperspectral data sent by the hardware platform;

[0019] The band selection information is obtained by extracting features from the hyperspectral data of the target frame using a band selection algorithm.

[0020] The band selection information is sent to the hardware platform so that the hardware platform can perform dimensionality reduction processing on each frame of hyperspectral data according to the band selection information.

[0021] Thirdly, the present invention provides a hardware and software collaborative acceleration method for hyperspectral data dimensionality reduction based on FPGA, executed by a hardware platform in the hardware and software collaborative acceleration system for hyperspectral data dimensionality reduction based on FPGA described in the first aspect or any corresponding embodiment; the method includes:

[0022] Receive target frame hyperspectral data sent by a hyperspectral camera;

[0023] The system receives a low-speed mode switching command from the host computer and sends target frame hyperspectral data to the host computer so that the host computer can extract features from the target frame hyperspectral data using a band selection algorithm to obtain band selection information.

[0024] Receive band selection information sent by the host computer;

[0025] After receiving the high-speed mode switching command sent by the host computer, the system receives each frame of hyperspectral data sent by the hyperspectral camera, and performs dimensionality reduction processing on each frame of hyperspectral data according to the band selection information, and sends the dimensionality-reduced hyperspectral data of each frame to the host computer.

[0026] Fourthly, the present invention provides a hardware and software collaborative acceleration device for hyperspectral data dimensionality reduction based on FPGA, applied to the host computer in the hardware and software collaborative acceleration system for hyperspectral data dimensionality reduction based on FPGA described in the first aspect or any corresponding embodiment; the device includes:

[0027] The first receiving module is used to receive target frame hyperspectral data sent by the hardware platform;

[0028] The band extraction module is used to extract features from the hyperspectral data of the target frame using a band selection algorithm to obtain band selection information.

[0029] The first transmitting module is used to transmit the band selection information to the hardware platform, so that the hardware platform can perform dimensionality reduction processing on each frame of hyperspectral data according to the band selection information.

[0030] Fifthly, the present invention provides a hardware and software co-acceleration device for hyperspectral data dimensionality reduction based on FPGA, applied to the hardware platform of the hardware and software co-acceleration system for hyperspectral data dimensionality reduction based on FPGA described in the first aspect or any corresponding embodiment; the device includes:

[0031] The second receiving module is used to receive target frame hyperspectral data sent by the hyperspectral camera;

[0032] The second sending module is used to receive the low-speed mode switching command sent by the host computer and send the target frame hyperspectral data to the host computer so that the host computer can extract features from the target frame hyperspectral data through a band selection algorithm to obtain band selection information.

[0033] The third receiving module is used to receive band selection information sent by the host computer;

[0034] The dimensionality reduction module is used to receive each frame of hyperspectral data sent by the hyperspectral camera after receiving the high-speed mode switching command sent by the host computer, and to perform dimensionality reduction processing on each frame of hyperspectral data according to the band selection information, so as to obtain each frame of hyperspectral dimensionality reduction data and send it to the host computer.

[0035] In a sixth aspect, the present invention provides a computer device, comprising: a memory and a processor, wherein the memory and the processor are communicatively connected to each other, the memory stores computer instructions, and the processor executes the computer instructions to perform the FPGA-based hyperspectral data dimensionality reduction hardware and software co-acceleration method described in the second or third aspect above.

[0036] In a seventh aspect, the present invention provides a computer-readable storage medium storing computer instructions for causing a computer to execute the FPGA-based hardware and software co-acceleration method for hyperspectral data dimensionality reduction described in the second or third aspect above.

[0037] Eighthly, the present invention provides a computer program product, including computer instructions for causing a computer to execute the FPGA-based hyperspectral data dimensionality reduction hardware and software co-acceleration method described in the second or third aspect above.

[0038] The technical solution provided by this invention may include the following beneficial effects:

[0039] This invention provides a hardware and software collaborative acceleration system for hyperspectral data dimensionality reduction based on FPGA. The system includes a hardware platform and a host computer. The hardware platform receives frames of hyperspectral data sent by a hyperspectral camera and performs dimensionality reduction processing on each frame according to band selection information, obtaining dimensionality-reduced hyperspectral data which is then sent to the host computer. The host computer sends band selection information to the hardware platform and camera configuration information. The hardware platform also forwards the camera configuration information to the hyperspectral camera, enabling it to acquire each frame of hyperspectral data according to the configuration information. This solution, by sending camera configuration information through the host computer, forwarding it to the hyperspectral camera through the hardware platform to adjust the camera's data acquisition status, and sending band selection information to the hardware platform, allows the hardware platform to perform dimensionality reduction processing on each frame of hyperspectral data, reducing the data volume before transmission to the host computer. This improves data transmission efficiency and enables collaborative work between the host computer, hardware platform, and hyperspectral camera, thereby achieving real-time transmission of hyperspectral data. Attached Figure Description

[0040] To more clearly illustrate the specific embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the specific embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0041] Figure 1 This is a schematic diagram of the hardware and software collaborative acceleration system for FPGA-based hyperspectral data dimensionality reduction according to an embodiment of the present invention;

[0042] Figure 2 This is a schematic diagram of data interaction according to an embodiment of the present invention;

[0043] Figure 3 This is a flowchart illustrating the hardware and software collaborative acceleration method for dimensionality reduction of hyperspectral data based on FPGA according to an embodiment of the present invention.

[0044] Figure 4 This is a flowchart illustrating another hardware and software collaborative acceleration method for dimensionality reduction of hyperspectral data based on FPGA according to an embodiment of the present invention.

[0045] Figure 5 This is a flowchart illustrating another hardware and software collaborative acceleration method for hyperspectral data dimensionality reduction based on FPGA according to an embodiment of the present invention.

[0046] Figure 6 This is a structural block diagram of a hardware and software co-acceleration device for FPGA-based hyperspectral data dimensionality reduction according to an embodiment of the present invention.

[0047] Figure 7 This is a structural block diagram of another FPGA-based hardware and software co-acceleration device for hyperspectral data dimensionality reduction according to an embodiment of the present invention;

[0048] Figure 8 This is a schematic diagram of the hardware structure of a computer device according to an embodiment of the present invention. Detailed Implementation

[0049] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0050] In hyperspectral imaging system architectures, the Camera Link interface, due to its ultra-high theoretical transmission bandwidth, has become one of the high-speed data transmission interfaces used in current digital industrial cameras and hyperspectral cameras. Camera Link transmits signals based on the Low Voltage Differential Signaling (LVDS) format, achieving data rates of up to 2.38 Gbit / s. Furthermore, Camera Link's use of a combined serial and parallel transmission method for transmitting massive amounts of image data offers unique advantages in terms of interference resistance. To adapt to application scenarios with different resolutions and frame rates, the Camera Link standard defines multiple scalable configuration levels. Its standardized hierarchical structure and robust, reliable MDR-26 physical connector have made Camera Link a long-standing mainstream interface technology in industrial automation and high-precision inspection fields, thanks to its extremely low latency and stable transmission performance.

[0051] Compared to traditional RGB or multispectral images, hyperspectral data typically contains a large amount of band information, resulting in a massive data volume. Directly storing data from all bands would incur huge storage costs.

[0052] In related technologies, software computation is used to process hyperspectral data to reduce the burden of data storage and transmission. However, in the case of high-speed data streams, software computation often cannot meet real-time requirements. Furthermore, the computer systems in these technologies face dual compatibility limitations with Camera Link: 1. PCIe bus-based image acquisition cards have interface protocol incompatibility issues and cannot directly parse Camera Link's physical layer encoding; 2. The system architecture struggles to meet the real-time throughput requirements of hyperspectral data. Even with physical layer adaptation achieved through dedicated frame capture, x86 architecture computers still face storage bandwidth bottlenecks.

[0053] This embodiment provides a hardware and software co-acceleration system for dimensionality reduction of hyperspectral data based on FPGA. Figure 1 This is a schematic diagram of the hardware and software co-acceleration system for FPGA-based hyperspectral data dimensionality reduction according to an embodiment of the present invention, as shown below. Figure 1 As shown, the system includes a hardware platform and a host computer. The hardware platform is connected to the host computer and also to a hyperspectral camera.

[0054] The hardware platform is used to receive each frame of hyperspectral data sent by the hyperspectral camera, and to perform dimensionality reduction processing on each frame of hyperspectral data according to the band selection information. The necessary bands in each frame of hyperspectral data are retained, and the unnecessary bands are removed to obtain the dimensionality-reduced hyperspectral data of each frame. The dimensionality-reduced hyperspectral data of each frame is then sent to the host computer.

[0055] The host computer sends band selection information to the hardware platform, indicating which bands should be retained in each frame of hyperspectral data. This band selection information can be pre-set by the user according to the actual application scenario. The host computer also sends camera configuration information to the hardware platform, indicating the parameters for the hyperspectral camera's data acquisition, such as when to start acquisition, when to stop acquisition, the acquisition speed, and the number of pixels. The hardware platform further forwards this camera configuration information to the hyperspectral camera, enabling the camera to acquire each frame of hyperspectral data according to the configuration information.

[0056] In one optional implementation, the hardware platform is further configured to receive a low-speed mode switching command from a host computer, and send target frame hyperspectral data to the host computer. This target frame hyperspectral data provides a complete frame of hyperspectral data. The host computer is also configured to receive the target frame hyperspectral data, and to perform feature extraction on the target frame hyperspectral data using a band selection algorithm. This extracts representative spectral bands from the complete frame of hyperspectral data, reduces redundant information, improves computational efficiency, generates a set of optimized band indices as band selection information, and sends the band selection information to the hardware platform. The host computer is also configured to send a high-speed mode switching command to the hardware platform. Upon receiving the high-speed mode switching command, the hardware platform is further configured to receive each frame of hyperspectral data sent by the hyperspectral camera. In other words, this FPGA-based hyperspectral data dimensionality reduction hardware and software collaborative acceleration system includes both low-speed and high-speed modes. In low-speed mode, the hardware platform provides a complete frame of hyperspectral data, i.e., the target frame hyperspectral data, to the host computer. This target frame hyperspectral data can be the first frame of hyperspectral data sent by the hyperspectral camera to the hardware platform. The host computer uses a band selection algorithm to generate band extraction information corresponding to the target frame hyperspectral data, i.e., the band extraction information corresponding to the hyperspectral data output by the hyperspectral camera. In high-speed mode, each frame of hyperspectral data acquired by the hyperspectral camera is transmitted to the hardware platform in real time. The hardware platform performs dimensionality reduction processing on each frame of hyperspectral data in real time based on the band index in the band extraction information generated in low-speed mode, obtaining dimensionality-reduced hyperspectral data for each frame, and then transmits it to the host computer in real time, thus achieving real-time transmission of hyperspectral data. It should be noted that low-speed mode transmits all hyperspectral data, resulting in a large data volume and long transmission time, hence the relatively low speed. High-speed mode, on the other hand, transmits band-selected hyperspectral data, retaining only a small number of useful bands. Compared to low-speed mode, the data volume is greatly reduced, and the transmission time is significantly shortened, hence the relatively high speed.

[0057] Optionally, the band selection algorithm can employ algorithms in related technologies that possess the functions of feature extraction and band selection for hyperspectral data. The band selection algorithm can calculate the weights corresponding to each band in a complete frame of hyperspectral data based on the needs of the actual application scenario (e.g., plastic sorting or color comparison scenarios, where different scenarios require different band selections). It then sorts the bands according to their weights, placing higher-weighted bands first and lower-weighted bands last. Finally, based on actual needs, it selects the target number of bands with higher weights as the bands to be retained during dimensionality reduction.

[0058] In one alternative implementation, the system further includes a Camera Link interface through which the hyperspectral camera sends frames of hyperspectral data to the hardware platform.

[0059] In one alternative implementation, the system further includes a first UART (Universal Asynchronous Receiver Transmitter) serial port; the hardware platform forwards the camera configuration information to the hyperspectral camera through the first UART serial port.

[0060] In one alternative implementation, the camera configuration information includes at least one of camera frame rate, pixel cropping, camera gain, and camera model.

[0061] In one alternative implementation, the system also includes a UDP (User Datagram Protocol) Gigabit Ethernet interface; the hardware platform sends each frame of hyperspectral dimensionality reduction data to the host computer via the UDP Gigabit Ethernet interface.

[0062] In one alternative implementation, the system further includes a second UART serial port; the host computer sends band selection information to the hardware platform through the second UART serial port.

[0063] In one alternative implementation, the hardware platform is based on an FPGA (Field Programmable Gate Array). Through a pipelined architecture, the hardware platform breaks down complex processes into a series of simpler, sequential stages, each completing its task within one clock cycle. This simplifies the logic of each stage, allowing the entire system to operate at higher clock frequencies and pass results to the next stage. Simultaneously, the processing latency is a fixed N clock cycles, independent of data content or system load, which is crucial for real-time systems. For example, simultaneously receiving or processing eight pixels and correspondingly activating eight identical pipelines can increase the speed by eight times.

[0064] For example, Figure 2 This is a schematic diagram of data interaction according to an embodiment of the present invention.

[0065] In one alternative implementation, the hardware platform includes an image storage module for caching received frames of hyperspectral data.

[0066] In one alternative implementation, the hardware platform further includes an image receiving module for receiving frames of hyperspectral data transmitted by the hyperspectral camera.

[0067] In one optional implementation, the hardware platform further includes a command receiving module for receiving command information sent by a host computer. This command information includes band selection information, camera configuration information, low-speed mode switching instructions, and high-speed mode switching instructions. The command receiving module is also used to forward the received camera configuration information to the hyperspectral camera.

[0068] In one optional implementation, the hardware platform further includes a band selection module. The command receiving module is also used to forward the received band selection information to the band selection module. The band selection module is used to perform dimensionality reduction processing on each frame of hyperspectral data according to the band selection information to obtain each frame of hyperspectral dimensionality-reduced data.

[0069] In one optional implementation, the hardware platform further includes a UDP transmission module, and the band selection module is also used to send each frame of hyperspectral dimensionality reduction data to the UDP transmission module, which is used to send each frame of hyperspectral dimensionality reduction data to the host computer.

[0070] It should be noted that the FPGA-based hyperspectral data dimensionality reduction hardware and software co-acceleration system provided in this embodiment has a certain degree of configurability and scalability. It supports dynamic configuration of band selection in different scenarios, adapts to different types of hyperspectral cameras using the Camera Link protocol, and reserves resources for subsequent algorithm expansion. At the input end, the FPGA-based hyperspectral data dimensionality reduction hardware and software co-acceleration system provided in this embodiment utilizes the strict synchronization clock and frame validity signal of the Camera Link interface, combined with its anti-interference differential physical layer, to ensure accurate and complete capture of each frame of valid data from the source. At the output end, the CRC cyclic redundancy check mechanism of the Ethernet protocol ensures the bit-level accuracy of each data packet during transmission. Simultaneously, by embedding a custom sequence number in the data, the system logically solves the problems of out-of-order delivery and packet loss that may occur during network transmission. Therefore, the system meets the requirements of real-time performance, high throughput, and reasonable resource utilization, ensuring that the latency from data reception to processing output is within an acceptable range, supporting the actual output rate of the hyperspectral camera, and optimizing the use of FPGA hardware resources.

[0071] The FPGA-based hyperspectral data dimensionality reduction hardware and software collaborative acceleration system provided in this embodiment sends camera configuration information to the hyperspectral camera via a host computer, forwards the camera configuration information to the hyperspectral camera via a hardware platform, adjusts the data acquisition status of the hyperspectral camera, and sends band selection information to the hardware platform via the host computer. The hardware platform performs dimensionality reduction processing on each frame of hyperspectral data, reduces the data volume, and then transmits it to the host computer, improving data transmission efficiency. This enables collaborative work between the host computer, hardware platform, and hyperspectral camera, thereby achieving real-time transmission of hyperspectral data.

[0072] According to an embodiment of the present invention, a hardware and software collaborative acceleration method for dimensionality reduction of hyperspectral data based on FPGA is provided. It should be noted that the steps shown in the flowchart in the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions. Furthermore, although a logical order is shown in the flowchart, in some cases, the steps shown or described may be executed in a different order than that shown here.

[0073] This embodiment provides a hardware and software co-acceleration method for dimensionality reduction of hyperspectral data based on FPGA, which is... Figure 1 In the embodiment, the host computer in the FPGA-based hyperspectral data dimensionality reduction hardware and software collaborative acceleration system is executed. The host computer can be a laptop, desktop computer, server, etc. Figure 3 This is a flowchart illustrating a hardware and software collaborative acceleration method for FPGA-based hyperspectral data dimensionality reduction according to an embodiment of the present invention, as shown below. Figure 3 As shown, the process includes the following steps:

[0074] Step S301: Receive the target frame hyperspectral data sent by the hardware platform.

[0075] Before transmitting each frame of hyperspectral data acquired by the hyperspectral camera in real time, the host computer first needs to receive the single frame of hyperspectral data from the hyperspectral camera, i.e., the target frame hyperspectral data, through the hardware platform.

[0076] Step S302: Feature extraction is performed on the hyperspectral data of the target frame using a band selection algorithm to obtain band selection information.

[0077] The host computer extracts features from the target frame hyperspectral data using a band selection algorithm, extracts representative spectral bands from a single frame of hyperspectral data to generate a band index, and then obtains band selection information.

[0078] Step S303: Send the band selection information to the hardware platform so that the hardware platform can perform dimensionality reduction processing on each frame of hyperspectral data according to the band selection information.

[0079] After receiving the band selection information, the host computer sends it to the hardware platform, enabling real-time transmission of hyperspectral data. The hardware platform then performs dimensionality reduction processing on each frame of hyperspectral data based on the band index extracted from the band extraction information, obtaining dimensionality-reduced hyperspectral data for each frame, which is then transmitted back to the host computer in real time. Upon receiving the real-time transmitted dimensionality-reduced hyperspectral data, the host computer can further perform storage, visualization, or use it for subsequent hyperspectral data analysis tasks.

[0080] This embodiment provides a hardware and software co-acceleration method for dimensionality reduction of hyperspectral data based on FPGA, which is... Figure 1The hardware platform execution in the FPGA-based hyperspectral data dimensionality reduction hardware and software co-acceleration system in the embodiment. Figure 4 This is a flowchart illustrating another FPGA-based hardware and software co-acceleration method for hyperspectral data dimensionality reduction according to an embodiment of the present invention, as shown below. Figure 4 As shown, the process includes the following steps:

[0081] Step S401: Receive the target frame hyperspectral data sent by the hyperspectral camera.

[0082] Before transmitting each frame of hyperspectral data acquired by the hyperspectral camera in real time, the hardware platform needs to receive a single frame of hyperspectral data from the hyperspectral camera, i.e., the target frame hyperspectral data.

[0083] Step S402: Receive the low-speed mode switching command sent by the host computer, and send the target frame hyperspectral data to the host computer so that the host computer can extract features from the target frame hyperspectral data using a band selection algorithm to obtain band selection information.

[0084] When the hardware platform receives a low-speed mode switching command from the host computer, it switches to low-speed mode and sends the target frame hyperspectral data to the host computer. The host computer uses a band selection algorithm to extract features from the target frame hyperspectral data, extracts representative spectral bands from a single frame of hyperspectral data to generate a band index, and then obtains band selection information to send to the hardware platform.

[0085] Step S403: Receive band selection information sent by the host computer.

[0086] The hardware platform receives band selection information sent by the host computer for subsequent real-time transmission.

[0087] Step S404: After receiving the high-speed mode switching command sent by the host computer, receive each frame of hyperspectral data sent by the hyperspectral camera, and perform dimensionality reduction processing on each frame of hyperspectral data according to the band selection information to obtain each frame of hyperspectral dimensionality reduction data and send it to the host computer.

[0088] After receiving the high-speed mode switching command sent by the host computer, the hardware platform switches to high-speed mode, receives each frame of hyperspectral data sent by the hyperspectral camera in real time, and performs dimensionality reduction processing on each frame of hyperspectral data in real time according to the band selection information to obtain each frame of hyperspectral dimensionality reduction data, and sends each frame of hyperspectral dimensionality reduction data to the host computer.

[0089] As one or more specific application embodiments of the present invention, the optimal implementation scheme or the scheme that the inventors most want to embody is described below in conjunction with specific application scenarios.

[0090] Figure 5This is a flowchart illustrating another FPGA-based hardware and software collaborative acceleration method for hyperspectral data dimensionality reduction according to an embodiment of the present invention. First, the hardware platform receives a complete frame of hyperspectral data output from the hyperspectral camera via the Camera Link interface and caches it in on-chip memory or external DDR (Double Data Rate) memory to ensure data integrity and stability. Subsequently, the hardware platform transmits the complete frame of hyperspectral data to the host computer at low speed via UDP. After receiving the first complete frame of hyperspectral data, the host computer analyzes the data using a band selection algorithm, extracts representative spectral bands, generates a set of optimized band indices, and feeds them back to the hardware platform via UART serial communication, providing a basis for subsequent real-time band selection. After receiving the band selection index command, the hyperspectral camera continuously outputs a hyperspectral data stream. The hardware platform receives the data via the Camera Link interface and filters the data according to the band indices provided by the host computer. The hardware platform extracts only the data of the selected bands and transmits it to the host computer at high speed via UDP, thereby reducing data transmission bandwidth requirements and improving the overall throughput and processing efficiency of the system. After receiving the filtered hyperspectral data, the host computer can further perform storage, visualization, or use it for subsequent hyperspectral analysis tasks.

[0091] Using the aforementioned FPGA-based hardware and software collaborative acceleration method for hyperspectral data dimensionality reduction, the original hyperspectral data is transmitted to the hardware platform via the Camera Link interface. The hardware platform executes a band selection algorithm, retaining only 50 key bands, and transmits the filtered data to the host computer via Gigabit Ethernet. The host computer receives and parses all the filtered band data, ultimately reducing the number of bands in the hyperspectral data cube (a three-dimensional data structure for storing hyperspectral data) from 256 to 50. Testing the data processing speed of each hardware module showed a latency of approximately 2ms, meeting the real-time requirements of hyperspectral data processing. Furthermore, during an 8-hour continuous test, the system consistently and stably output the filtered hyperspectral data without any abnormal interruptions. Data transmission integrity was excellent, with no data loss or sequence errors detected.

[0092] This embodiment also provides a hardware and software collaborative acceleration device for hyperspectral data dimensionality reduction based on FPGA. This device is used to implement the above embodiments and preferred embodiments, and details already described will not be repeated. As used below, the term "module" can be a combination of software and / or hardware that implements a predetermined function. Although the device described in the following embodiments is preferably implemented in software, hardware implementation, or a combination of software and hardware, is also possible and contemplated.

[0093] This embodiment provides a hardware and software co-acceleration device for FPGA-based hyperspectral data dimensionality reduction, applied to... Figure 1 The host computer in the FPGA-based hyperspectral data dimensionality reduction hardware and software collaborative acceleration system in the embodiment. Figure 6 This is a structural block diagram of a hardware and software co-acceleration device for FPGA-based hyperspectral data dimensionality reduction according to an embodiment of the present invention, as shown below. Figure 6 As shown, the device includes:

[0094] The first receiving module 601 is used to receive target frame hyperspectral data sent by the hardware platform;

[0095] The band extraction module 602 is used to extract features from the hyperspectral data of the target frame using a band selection algorithm to obtain band selection information.

[0096] The first transmitting module 603 is used to send the band selection information to the hardware platform so that the hardware platform can perform dimensionality reduction processing on each frame of hyperspectral data according to the band selection information.

[0097] This embodiment provides a hardware and software co-acceleration device for FPGA-based hyperspectral data dimensionality reduction, applied to... Figure 1 The hardware platform in the FPGA-based hyperspectral data dimensionality reduction hardware and software co-acceleration system in the embodiment. Figure 7 This is a structural block diagram of another FPGA-based hardware and software co-acceleration device for hyperspectral data dimensionality reduction according to an embodiment of the present invention, such as... Figure 7 As shown, the device includes:

[0098] The second receiving module 701 is used to receive target frame hyperspectral data sent by the hyperspectral camera;

[0099] The second sending module 702 is used to receive the low-speed mode switching command sent by the host computer and send the target frame hyperspectral data to the host computer so that the host computer can extract features from the target frame hyperspectral data through the band selection algorithm to obtain band selection information.

[0100] The third receiving module 703 is used to receive band selection information sent by the host computer;

[0101] The dimensionality reduction module 704 is used to receive each frame of hyperspectral data sent by the hyperspectral camera after receiving the high-speed mode switching command sent by the host computer, and to perform dimensionality reduction processing on each frame of hyperspectral data according to the band selection information, so as to obtain each frame of hyperspectral dimensionality reduction data and send it to the host computer.

[0102] Further functional descriptions of the above modules and units are the same as those in the corresponding embodiments described above, and will not be repeated here.

[0103] In this embodiment, the FPGA-based hyperspectral data dimensionality reduction hardware and software co-acceleration device is presented in the form of functional units. Here, a unit refers to an ASIC (Application Specific Integrated Circuit) circuit, a processor and memory that execute one or more software or fixed programs, and / or other devices that can provide the above functions.

[0104] This invention also provides a computer device having the above-described features. Figure 6 or Figure 7 The illustrated device is a hardware and software co-acceleration device for dimensionality reduction of hyperspectral data based on FPGA.

[0105] Please see Figure 8 , Figure 8 This is a schematic diagram of the structure of a computer device provided in an optional embodiment of the present invention, such as... Figure 8 As shown, the computer device includes one or more processors 10, memory 20, and interfaces for connecting the components, including high-speed interfaces and low-speed interfaces. The components communicate with each other via different buses and can be mounted on a common motherboard or otherwise installed as needed. The processors can process instructions executed within the computer device, including instructions stored in or on memory to display graphical information of a GUI on external input / output devices (such as display devices coupled to the interfaces). In some alternative implementations, multiple processors and / or multiple buses can be used with multiple memories and multiple memory modules, if desired. Similarly, multiple computer devices can be connected, each providing some of the necessary operations (e.g., as a server array, a group of blade servers, or a multiprocessor system). Figure 8 Take a processor 10 as an example.

[0106] Processor 10 may be a central processing unit, a network processor, or a combination thereof. Processor 10 may further include a hardware chip. The hardware chip may be an application-specific integrated circuit (ASIC), a programmable logic device (PLD), or a combination thereof. The programmable logic device may be a complex programmable logic device (CAMP), a field-programmable gate array (FPGA), a general-purpose array logic (GDA), or any combination thereof.

[0107] The memory 20 stores instructions executable by at least one processor 10 to cause the at least one processor 10 to perform the method shown in the above embodiments.

[0108] The memory 20 may include a program storage area and a data storage area. The program storage area may store the operating system and applications required for at least one function; the data storage area may store data created based on the use of the computer device. Furthermore, the memory 20 may include high-speed random access memory and may also include non-transitory memory, such as at least one disk storage device, flash memory device, or other non-transitory solid-state storage device. In some alternative embodiments, the memory 20 may optionally include memory remotely located relative to the processor 10, and these remote memories may be connected to the computer device via a network. Examples of such networks include, but are not limited to, the Internet, intranets, local area networks, mobile communication networks, and combinations thereof.

[0109] The memory 20 may include volatile memory, such as random access memory; the memory may also include non-volatile memory, such as flash memory, hard disk or solid-state drive; the memory 20 may also include a combination of the above types of memory.

[0110] The computer device also includes an input device 30 and an output device 40. The processor 10, memory 20, input device 30, and output device 40 can be connected via a bus or other means. Figure 8 Taking the example of a connection between China and Israel via a bus.

[0111] Input device 30 can receive input numerical or character information, and generate key signal inputs related to user settings and function control of the computer device, such as a touchscreen, keypad, mouse, trackpad, touchpad, joystick, one or more mouse buttons, trackball, joystick, etc. Output device 40 may include display devices, auxiliary lighting devices (e.g., LEDs), and haptic feedback devices (e.g., vibration motors). The aforementioned display devices include, but are not limited to, liquid crystal displays, light-emitting diodes, displays, and plasma displays. In some alternative embodiments, the display device may be a touchscreen.

[0112] This invention also provides a computer-readable storage medium. The methods described above according to embodiments of the invention can be implemented in hardware or firmware, or implemented as computer code that can be recorded on a storage medium, or implemented as computer code downloaded via a network and originally stored on a remote storage medium or a non-transitory machine-readable storage medium and then stored on a local storage medium. Thus, the methods described herein can be processed by software stored on a storage medium using a general-purpose computer, a dedicated processor, or programmable or dedicated hardware. The storage medium can be a magnetic disk, optical disk, read-only memory, random access memory, flash memory, hard disk, or solid-state drive, etc.; further, the storage medium can also include combinations of the above types of memory. It is understood that computers, processors, microprocessor controllers, or programmable hardware include storage components capable of storing or receiving software or computer code, which, when accessed and executed by the computer, processor, or hardware, implements the methods shown in the above embodiments.

[0113] A portion of this invention can be applied as a computer program product, such as computer program instructions, which, when executed by a computer, can invoke or provide the methods and / or technical solutions according to the invention through the operation of the computer. Those skilled in the art will understand that the forms in which computer program instructions exist in a computer-readable medium include, but are not limited to, source files, executable files, installation package files, etc. Correspondingly, the ways in which computer program instructions are executed by a computer include, but are not limited to: the computer directly executing the instructions, or the computer compiling the instructions and then executing the corresponding compiled program, or the computer reading and executing the instructions, or the computer reading and installing the instructions and then executing the corresponding installed program. Here, the computer-readable medium can be any available computer-readable storage medium or communication medium accessible to a computer.

[0114] Although embodiments of the present invention have been described in conjunction with the accompanying drawings, those skilled in the art can make various modifications and variations without departing from the spirit and scope of the present invention, and all such modifications and variations fall within the protection scope of the present invention.

Claims

1. A hardware and software co-acceleration system for dimensionality reduction of hyperspectral data based on FPGA, characterized in that, The system includes: The hardware platform is used to receive each frame of hyperspectral data sent by the hyperspectral camera, and to perform dimensionality reduction processing on each frame of hyperspectral data according to the band selection information, so as to obtain each frame of hyperspectral dimensionality reduction data and send it to the host computer. The host computer is used to send band selection information to the hardware platform; the host computer is also used to send camera configuration information to the hardware platform; the hardware platform is also used to forward the camera configuration information to the hyperspectral camera, so that the hyperspectral camera can collect each frame of hyperspectral data according to the camera configuration information. The hardware platform is also used to receive low-speed mode switching instructions sent by the host computer and send target frame hyperspectral data to the host computer. The host computer is also used to receive the target frame hyperspectral data; the host computer is also used to extract features from the target frame hyperspectral data through a band selection algorithm to obtain band selection information, and send the band selection information to the hardware platform; The host computer is also used to send a high-speed mode switching command to the hardware platform; the hardware platform is also used to receive each frame of hyperspectral data sent by the hyperspectral camera after receiving the high-speed mode switching command.

2. The system according to claim 1, characterized in that, The system also includes a Camera Link interface; the hyperspectral camera sends each frame of hyperspectral data to the hardware platform through the Camera Link interface.

3. The system according to claim 2, characterized in that, The system also includes a first UART serial port; the hardware platform forwards the camera configuration information to the hyperspectral camera through the first UART serial port.

4. The system according to claim 3, characterized in that, The camera configuration information includes at least one of the following: camera frame rate, pixel cropping, camera gain, and camera model.

5. The system according to claim 1, characterized in that, The system also includes a UDP gigabit Ethernet interface; the hardware platform sends each frame of hyperspectral dimensionality reduction data to the host computer through the UDP gigabit Ethernet interface.

6. The system according to claim 5, characterized in that, The system also includes a second UART serial port; the host computer sends band selection information to the hardware platform through the second UART serial port.

7. The system according to claim 1, characterized in that, The hardware platform includes an image storage module; the image storage module is used to cache each frame of received hyperspectral data.

8. A hardware and software collaborative acceleration method for dimensionality reduction of hyperspectral data based on FPGA, characterized in that, The method is executed by the host computer in the FPGA-based hyperspectral data dimensionality reduction hardware and software collaborative acceleration system according to any one of claims 1-7; the method includes: Receive target frame hyperspectral data sent by the hardware platform; The band selection information is obtained by extracting features from the hyperspectral data of the target frame using a band selection algorithm. The band selection information is sent to the hardware platform so that the hardware platform can perform dimensionality reduction processing on each frame of hyperspectral data according to the band selection information.

9. A hardware and software collaborative acceleration method for dimensionality reduction of hyperspectral data based on FPGA, characterized in that, The method is executed by the hardware platform in the FPGA-based hyperspectral data dimensionality reduction hardware and software co-acceleration system according to any one of claims 1-7; the method includes: Receive target frame hyperspectral data sent by a hyperspectral camera; The system receives a low-speed mode switching command from the host computer and sends target frame hyperspectral data to the host computer so that the host computer can extract features from the target frame hyperspectral data using a band selection algorithm to obtain band selection information. Receive band selection information sent by the host computer; After receiving the high-speed mode switching command sent by the host computer, the system receives each frame of hyperspectral data sent by the hyperspectral camera, and performs dimensionality reduction processing on each frame of hyperspectral data according to the band selection information, and sends the dimensionality-reduced hyperspectral data of each frame to the host computer.

Images