Image acquisition method of flight control device and flight control device
By adopting the PL and PS architecture of the FMQL processor in the flight control device and using status signal values to control the image data storage mode, the problems of cost and size limitations are solved, and long-term image data storage and transmission are realized.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- 贵州航天控制技术有限公司
- Filing Date
- 2026-04-27
- Publication Date
- 2026-07-14
AI Technical Summary
Existing flight control devices are not designed to achieve long-term image data storage while reducing costs, due to limitations such as high integration, power consumption, and size.
The system adopts the PL and PS architecture of the FMQL processor, controls the switching of the working mode of the camera IP core and PS through status signal values, stores image data in the DDR storage module and finally writes it to the data logger, and combines asynchronous FIFO and AXI top-level protocol modules to realize the storage and transmission of image data.
This technology enables long-term storage of image data while reducing the cost of flight control devices, thereby reducing device size and increasing data capacity.
Smart Images
Figure CN122120400B_ABST
Abstract
Description
Technical Field
[0001] This application relates to the field of flight control device technology, and in particular to an image acquisition method and a flight control device. Background Technology
[0002] The flight control unit is the core of the UAV's control system. It consists of BeiDou navigation system, inertial navigation system, data logger, atmospheric measurement system, and environmental measurement system (sensor devices + camera). The camera is responsible for capturing and recording images of the UAV at key locations during its high-altitude flight, from release to wing deployment and cruise. Due to the high integration of the flight control unit and limitations in power consumption, cost, and size, existing design solutions are all independent modules, which cannot meet the system design requirements. Summary of the Invention
[0003] In view of this, one of the technical problems solved by the embodiments of this application is to provide an image acquisition method and a flight control device, which solves the problem of achieving long-term data storage while reducing the cost of the flight control device.
[0004] According to a first aspect of the embodiments of this application, an image acquisition method for a flight control device is provided. The flight control device is equipped with a camera module and includes an FMQL processor and a data logger. The FMQL processor includes a PL terminal and a PS terminal. The PL terminal creates a camera IP core. The method includes:
[0005] When a control command for the camera module is detected, the command type is determined;
[0006] If the instruction type is a camera instruction, then the corresponding processing is performed based on the status signal value of the PS end and the status signal value of the camera IP core, so that after the camera IP core writes the image data collected by the camera module into the DDR storage module of the FMQL processor, the PS end writes the image data in the DDR storage module into the data recorder.
[0007] Furthermore, based on the status signal values from the PS end and the camera IP core, corresponding processing is performed, including:
[0008] Based on the operation of writing the image data collected by the camera module into the DDR storage module of the FMQL processor, the status signal value of the camera IP core is determined;
[0009] When the status signal value of the camera IP core is the fifth value, the image storage operation from the DDR storage module to the data logger is executed on the PS end, and the status signal value of the PS end is adjusted according to the storage operation.
[0010] Furthermore, the method also includes:
[0011] When the status signal value detected at the PS end is the fourth value, the process continues to execute the step of determining the command type when a control command for the camera module is detected.
[0012] Furthermore, based on the operation of writing image data acquired by the camera module into the DDR memory module of the FMQL processor using the camera IP core, the status signal value of the camera IP core is determined, including:
[0013] When the camera IP core is initialized, the status signal value of the camera IP core is set to the sixth value;
[0014] When the camera IP core initialization is detected to be complete, the image data collected by the camera module is obtained, and the status signal value of the camera IP core is set to the seventh value;
[0015] When the image data has been written to the DDR storage module, the status signal value of the camera IP core is set to the fifth value.
[0016] Furthermore, before performing a storage operation on the image data acquired by the camera module based on the camera IP core and determining the status signal value of the camera IP core, the method includes: when the camera software on the PS side is initializing, setting the status signal value of the PS side to a first value, and setting the status signal value of the PS side to a second value when initialization is complete; determining whether the status signal value of the camera IP core is a third value; performing a storage operation on the image data acquired by the camera module based on the camera IP core and determining the status signal value of the camera IP core includes: if the status signal value of the camera IP core is a third value, performing a storage operation on the image data acquired by the camera module based on the camera IP core and determining the status signal value of the camera IP core.
[0017] Furthermore, the method also includes:
[0018] If the instruction type is "download image data instruction", then the image data is read from the data logger and uploaded to the host computer.
[0019] A second aspect of this application discloses a flight control device, the device comprising:
[0020] The camera module is used to acquire image data;
[0021] The flight control module includes an FMQL processor and a data logger. The FMQL processor includes a PL terminal and a PS terminal. The PL terminal includes a camera IP core. The PS terminal is used to write the image data acquired by the camera IP core to the DDR storage module of the FMQL processor and then write the image data from the DDR storage module to the data logger.
[0022] Furthermore, the camera IP core includes a write DDR control module, an asynchronous FIFO module, and an AXI top-level protocol module. The asynchronous FIFO module is used to acquire image data captured by the camera module, and the write DDR control module is used to read image data from the asynchronous FIFO module and write the image data to the DDR storage module based on the AXI top-level protocol module.
[0023] This application embodiment controls the switching of the working mode of the PS terminal and the camera IP core by querying the status signal value, and stores the image data to the data logger, thereby achieving the purpose of long-term data storage while reducing the cost of the flight control device. Attached Figure Description
[0024] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application.
[0025] Figure 1 A flowchart illustrating an image acquisition method for a flight control device provided in one embodiment of this application;
[0026] Figure 2 This is a schematic diagram of a flight control device architecture provided in one embodiment of this application;
[0027] Figure 3 A schematic diagram illustrating the workflow of a flight control device architecture provided in one embodiment of this application;
[0028] Figure 4 This is a block diagram of a flight control device provided in one embodiment of this application;
[0029] Figure 5 This is a schematic diagram of the structure of the camera IP core of a flight control device provided in one embodiment of this application. Detailed Implementation
[0030] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.
[0031] It should be noted that although functional modules are divided in the device schematic diagram and the logical order is shown in the flowchart, in some cases, the steps shown or described may be performed in a different order than the module division in the device or the order in the flowchart.
[0032] First, let's introduce and explain several terms used in this application:
[0033] In this embodiment of the application, the FMQL processor refers to the FMQL20S400M heterogeneous multi-core SoC processor, which adopts a quad-core ARM Cortex-A7 (PS side) and FPGA programmable logic resources (PL side) architecture.
[0034] In this embodiment of the application, PL (Programmable Logic) refers to programmable logic.
[0035] In this embodiment of the application, FPGA (Field Programmable Gate Array) refers to a field-programmable gate array.
[0036] In this embodiment, PS (Processing System) refers to the processing system.
[0037] In this application embodiment, IP (Intellectual Property) refers to intellectual property rights, and IP cores represent verified and reusable circuit function modules.
[0038] To make the objectives, technical solutions, and advantages of this application clearer, the embodiments of this application will be described in further detail below with reference to the accompanying drawings.
[0039] According to one embodiment of this application, an image acquisition method for a flight control device is provided. The flight control device is equipped with a camera module and includes an FMQL processor and a data logger. The FMQL processor includes a PL terminal and a PS terminal. The PL terminal creates a camera IP core, and the PS terminal is connected to a DDR storage module. Figure 1 As shown, the method includes:
[0040] Step S101: When a control command for the camera module is detected, determine the command type.
[0041] It should be noted that the command type refers to the information used to control the flight control device to acquire and process images. Specifically, this includes camera commands, stop camera commands, and image data download commands.
[0042] Step S102: If the instruction type is a camera instruction, then the corresponding processing is performed based on the status signal value of the PS terminal and the status signal value of the camera IP core. After the camera IP core writes the image data collected by the camera module into the DDR storage module of the FMQL processor, the PS terminal writes the image data in the DDR storage module into the data recorder.
[0043] It should be noted that this embodiment has a pre-set status signal value table, which includes the working instructions of the PS end and the camera IP core and the truth values and meanings of the corresponding status signals. Thus, after receiving the control instruction, the working mode switching of the PS end and the camera IP core can be controlled by looking up the table.
[0044] This application embodiment controls the switching of the working mode of the PS terminal and the camera IP core by querying the status signal value, and stores the image data to the data logger, thereby achieving the purpose of long-term data storage while reducing the cost of the flight control device.
[0045] In some embodiments, step S102 further includes:
[0046] Based on the camera IP core, perform storage operations on the image data collected by the camera module and determine the status signal value of the camera IP core;
[0047] When the status signal value of the camera IP core is the fifth value, the image storage operation from the DDR storage module to the data logger is executed on the PS end, and the status signal value of the PS end is adjusted according to the storage operation.
[0048] In this embodiment, the status signal value of the camera IP core is the fifth value, which is used to indicate that the image data has been written to the DDR storage module. The camera software on the PS side can read the image data and write it to the data recorder.
[0049] In this embodiment, the PL end is an FPGA, meaning the camera IP core is located on the FPGA. The digital video transmission parallel interface used by the OV5640 camera module in this embodiment is the DVP interface. The camera IP core can match the timing requirements of the low-speed parallel interface (DVP interface) and can achieve zero-error acquisition. At the same time, due to the parallel pipeline of the FPGA, it will not occupy the CPU cycle of the PS end.
[0050] The image storage process in this embodiment is as follows: After the camera IP core completes the writing of image data to the DDR storage module, the PS terminal performs the operation of writing to the data logger. This process is determined by detecting the status signal value of the camera IP core.
[0051] In some embodiments, the method further includes:
[0052] When the status signal value detected at the PS end is the fourth value, the process continues to execute the step of determining the command type when a control command for the camera module is detected.
[0053] In this embodiment, the status signal value of the PS terminal is the fourth value, which is used to indicate that writing to the SD card is complete and waiting for the camera IP core to write the next frame of image data.
[0054] In some embodiments, a storage operation is performed on the image data acquired by the camera module based on the camera IP core to determine the status signal value of the camera IP core, including:
[0055] When the camera IP core is initialized, the status signal value of the camera IP core is set to the sixth value;
[0056] When the camera IP core initialization is detected to be complete, the image data collected by the camera module is obtained, and the status signal value of the camera IP core is set to the seventh value;
[0057] When the image data has been written to the DDR storage module, the status signal value of the camera IP core is set to the fifth value.
[0058] In this embodiment, the sixth value is used to indicate that the camera IP core is being initialized.
[0059] In this embodiment, the seventh value is used to indicate that the camera IP core initialization is complete and image data is being written into the DDR.
[0060] In this embodiment, the third value is used to indicate that the PS-side camera software has finished reading DDR data and is writing the data to the SD card. At this time, the camera IP core cannot write data into DDR.
[0061] In some embodiments, before performing a storage operation on the image data acquired by the camera module based on the camera IP core, and before determining the status signal value of the camera IP core, the method includes:
[0062] When the camera software on the PS side is initializing, set the status signal value on the PS side to the first value, and when the initialization is complete, set the status signal value on the PS side to the second value.
[0063] Determine if the status signal value of the camera IP core is a third value;
[0064] Based on the camera IP core, the image data acquired by the camera module is stored, and the status signal values of the camera IP core are determined, including:
[0065] If the status signal value of the camera IP core is the third value, perform a storage operation on the image data collected by the camera module based on the camera IP core, and determine the status signal value of the camera IP core.
[0066] Specifically, the status signal value table, as shown in Table 1, includes working instructions and their corresponding truth values and meanings.
[0067] Table 1: Comparison Table of Work Instruction Meanings
[0068]
[0069] In this embodiment, the working mode flow is switched by querying the working instructions and their truth values in Table 1. The working state of the camera IP core is represented by ip_work_state, and the PS end switches the working mode by reading its value; the working state of the PS end is represented by cpu_work_state, and the camera IP core controls the timing of image data writing according to its value.
[0070] In some embodiments, the method further includes:
[0071] If the instruction type is "download image data instruction", then the image data is read from the data logger and uploaded to the host computer.
[0072] This embodiment achieves interaction between the flight control device and the host computer by downloading image data commands, thereby downloading the image data stored in the data recorder to the host computer for processing and analysis.
[0073] Specifically, the host computer communicates with the PS terminal via a serial interface.
[0074] To further illustrate the method of this embodiment, the following description is provided. Figure 2 and Figure 3 Please provide a detailed explanation.
[0075] First, the flight control device in this embodiment is implemented using FMQL, as follows: Figure 2 The diagram shows the PS (Power Switch) and PL (Power Processor) terminals. The PS terminal runs control and measurement software and camera software. The control and measurement software communicates with the ground-based host computer, receiving commands from the host computer to start recording, stop recording, and download image data. The camera software controls the camera and manages data storage. During image acquisition, the camera is first initialized. Then, it receives commands from the control and measurement software via inter-core shared content and determines whether the command is a camera start command. If the command is a camera start command, and the camera software completes initialization, `cpu_work_state` is set to 2. At this point, the camera software waits for the camera IP core on the PL terminal to write image data. Next, when `ip_work_state` is 3, indicating that the camera IP core has completed writing image data to DDR, the camera software reads the image data from DDR and writes it to the SD card (i.e., the data logger). During this process, `cpu_work_state` is set to 3, and after writing to the SD card, `cpu_work_state` is set to 4. The PS terminal continues to execute the steps of receiving commands from the control and measurement software via inter-core shared content. Additionally, if the instruction is not a camera start instruction, it determines whether it is a camera stop instruction; otherwise, it determines whether it is an image data download instruction. If the instruction is an image data download instruction, it reads the image data from the SD card and downloads it to the host computer.
[0076] AXI_LITE_SLAVE indicates a slave interface that implements the AXI protocol. It is a common bus interface name in FPGA design, used for lightweight control and status register access. EMIO_0:OV_SCL is the clock line of the PS section I2C (Inter-Integrated Circuit) controller, and EMIO_0:OV_SDA is the data line of the PS section I2C controller.
[0077] One embodiment of this application provides a flight control device, such as... Figure 4 As shown, the device 100 includes a camera module 10 and a flight control module 20.
[0078] The camera module is used to acquire image data. In this embodiment, the camera module refers to a module with image sensing and data transmission capabilities. Compared to standalone camera products, it does not include a processor or storage, thus reducing size and cost while still providing image acquisition functionality. This embodiment uses the OV5640 camera module.
[0079] The flight control module includes an FMQL processor and a data logger. The FMQL processor comprises a PL (Programmable Array) terminal and a PS (Power Seal) terminal. The PL terminal includes a camera IP core, and the PS terminal is used to write the image data acquired by the camera IP core to the DDR storage module of the FMQL processor, and then write the image data from the DDR storage module to the data logger. The data logger is used to store the image data; specifically, the data logger includes an SD card. In application, to prevent the SD card from becoming loose or making poor contact in high-G and high-vibration environments during flight, the SD card can be soldered to the mounting plate of the data logger. The flight control module in this embodiment uses the JFMQL20S400 processor chip, which leverages the advantages of its 4-core ARM CPU + PL (FPGA) on-chip SOC architecture. By constructing a camera IP core in the FPGA section, it realizes the timing of image data acquisition and writing image data to DDR. The camera software running in the ARM section writes the data to the SD card mounted on the PS (included in the data logger). At the same time, the measurement and control software runs in another CPU in the ARM section to realize communication with the host computer to start and stop the camera, download image data, etc., and synchronously transmits instructions to the camera software through shared memory.
[0080] This embodiment expands the data capacity by setting up a data logger, enabling the flight control device to acquire images over a long period of time. At the same time, since the image acquisition is controlled by the PL terminal, only a low-cost image acquisition circuit is needed, which reduces the size of the flight control device and lowers the cost.
[0081] In some embodiments, such asFigure 5 As shown, the camera IP core includes a DDR write control module, an asynchronous FIFO module, and an AXI top-level protocol module. The asynchronous FIFO module is used to acquire image data captured by the camera module, and the DDR write control module is used to read image data from the asynchronous FIFO module and write the image data to the DDR storage module based on the AXI top-level protocol module. In this embodiment, the camera IP core is mainly used to complete image acquisition by the OV5640 camera module, cache the data in the asynchronous FIFO, and interact with the camera software, i.e., to complete operations such as writing to DDR and writing completion through the ip_work_state and cpu_work_state values.
[0082] AXI_DATA represents the data path, used for high-speed data stream transmission. AXI_CTL represents the control path, responsible for configuration and control, such as reading and writing registers within the module and setting parameters. ov_plck is a clock signal output by the camera, and ov_data is a multi-bit wide data bus used to transmit pixel values in the camera interface. READ_EN represents the FIFO read enable signal, used to enable the FIFO to output data, and FIFO_DATA represents the data output by the FIFO.
[0083] The flight control device in this embodiment can execute the image acquisition method of the flight control device shown in the embodiment of this application. The implementation principle is similar and will not be described again here.
[0084] The above is a detailed description of the preferred embodiments of this application. However, this application is not limited to the above embodiments. Those skilled in the art can make various equivalent modifications or substitutions without departing from the spirit of this application. All such equivalent modifications or substitutions are included within the scope defined by the claims of this application.
Claims
1. An image acquisition method for a flight control device, characterized in that, The flight control device is equipped with a camera module and includes an FMQL processor and a data logger. The FMQL processor includes a PL terminal and a PS terminal. The PL terminal creates a camera IP core. The method includes: When a control command for the camera module is detected, the command type is determined; If the instruction type is a camera instruction, then corresponding processing is performed based on the status signal value of the PS terminal and the status signal value of the camera IP core, so that after the camera IP core writes the image data collected by the camera module into the DDR storage module of the FMQL processor, the PS terminal writes the image data in the DDR storage module into the data recorder. The corresponding processing based on the status signal value of the PS terminal and the status signal value of the camera IP core includes: Based on the operation of writing the image data collected by the camera module into the DDR storage module of the FMQL processor based on the camera IP core, the status signal value of the camera IP core is determined; When the status signal value of the camera IP core is the fifth value, the image storage operation from the DDR storage module to the data recorder is executed based on the PS terminal, and the status signal value of the PS terminal is adjusted according to the storage operation; Before determining the status signal value of the camera IP core, the method includes the following steps: When the camera software on the PS terminal is initializing, the status signal value of the PS terminal is set to a first value, and when the initialization is complete, the status signal value of the PS terminal is set to a second value. Determine whether the status signal value of the camera IP core is a third value; The operation of writing the image data acquired by the camera module into the DDR storage module of the FMQL processor based on the camera IP core, and determining the status signal value of the camera IP core, includes: If the status signal value of the camera IP core is a third value, the operation of writing the image data collected by the camera module into the DDR storage module of the FMQL processor based on the camera IP core is executed to determine the status signal value of the camera IP core.
2. The image acquisition method for the flight control device according to claim 1, characterized in that, The method further includes: When the status signal value of the PS terminal is detected to be the fourth value, the step of determining the instruction type when the control instruction for the camera module is detected continues to be executed.
3. The image acquisition method for the flight control device according to claim 1, characterized in that, The operation of writing the image data acquired by the camera module into the DDR storage module of the FMQL processor based on the camera IP core, and determining the status signal value of the camera IP core, includes: When the camera IP core is initialized, the status signal value of the camera IP core is set to the sixth value; When the camera IP core initialization is detected to be complete, the image data collected by the camera module is acquired, and the status signal value of the camera IP core is set to the seventh value; When the image data is written to the DDR storage module, the status signal value of the camera IP core is set to the fifth value.
4. The image acquisition method for the flight control device according to claim 1, characterized in that, The method further includes: If the instruction type is a download image data instruction, then the image data is read from the data logger and uploaded to the host computer.
5. A flight control device, characterized in that, include: The camera module is used to acquire image data; The flight control module includes an FMQL processor and a data logger. The FMQL processor includes a PL terminal and a PS terminal. The PL terminal includes a camera IP core. The PS terminal is used to write the image data acquired by the camera IP core into the DDR storage module of the FMQL processor and then write the image data from the DDR storage module into the data logger. The flight control module is specifically used to write the image data collected by the camera module into the DDR storage module of the FMQL processor based on the camera IP core, and to determine the status signal value of the camera IP core; when the status signal value of the camera IP core is the fifth value, the PS terminal performs the image storage operation from the DDR storage module to the data logger, and adjusts the status signal value of the PS terminal according to the storage operation; The flight control module is specifically used to set the status signal value of the PS terminal to a first value when the camera software on the PS terminal is initializing, and to set the status signal value of the PS terminal to a second value when the initialization is complete; to determine whether the status signal value of the camera IP core is a third value; if the status signal value of the camera IP core is a third value, to perform the operation of writing the image data collected by the camera module based on the camera IP core into the DDR storage module of the FMQL processor, and to determine the status signal value of the camera IP core.
6. The flight control device according to claim 5, characterized in that, The camera IP core includes a write DDR control module, an asynchronous FIFO module, and an AXI top-level protocol module. The asynchronous FIFO module is used to acquire image data collected by the camera module. The write DDR control module is used to read the image data from the asynchronous FIFO module and write the image data to the DDR storage module based on the AXI top-level protocol module.
7. The flight control device according to claim 5, characterized in that, The PL terminal is an FPGA, and the PS terminal is an ARM.