A high-resolution image detection storage method and system

Abstract

The application provides a high-resolution image detection storage method and system, and belongs to the technical field of image processing. The method comprises the following steps: S1: an image acquisition processing module receives image data and stores the image data in a plurality of logical channels in the image acquisition processing module; the image acquisition processing module pre-processes the image data; the image acquisition processing module calculates the data amount cached in each logical channel in real time, and sends a target data transfer notification to an intelligent scheduling module when the data amount in a target logical channel is not less than a preset value; S2: an AI calculation module calculates the resource utilization rate of each NPU calculation unit in the AI calculation module in real time; S3: the intelligent scheduling module notifies a target NPU calculation unit with a preset idle rate according to the resource utilization rate of each NPU; and S4: after receiving the notification instruction, the AI calculation module controls the target NPU calculation unit to transfer the target data in the FPGA to the NPU calculation unit. The data cached in the image acquisition processing module is directly forwarded to the AI calculation module.

Description

Technical Field

[0001] This invention relates to the field of image processing technology, and more specifically, to a high-resolution image detection and storage method and system. Background Technology

[0002] With the development of aerial cameras and the rapid advancement of artificial intelligence technology, users have increasingly higher demands for the analysis and processing of aerial image data. The performance requirements for aerial image detection and recognition functions, data analysis and processing, and data storage are becoming increasingly stringent. Existing technologies for high-resolution image detection and storage systems generally include an image acquisition and processing module, a resource scheduling module, an AI computing unit, and a storage module. The image acquisition and processing module typically consists of an FPGA or a dedicated image processing chip and peripheral chips. The resource scheduling module comprises a CPU, DDR, operating system, drivers, and application software. The AI ​​computing unit, depending on the computational scale and complexity, consists of a single NPU or multiple NPUs. The storage module, depending on the storage capacity, can consist of a single SSD or multiple SSDs.

[0003] The following problems exist in existing high-resolution image detection and storage systems: 1. Image preprocessing performance issues: On general-purpose processors, the CPU's computing power cannot meet the preprocessing requirements of ultra-high-resolution images. On dedicated ISP chips, image preprocessing algorithms are specific to certain scenes or data formats, lacking protocol interface expansion and multi-scene processing capabilities, and lacking generalization characteristics, making it difficult to meet the preprocessing needs of aerial drone images under complex conditions; 2. Image data storage performance issues: Ultra-high-resolution images consume a large amount of data bandwidth. Using standard IO fwrite file system data storage based on CPU memory consumes a large amount of bandwidth and CPU resources, resulting in insufficient processor power when processing high-definition image data; 3. Data transmission channel bandwidth bottlenecks: Traditional intelligent detection and data storage methods require data to pass through the CPU cache. Faced with the massive data transmission of ultra-high-resolution images, even using DMA channels will exceed the performance limit of the CPU cache; 4. Power consumption and size issues: The architecture using a graphics card + GPU coprocessor is very large and cumbersome, and consumes a lot of power. While the architecture of integrating an NPU into a SoC can achieve smaller size and lower power consumption, the CPU and NPU computing power of the SoC are relatively low. It is generally used in conventional video scenarios or in situations where the requirements for intelligent detection and recognition performance are not high. Its computing power cannot meet the needs of ultra-high resolution aerial drone image processing. Summary of the Invention

[0004] To address the aforementioned issues, embodiments of this application provide a high-resolution image detection and storage method and system.

[0005] In a first aspect, this application provides a high-resolution image detection and storage method, comprising the following steps:

[0006] S1: The image acquisition and processing module receives image data and stores it in multiple logical channels within it. The image acquisition and processing module preprocesses the image data. The image acquisition and processing module calculates the amount of data cached in each logical channel in real time and sends a target data transfer notification to the intelligent scheduling module when the amount of data in the target logical channel is not less than the preset value.

[0007] S2: The AI ​​computing module calculates the resource utilization of each NPU computing unit within it in real time;

[0008] S3: When the intelligent scheduling module receives the target data transfer notification, it notifies the target NPU computing unit with a preset idle rate based on the utilization rate of each NPU resource;

[0009] S4: After receiving the notification instruction, the AI ​​computing module controls the target NPU computing unit to move the target data in the FPGA to the NPU computing unit.

[0010] Furthermore, the process of the image acquisition and processing module receiving image data and storing it in its internal logical channels in step S1 specifically includes:

[0011] The FPGA in the image acquisition and processing module receives image data in real time and defines each logical channel according to the image type or interface type. After receiving the image data, each logical channel stores it in the DDR cache defined for each logical channel.

[0012] Furthermore, the step S1 of sending a target data transfer notification to the intelligent scheduling module when the amount of data in the target logical channel is not less than a preset value specifically includes:

[0013] When the amount of data in the target logical channel is not less than a preset value, the FPGA in the image acquisition and processing module issues a target data transfer notification in an interrupt manner. The parameters of the target data transfer notification include the current data's DDR starting address, the effective length of the data, the logical channel ID, the original image ID, and the slice ID.

[0014] Furthermore, step S3 specifically includes: the intelligent scheduling module first parses the parameters, and then notifies the target NPU computing unit with the highest idle rate of the received preprocessed image address and related parameters through register commands.

[0015] Furthermore, step S4 specifically includes: after receiving the control command, the target NPU computing unit starts DMA and moves the data in the FPGA's DDR to the target NPU computing unit's DDR through the PCIE Switch bus addressing method.

[0016] Furthermore, step S4 is followed by:

[0017] S5: The target NPU computing unit performs detection, identification, and calculation on the target data, and the AI ​​computing module feeds back the identification results to the intelligent scheduling module in an interrupt manner.

[0018] S6: The intelligent scheduling module sends post-processing instructions to the FPGA according to the preset image post-processing scheme;

[0019] S7: The FPGA performs post-processing on the original or pre-processed image cached in DDR based on the recognition results. The post-processed image then defines a new logical channel and stores the data in the DDR cache defined by the logical channel.

[0020] Furthermore, the reception of the preprocessed image address and related parameters specifically includes: upon receiving the FPGA data transfer notification, first parsing the parameters, then finding the file corresponding to the channel based on the logical channel ID, and finally calling the file system's extended interface to transmit the FPGA DDR start address and effective length.

[0021] Secondly, embodiments of this application provide a high-resolution image detection and storage system, including an image acquisition and processing module, an AI computing module, and an intelligent scheduling module. The image acquisition and processing module includes an FPGA and multiple logic channels, and the AI ​​computing module includes multiple NPU computing units.

[0022] The image acquisition and processing module is used to receive image data and store it in multiple logical channels. It preprocesses the image data, calculates the amount of data cached in each logical channel in real time, and sends a target data transfer notification to the intelligent scheduling module when the amount of data in the target logical channel is not less than a preset value.

[0023] The AI ​​computing module is used to calculate the resource utilization of each NPU computing unit in real time.

[0024] The image acquisition and processing module is used to send a target data transfer notification to the intelligent scheduling module when the amount of data in the target logical channel is not less than a preset value.

[0025] The intelligent scheduling module is used to notify the target NPU computing units with a preset idle rate based on the utilization rate of each NPU resource when receiving the target data transfer notification;

[0026] The AI ​​computing module is used to control the target NPU computing unit to move the target data in the FPGA to the NPU computing unit after the target NPU computing unit receives the notification instruction.

[0027] Thirdly, embodiments of this application provide an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the steps of the method provided as in the first aspect or any possible implementation of the first aspect.

[0028] Fourthly, embodiments of this application provide a computer-readable storage medium having a computer program stored thereon, wherein the computer program, when executed by a processor, implements the steps of the method provided as in the first aspect or any possible implementation thereof.

[0029] The beneficial effects of this invention are as follows: This application integrates high-resolution image acquisition, processing, intelligent detection, and high-speed storage. The data cached in the image acquisition and processing module is directly forwarded to the AI ​​computing module, thereby realizing that the data transmission path does not pass through the intelligent scheduling module, further reducing the resource consumption of the intelligent scheduling module and significantly improving data storage performance. At the same time, more NPU computing units can be expanded for parallel computing, thereby improving the real-time scheduling of the intelligent detection system. Furthermore, under various measures to reduce device power consumption, the intelligent recognition and processing flow of the device can still be guaranteed to remain unaffected. Attached Figure Description

[0030] To more clearly illustrate the technical solutions in the embodiments of this application, the accompanying drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0031] Figure 1 A schematic flowchart illustrating a high-resolution image detection and storage method provided in an embodiment of this application;

[0032] Figure 2 This is a schematic diagram of the structure of a high-resolution image detection and storage system provided in an embodiment of this application;

[0033] Figure 3 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application;

[0034] Figure 4 A schematic diagram of the architecture of an existing high-resolution image detection and storage system;

[0035] Figure 5 This is a schematic diagram of the process of image acquisition, preprocessing, detection and recognition, and storage in an existing high-resolution image detection and storage system.

[0036] Figure 6This application provides an architectural diagram of a high-resolution image detection and storage system according to an embodiment of the present application.

[0037] Figure 7 This is a schematic diagram illustrating the data flow of a high-resolution image detection and storage system provided in this application embodiment, covering data acquisition, storage, and detection.

[0038] Figure 8 This is a schematic diagram illustrating the data flow of a high-resolution image detection and storage system provided in this application embodiment, showing the recognition results and post-processed images.

[0039] In the diagram, 201 is the image acquisition and processing module, 202 is the AI ​​computing module, and 203 is the intelligent scheduling module. Detailed Implementation

[0040] The technical solutions in the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings.

[0041] In the following description, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance. The following description provides multiple embodiments of this application, which can be substituted or combined with each other. Therefore, this application can also be considered to include all possible combinations of the same and / or different embodiments described. Thus, if one embodiment includes features A, B, and C, and another embodiment includes features B and D, then this application should also be considered to include embodiments containing one or more other possible combinations of A, B, C, and D, even if such embodiments are not explicitly described in the following text.

[0042] The following description provides examples and does not limit the scope, applicability, or examples set forth in the claims. Changes may be made to the function and arrangement of the described elements without departing from the scope of this application. Various processes or components may be appropriately omitted, substituted, or added to the examples. For example, the described methods may be performed in a different order than described, and various steps may be added, omitted, or combined. Furthermore, features described with respect to some examples may be combined into other examples.

[0043] Please see Figure 1 , 6 7, 8. Figure 1 This is a flowchart illustrating a high-resolution image detection and storage method provided in an embodiment of this application. Figure 6 This is a schematic diagram of the architecture of a high-resolution image detection and storage system provided in an embodiment of this application. Figure 7 This is a schematic diagram illustrating the data flow of a high-resolution image detection and storage system provided in this application embodiment, covering data acquisition, storage, and detection. Figure 8This diagram illustrates the data flow of a high-resolution image detection and storage system for recognizing results and post-processing images, as provided in an embodiment of this application. In this embodiment, the method includes the following steps:

[0044] S1: The image acquisition and processing module 201 receives image data and stores it in multiple logical channels within it. The image acquisition and processing module 201 preprocesses the image data. The image acquisition and processing module 201 calculates the amount of data cached in each logical channel in real time and sends a target data transfer notification to the intelligent scheduling module 203 when the amount of data in the target logical channel is not less than a preset value.

[0045] S2: The AI ​​computing module 202 calculates the resource utilization of each NPU computing unit within it in real time;

[0046] S3: When the intelligent scheduling module 203 receives the target data transfer notification, it notifies the target NPU computing unit with a preset idle rate according to the utilization rate of each NPU resource;

[0047] S4: After receiving the notification instruction, the AI ​​computing module 202 controls the target NPU computing unit to move the target data in the FPGA to the NPU computing unit.

[0048] In this embodiment, with the development of aerial cameras and the rapid advancement of artificial intelligence technology, users have increasingly higher demands for the analysis and processing of aerial image data. The performance requirements for aerial image detection and recognition functions, data analysis and processing, and data storage are becoming increasingly stringent. The objective of this invention is to ensure high-resolution aerial image target recognition rates, real-time detection and recognition, real-time data retrieval and analysis, and high-speed data storage, while also featuring lightweight, low-power consumption, and small footprint.

[0049] In this embodiment, with the innovation of science and technology, aerial cameras have made great progress, with significant improvements in sensor resolution, frame rate, sensitivity, and response speed. However, this has also led to correspondingly higher performance requirements for processing and analyzing massive amounts of image data. Airborne intelligent detection equipment is an embedded standalone device. When performing high-resolution image processing and computation, the standalone device faces limitations in computing resources, power consumption, size, CPU resource utilization, and memory bandwidth consumption, which restrict the full utilization of external computing resources. See also... Figure 4 , Figure 4This is a schematic diagram of the architecture of an existing high-resolution image detection and storage system. An existing high-resolution image detection and storage system includes an image acquisition and processing module, a resource scheduling module, an AI computing unit, and a storage module. The image acquisition and processing module is generally composed of an FPGA or a dedicated image processing chip and peripheral chips. The resource scheduling module consists of a CPU, DDR, operating system, drivers, and application software. The AI ​​computing unit, depending on the computing scale and complexity, consists of a single NPU or multiple NPUs. The storage module, depending on the storage scale, can consist of a single or multiple SSDs, generally using NVMe protocol SSDs. See also... Figure 5 , Figure 5 This is a schematic diagram of the process of image acquisition, preprocessing, detection and recognition, and storage in an existing high-resolution image detection and storage system. The process of image acquisition, preprocessing, detection and recognition, and storage in an existing high-resolution image detection and storage system is as follows: After image acquisition, it is preprocessed by the hardware module and divided into raw data and preprocessed image data. The data is then DMA'd into the resource scheduling software cache. The preprocessed image is forwarded to the NPU computing module for detection and recognition. Another copy of the raw preprocessed image data and the raw image data are written to the data storage module according to the data storage rules.

[0050] In one possible implementation, step S1, in which the image acquisition and processing module 201 receives image data and stores it in its internal logical channels, specifically includes:

[0051] The FPGA in the image acquisition and processing module 201 receives image data in real time and defines each logical channel according to the image type or interface type. After receiving the image data, each logical channel stores it in the DDR cache defined by each logical channel.

[0052] In one possible implementation, step S1, which involves sending a target data transfer notification to the intelligent scheduling module 203 when the amount of data in the target logical channel is not less than a preset value, specifically includes:

[0053] When the amount of data in the target logical channel is not less than a preset value, the FPGA in the image acquisition and processing module 201 issues a target data transfer notification in an interrupt manner. The parameters of the target data transfer notification include the current data's DDR starting address, the effective length of the data, the logical channel ID, the original image ID, and the slice ID.

[0054] In one possible implementation, step S3 specifically includes: the intelligent scheduling module 203 first parses the parameters, and then notifies the target NPU computing unit with the highest idle rate of the received preprocessed image address and related parameters through register commands.

[0055] In one possible implementation, step S4 specifically includes: after receiving the control command, the target NPU computing unit starts DMA and moves the data in the FPGA's DDR to the target NPU computing unit's DDR through the PCIE Switch bus addressing method.

[0056] In one possible implementation, step S4 is followed by:

[0057] S5: The target NPU computing unit performs detection, identification and calculation on the target data, and the AI ​​computing module 202 feeds back the identification result to the intelligent scheduling module 203 in an interrupt manner.

[0058] S6: The intelligent scheduling module 203 sends post-processing instructions to the FPGA according to the preset image post-processing scheme;

[0059] S7: The FPGA performs post-processing on the original or pre-processed image cached in DDR based on the recognition results. The post-processed image then defines a new logical channel and stores the data in the DDR cache defined by the logical channel.

[0060] In one possible implementation, receiving the preprocessed image address and related parameters specifically includes: first parsing the parameters upon receiving the FPGA data transfer notification, then finding the file corresponding to the channel based on the logical channel ID, and finally calling the file system's extended interface to transmit the FPGA DDR start address and effective length.

[0061] In this embodiment, high-definition image acquisition is achieved using an FPGA, and image preprocessing (cropping, noise reduction, dehazing, etc.) is performed using heterogeneous FPGA resources. The CPU software implements multi-NPU resource scheduling. The original image data and the processed image data do not pass through the CPU cache, but are directly stored or forwarded to the NPU through the FPGA, which further reduces CPU resource consumption and significantly improves data storage performance. At the same time, more NPU modules can be expanded for parallel computing, thereby improving the real-time scheduling of the intelligent detection system. Furthermore, under various measures to reduce device power consumption, the intelligent recognition and processing flow of the device can still be guaranteed to remain unaffected.

[0062] In this embodiment, with the development of aerial reconnaissance technology, the computational and storage tasks of airborne intelligent detection and identification storage systems are increasing, placing higher demands on the computing power and storage resources of the equipment. Furthermore, the limited airborne environment also presents greater challenges in terms of the size and power consumption of multiple devices. Therefore, improving the computing and storage capabilities of the equipment under existing resource conditions is of great significance. In the intelligent detection system architecture, a new data transmission path is added, allowing data cached in the FPGA to be directly forwarded to the storage and NPU, thus achieving a data transmission path that does not pass through the CPU. See the architecture diagram of this application. Figure 6As shown, the data flow of the data acquisition, storage, and detection process is described in the attached diagram. Figure 7 As shown, the recognition results and post-processed image data flow can be found in [reference needed]. Figure 8 As shown.

[0063] In this embodiment, because the amount of aerial image data is very large, the data transmission process will consume a lot of CPU computing resources and DDR bandwidth resources. In this application, the data transmission path of the FPGA image acquisition DMA architecture is modified. The original data flow of first moving the raw and preprocessed image data from the FPGA DDR to the CPU memory and then from the CPU memory to the storage and NPU is modified to a data flow of directly moving the FPGA DDR image data to the SSD storage or NPU. The specific implementation process is as follows:

[0064] 1) The FPGA receives image data in real time and defines each logic channel according to image type or interface type. After receiving image data, each channel stores it in the DDR cache defined for that logic channel.

[0065] 2) The FPGA receives image data in real time and performs preprocessing on the image (including but not limited to noise reduction, dehazing, image bit-width conversion, scaling, slicing, etc.). When the FPGA receives image data, it can perform preprocessing on the completed portion of the image once the data has been received row by row up to the set slice height (preprocessing is performed while the image is being received). The preprocessed image is then distributed in two paths: one is buffered in the data recording logic channel, and the other is buffered in the intelligent detection and recognition logic channel.

[0066] 3) When the FPGA calculates the amount of buffered data in each logic channel in real time and the amount of data can be moved, it notifies the intelligent scheduling module 203 via an interrupt. The parameters include: the current data DDR starting address in the FPGA, the effective length of the data, the logic channel ID, the original image ID, the slice ID, and other information.

[0067] 4) The intelligent scheduling module 203 queries the resource utilization rate of each NPU computing unit in the AI ​​computing module 202 in real time. When it receives the notification that the FPGA data can be transferred, the intelligent scheduling module 203 first parses the parameters, and then notifies the NPU computing unit with the highest idle rate of the received preprocessed image address and related parameters through register commands.

[0068] 5) After receiving the control command, the NPU computing unit starts DMA and moves the data in the FPGA DDR to the NPU computing unit's DDR through the PCIE Switch bus addressing mode. The NPU computing unit then begins to perform detection and recognition calculations.

[0069] 6) After the NPU computing unit completes the detection and recognition of an image frame, it feeds back the detection and recognition results to the intelligent scheduling module 203 via an interrupt. The intelligent scheduling module 203 then sends post-processing instructions to the FPGA according to the preset image post-processing scheme. The FPGA performs post-processing (cropping, OSD, scaling, etc.) on the original image or pre-processed image cached in DDR according to the recognition results. The post-processed image defines a new logical channel and stores the data in the DDR cache.

[0070] 7) When the FPGA data is received as a transferable notification, the intelligent scheduling module 203 first parses the parameters, then finds the file corresponding to the channel based on the logical channel ID, and finally calls the file system's extended interface to pass the FPGA DDR starting address and effective length.

[0071] 8) The kernel file system forms a write operation based on the logical address. The memory driver submits an I / O command to the memory, whose parameters include the data storage FPGA DDR starting address, effective length, and storage logical address.

[0072] 9) The memory receives the IO command, starts DMA, and moves the data in the FPGA DDR to the corresponding block address through the PCIE Switch bus addressing mode, thus completing the data writing to disk.

[0073] 10) The FPGA monitors the image acquisition and processing status in real time. When a complete image frame is acquired or a frame is processed, the application software is notified via an interrupt. The application software builds index information (image time, image type, sequence number, image size, bit width, image target information, corresponding file path and file name, file offset, etc.) based on the interrupt message and writes it to the database in real time.

[0074] The following will be combined with the appendix Figure 2 This application provides a detailed description of a high-resolution image detection and storage system according to embodiments. It should be noted that the appendix... Figure 2 The high-resolution image detection and storage system shown is used to execute the present application. Figure 1 The methods shown in the embodiments are for illustrative purposes only, illustrating the parts relevant to the embodiments of this application. For specific technical details not disclosed, please refer to this application. Figure 1 The example shown.

[0075] Please see Figure 2 , Figure 2 This is a schematic diagram of the structure of a high-resolution image detection and storage system provided in an embodiment of this application. Figure 2As shown, the system includes an image acquisition and processing module 201, an AI computing module 202, and an intelligent scheduling module 203. The image acquisition and processing module 201 includes an FPGA and multiple logic channels, and the AI ​​computing module 202 includes multiple NPU computing units.

[0076] The image acquisition and processing module 201 is used to receive image data and store it in multiple logical channels within it, preprocess the image data, calculate the amount of data cached in each logical channel in real time, and send a target data transfer notification to the intelligent scheduling module 203 when the amount of data in the target logical channel is not less than a preset value.

[0077] AI computing module 202 is used to calculate the resource utilization of each NPU computing unit in real time;

[0078] The image acquisition and processing module 201 is used to send a target data transfer notification to the intelligent scheduling module 203 when the amount of data in the target logical channel is not less than a preset value.

[0079] The intelligent scheduling module 203 is used to notify the target NPU computing unit with a preset idle rate based on the utilization rate of each NPU resource when receiving the target data transfer notification.

[0080] AI computing module 202 is used to control the target NPU computing unit to move the target data in the FPGA to the NPU computing unit after the target NPU computing unit receives the notification instruction.

[0081] In this embodiment, the image acquisition and processing module 201 includes an FPGA, a dedicated image processing chip, peripheral chips, and an image processing logic unit, responsible for image acquisition, preprocessing, post-processing, interface conversion, and image output. The intelligent scheduling module 203 includes a CPU, DDR, operating system, drivers, and resource scheduling software, responsible for intelligent process resource scheduling, data flow control, external communication, and parameter setting. The AI ​​computing module 202 includes a storage unit and multiple NPU computing units. The multiple NPU computing units are arranged in an array, forming an NPU parallel computing array. The combination of NPU computing unit arrays improves the speed of intelligent detection and recognition. The storage unit includes a single or multiple SSDs. To maintain speed, NVMe protocol SSDs are generally used. Multiple SSDs can use RAID to improve storage speed. The storage unit can be an NVMe storage unit.

[0082] In the embodiments of this application, compared with existing intelligent detection and storage devices, this application has the following advantages.

[0083] 1) Achieve high-speed storage in a simple way, reducing the dependence of high-speed storage on high-performance CPUs; at the same time, retain the original characteristics of the general file system. Data files generated by high-speed storage can be accessed through standard file access interfaces, which facilitates storage management.

[0084] 2) Compared with conventional methods, the real-time performance of detection and recognition is improved by receiving and preprocessing the original image, forwarding the preprocessed image directly to the NPU for detection and recognition calculation without passing through the CPU cache, and using multiple cascaded NPU modules for parallel computing.

[0085] 3) The recognition results and post-processed image results are linked and stored in a database, facilitating quick retrieval of the target and target image afterward. Data demonstration and playback do not require further image processing; the processed image can be directly forwarded, reducing system response time and ensuring real-time data retrieval and analysis.

[0086] 4) Compared to conventional methods, the raw image data and the processed image data do not pass through the CPU cache. Instead, they are directly stored or forwarded to the NPU computing unit via the FPGA. This significantly reduces the CPU computing resources and DMA bandwidth resources occupied by data interaction, allowing the system to expand to more NPU computing units and storage disks, thereby increasing the system's performance ceiling.

[0087] 5) Due to its low CPU resource consumption, this system can use a lower-power embedded processor to implement data flow scheduling and management. It can also reduce the processor's clock speed or disable some processor cores. Lower power consumption effectively reduces the device's heat dissipation area and power supply size, thus making the device lightweight.

[0088] Those skilled in the art will clearly understand that the technical solutions of the embodiments of this application can be implemented by means of software and / or hardware. In this specification, "unit", "module" and "part" refer to software and / or hardware that can independently complete or cooperate with other components to complete a specific function, wherein the hardware may be, for example, a field-programmable gate array (FPGA), an integrated circuit (IC), etc.

[0089] Each processing unit and / or module in the embodiments of this application can be implemented by an analog circuit that implements the functions described in the embodiments of this application, or by software that executes the functions described in the embodiments of this application.

[0090] See Figure 3 It shows a schematic diagram of the structure of an electronic device according to an embodiment of this application, which can be used to implement... Figure 1 The method in the illustrated embodiment. (As shown) Figure 3As shown, the electronic device 300 may include: at least one central processing unit 301, at least one network interface 304, user interface 303, memory 305, and at least one communication bus 302.

[0091] The communication bus 302 is used to enable communication between these components.

[0092] The user interface 303 may include a display screen and a camera. Optionally, the user interface 303 may also include a standard wired interface and a wireless interface.

[0093] The network interface 304 may optionally include a standard wired interface or a wireless interface (such as a Wi-Fi interface).

[0094] The central processing unit 301 may include one or more processing cores. The central processing unit 301 connects to various parts within the electronic device 300 using various interfaces and lines. It executes various functions of the terminal 300 and processes data by running or executing instructions, programs, code sets, or instruction sets stored in the memory 305, and by calling data stored in the memory 305. Optionally, the central processing unit 301 may be implemented using at least one hardware form of Digital Signal Processing (DSP), Field-Programmable Gate Array (FPGA), or Programmable Logic Array (PLA). The central processing unit 301 may integrate one or a combination of several of the following: a Central Processing Unit (CPU), a Graphics Processing Unit (GPU), and a modem. The CPU primarily handles the operating system, user interface, and applications; the GPU is responsible for rendering and drawing the content required for display on the screen; and the modem handles wireless communication. It is understood that the modem may also not be integrated into the central processing unit 301 and may be implemented as a separate chip.

[0095] The memory 305 may include random access memory (RAM) or read-only memory. Optionally, the memory 305 may include a non-transitory computer-readable storage medium. The memory 305 can be used to store instructions, programs, code, code sets, or instruction sets. The memory 305 may include a program storage area and a data storage area, wherein the program storage area may store instructions for implementing an operating system, instructions for at least one function (such as touch function, sound playback function, image playback function, etc.), instructions for implementing the above-described method embodiments, etc.; the data storage area may store data involved in the above-described method embodiments, etc. Optionally, the memory 305 may also be at least one storage device located remotely from the aforementioned central processing unit 301. Figure 3 As shown, the memory 305, which serves as a computer storage medium, may include an operating system, a network communication module, a user interface module, and program instructions.

[0096] exist Figure 3 In the illustrated electronic device 300, the user interface 303 is mainly used to provide an input interface for the user and to acquire user input data; while the central processing unit 301 can be used to call a high-resolution image detection storage application stored in the memory 305 and specifically perform the following operations:

[0097] S1: The image acquisition and processing module 201 receives image data and stores it in multiple logical channels within it. The image acquisition and processing module 201 preprocesses the image data. The image acquisition and processing module 201 calculates the amount of data cached in each logical channel in real time and sends a target data transfer notification to the intelligent scheduling module 203 when the amount of data in the target logical channel is not less than a preset value.

[0098] S2: The AI ​​computing module 202 calculates the resource utilization of each NPU computing unit within it in real time;

[0099] S3: When the intelligent scheduling module 203 receives the target data transfer notification, it notifies the target NPU computing unit with a preset idle rate according to the utilization rate of each NPU resource;

[0100] S4: After receiving the notification instruction, the AI ​​computing module 202 controls the target NPU computing unit to move the target data in the FPGA to the NPU computing unit.

[0101] This application also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the above-described method. The computer-readable storage medium may include, but is not limited to, any type of disk, including floppy disks, optical disks, DVDs, CD-ROMs, microdrives, as well as magneto-optical disks, ROMs, RAMs, EPROMs, EEPROMs, DRAMs, VRAMs, flash memory devices, magnetic cards or optical cards, nanosystems (including molecular memory ICs), or any type of medium or device suitable for storing instructions and / or data.

[0102] It should be noted that, for the sake of simplicity, the foregoing method embodiments are all described as a series of actions. However, those skilled in the art should understand that this application is not limited to the described order of actions, as some steps may be performed in other orders or simultaneously according to this application. Furthermore, those skilled in the art should also understand that the embodiments described in the specification are preferred embodiments, and the actions and modules involved are not necessarily essential to this application.

[0103] In the above embodiments, the descriptions of each embodiment have different focuses. For parts not described in detail in a certain embodiment, please refer to the relevant descriptions in other embodiments.

[0104] In the several embodiments provided in this application, it should be understood that the disclosed apparatus can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some service interface; the indirect coupling or communication connection between devices or units may be electrical or other forms.

[0105] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0106] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0107] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage device (CMD). Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a memory and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned memory includes various media capable of storing program code, such as USB flash drives, read-only memory (ROM), random access memory (RAM), portable hard drives, magnetic disks, or optical disks.

[0108] Those skilled in the art will understand that all or part of the steps in the various methods of the above embodiments can be implemented by a program instructing related hardware. The program can be stored in a computer-readable storage medium, which may include: a flash drive, a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disk, etc.

[0109] The foregoing description is merely an exemplary embodiment of this disclosure and should not be construed as limiting the scope of this disclosure. Any equivalent changes and modifications made in accordance with the teachings of this disclosure shall still fall within the scope of this disclosure. Those skilled in the art will readily conceive of embodiments of this disclosure upon considering the specification and practicing the disclosure herein. This application is intended to cover any variations, uses, or adaptations of this disclosure that follow the general principles of this disclosure and include common knowledge or customary techniques in the art not described herein. The specification and embodiments are to be considered exemplary only, and the scope and spirit of this disclosure are defined by the claims.

Claims

1. A high-resolution image detection and storage method, characterized in that, Includes the following steps: S1: The image acquisition and processing module (201) receives image data and stores it in multiple logical channels within it. The image acquisition and processing module (201) preprocesses the image data. The image acquisition and processing module (201) calculates the amount of data cached in each logical channel in real time and sends a target data transfer notification to the intelligent scheduling module (203) when the amount of data in the target logical channel is not less than the preset value. S2: The AI ​​computing module (202) calculates the resource utilization of each NPU computing unit within it in real time; S3: When receiving the target data transfer notification, the intelligent scheduling module (203) notifies the target NPU computing unit with a preset idle rate according to the utilization rate of each NPU resource; S4: After receiving the notification instruction, the AI ​​computing module (202) controls the target NPU computing unit to move the target data in the FPGA to the NPU computing unit.

2. The high-resolution image detection and storage method as described in claim 1, characterized in that, In step S1, the image acquisition and processing module (201) receives image data and stores it in its internal logical channels, specifically including: The FPGA in the image acquisition and processing module (201) receives image data in real time and defines each logical channel according to the image type or interface type. After receiving the image data, each logical channel stores it in the DDR cache defined by each logical channel.

3. A high-resolution image detection and storage method as described in claim 1 or 2, characterized in that, Step S1, which involves sending a target data transfer notification to the intelligent scheduling module (203) when the amount of data in the target logical channel is not less than a preset value, specifically includes: When the amount of data in the target logical channel is not less than a preset value, the FPGA in the image acquisition and processing module (201) issues a target data transfer notification in an interrupt manner. The parameters of the target data transfer notification include the current data DDR starting address, data effective length, logical channel ID, original image ID, and slice ID.

4. The high-resolution image detection and storage method as described in claim 1 or 2, characterized in that, Step S3 specifically includes: the intelligent scheduling module (203) first parses the parameters, and then notifies the target NPU computing unit with the highest idle rate of the received preprocessed image address and related parameters through register commands.

5. A high-resolution image detection and storage method as described in claim 1 or 2, characterized in that, Step S4 specifically includes: after receiving the control command, the target NPU computing unit starts DMA and moves the data in the FPGA's DDR to the target NPU computing unit's DDR through the PCIE Switch bus addressing method.

6. A high-resolution image detection and storage method as described in claim 1 or 2, characterized in that, Step S4 is followed by: S5: The target NPU computing unit performs detection, identification and calculation on the target data, and the AI ​​computing module (202) feeds back the identification results to the intelligent scheduling module (203) in an interrupt manner. S6: The intelligent scheduling module (203) sends post-processing instructions to the FPGA according to the preset image post-processing scheme; S7: The FPGA performs post-processing on the original or pre-processed image cached in DDR based on the recognition results. The post-processed image then defines a new logical channel and stores the data in the DDR cache defined by the logical channel.

7. A high-resolution image detection and storage method as described in claim 4, characterized in that, The specific steps for receiving the preprocessed image address and related parameters include: first, parsing the parameters upon receiving the FPGA data transfer notification; then, finding the file corresponding to the channel based on the logical channel ID; and finally, calling the file system's extended interface to transmit the FPGA DDR start address and effective length.

8. A high-resolution image detection and storage system, characterized in that: It includes an image acquisition and processing module (201), an AI computing module (202), and an intelligent scheduling module (203). The image acquisition and processing module (201) includes an FPGA and multiple logic channels, and the AI ​​computing module (202) includes multiple NPU computing units. The image acquisition and processing module (201) is used to receive image data and store it in multiple logical channels within it, preprocess the image data, calculate the amount of data cached in each logical channel in real time, and send a target data transfer notification to the intelligent scheduling module (203) when the amount of data in the target logical channel is not less than a preset value. The AI ​​computing module (202) is used to calculate the resource utilization rate of each NPU computing unit in real time; The image acquisition and processing module (201) is used to send a target data transfer notification to the intelligent scheduling module (203) when the amount of data in the target logical channel is not less than a preset value; The intelligent scheduling module (203) is used to notify the target NPU computing unit with a preset idle rate according to the utilization rate of each NPU resource when receiving the target data transfer notification; The AI ​​computing module (202) is used to control the target NPU computing unit to move the target data in the FPGA to the NPU computing unit after the target NPU computing unit receives the notification instruction.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the steps of the method as described in any one of claims 1-7.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the steps of the method as described in any one of claims 1-7.

Images