Remote sensing image processing method and device based on algorithm hardware collaborative optimization

Abstract

The invention relates to the technical field of remote sensing image processing, in particular to a remote sensing image processing method and device based on algorithm hardware collaborative optimization, equipment and a storage medium, and the method comprises the steps: carrying out the sea-land segmentation of a to-be-detected remote sensing image through a pre-constructed hybrid heterogeneous calculation model, and obtaining a land mask image; carrying out lightweight processing on the YOLOv5s architecture based on a mixed precision quantification strategy and a dynamic precision matching mechanism to obtain a ship detection lightweight model; regulating and controlling the ship detection lightweight model based on a dynamic parameter self-adaptive regulation and control strategy to obtain a self-adaptive ship detection model; and processing the remote sensing image to be detected by using the land mask image and the adaptive ship detection model to obtain a ship detection result. Through an algorithm hardware collaborative optimization strategy, the characteristics of a heterogeneous hardware platform are fully utilized, remote sensing image ship detection with high efficiency, low power consumption and strong stability is realized, the detection efficiency and precision are improved, the power consumption is reduced, and the system stability is enhanced.

Description

Technical Field

[0001] The present application relates to the field of remote sensing image processing technology, and in particular to a remote sensing image processing method, device, equipment and storage medium based on algorithm-hardware collaborative optimization. Background Art

[0002] Satellite remote sensing technology, due to its ability to cover large areas and monitor around the clock, has become a key data source for ship inspection. However, with the explosive growth of satellite remote sensing data, traditional satellite-to-ground transmission methods have led to bandwidth saturation, increasing the pressure on ground-based data processing centers and restricting the real-time nature of inspection results. To address the pressure of data transmission and processing, on-orbit computing technology has been proposed. This involves directly analyzing the raw data onboard equipment and transmitting only key results back to the ground, thereby reducing the burden on data transmission and ground system processing. However, the computing power and energy supply of onboard equipment are limited, making it difficult to meet the requirements of complex inspection tasks. Traditional ship detection methods rely on manually designed feature engineering. When faced with massive amounts of high-resolution remote sensing imagery, they suffer from low computational efficiency and insufficient ability to adapt to complex scenarios, making it difficult to meet real-time processing requirements.

[0003] In recent years, deep neural network detection models (such as the YOLO series) have performed well in natural scene target detection. However, their direct application in satellite remote sensing faces challenges such as high computing resource consumption, high hardware processing power requirements, and limited energy and space on satellite platforms. The sensitivity of ship detection tasks requires the priority use of domestically produced processors with higher security and controllability. Although domestic processors have made progress in the field of autonomous control, they still lag behind international advanced levels in terms of performance, memory resources, and energy efficiency. As a result, the deployment of deep neural networks on domestic hardware faces technical challenges such as model lightweighting and hardware adaptation. Existing research has mostly focused on single hardware optimization, while there is insufficient research on algorithm-hardware collaborative optimization for domestic heterogeneous computing platforms, which has limited the on-orbit deployment of ship detection systems on domestic hardware.

[0004] In summary, how to achieve efficient ship detection based on limited hardware platforms is an urgent problem that needs to be solved. Summary of the Invention

[0005] The present application aims to solve one of the technical problems in the related art at least to a certain extent.

[0006] To this end, the first purpose of this application is to propose a remote sensing image processing method based on algorithm-hardware collaborative optimization to solve the problems of existing technical means of ship detection technology in processing massive high-resolution remote sensing images, adapting to complex scenes, meeting real-time processing requirements, and application on on-orbit computing devices.

[0007] The second object of this application is to provide a device.

[0008] The third objective of this application is to provide an electronic device.

[0009] The fourth object of this application is to provide a computer-readable storage medium.

[0010] To achieve the above objectives, the first embodiment of the present application proposes a remote sensing image processing method based on algorithm-hardware collaborative optimization, comprising:

[0011] Use the pre-built hybrid heterogeneous computing model to perform land and sea segmentation on the remote sensing image to be detected and obtain the land mask image;

[0012] The YOLOv5s architecture is lightweighted based on the mixed precision quantization strategy and dynamic precision matching mechanism to obtain a lightweight model for ship detection.

[0013] The ship detection lightweight model is regulated based on a dynamic parameter adaptive control strategy to obtain an adaptive ship detection model;

[0014] The remote sensing image to be detected is processed using the land mask image and the adaptive ship detection model to obtain a ship detection result.

[0015] Preferably, the method of performing land and sea segmentation processing on the remote sensing image to be detected by using a pre-built hybrid heterogeneous computing model to obtain a land mask image includes:

[0016] Acquire the remote sensing image to be detected;

[0017] parsing the remote sensing image to be detected based on a network protocol stack to obtain a parsed remote sensing image;

[0018] Extracting green light band data and near infrared band data from the analyzed remote sensing image;

[0019] Calculating a normalized ratio based on the green light band data and the near infrared band data;

[0020] The segmentation threshold trained in advance using historical data is compared with the normalized ratio to obtain a binary land mask.

[0021] Preferably, the lightweight processing of the YOLOv5s architecture based on the mixed precision quantization strategy and the dynamic precision matching mechanism to obtain the lightweight model for ship detection includes:

[0022] Based on the computing characteristics of the processor, the YOLOv5s architecture is quantized by combining integer precision and half-floating point precision to obtain two quantization accuracy results;

[0023] Based on the two quantization accuracy results, a mathematical correlation model between quantization accuracy and model complexity is constructed, and a dynamic matching mechanism is added to obtain a lightweight model for ship detection.

[0024] Preferably, the controlling the lightweight ship detection model based on the dynamic parameter adaptive control strategy to obtain the adaptive ship detection model includes:

[0025] Get hardware status data;

[0026] Evaluate the hardware status data based on a noise suppression mechanism and a weighted status evaluation method to obtain a hardware data health index;

[0027] Dynamically adjust input resolution based on the hardware data health index, optimize parallel computing granularity, and build an adaptive adjustment strategy;

[0028] The ship detection lightweight model is adjusted based on the adaptive adjustment strategy to obtain an adaptive ship detection model.

[0029] Preferably, the acquiring of hardware status data includes:

[0030] A three-state collaborative perception network for graphics processor memory, temperature, and power consumption is constructed, and the hardware status indicators are periodically sampled using the three-state collaborative perception network.

[0031] Preferably, the noise suppression mechanism includes: using sliding window filtering to smooth the collected hardware status data to suppress instantaneous fluctuation interference, and the calculation formula is:

[0032]

[0033] Among them, S t is the current state parameter, is the estimated value after smoothing, and α is the dynamically adjusted smoothing factor.

[0034] Preferably, the parallel computing granularity optimization includes:

[0035] Dynamically adjust the number of parallel processing and batch size, and organize the number of parallel processing and batch size into a set of parallel granularities from small to large;

[0036] Based on the current hardware health index, adjust the parallel granularity. The calculation formula is:

[0037]

[0038] Where B={B1(1,1),B2(1,2),...,B max (16,16)} is the parallel granularity set, α b and β bis the dual threshold parameter.

[0039] To achieve the above objectives, the second embodiment of the present application proposes a remote sensing image processing device based on algorithm-hardware collaborative optimization, comprising:

[0040] The image processing module uses a pre-built hybrid heterogeneous computing model to perform land and sea segmentation on the remote sensing image to be detected and obtain a land mask image;

[0041] The lightweight processing module performs lightweight processing on the YOLOv5s architecture based on a mixed-precision quantization strategy and a dynamic precision matching mechanism to obtain a lightweight model for ship detection.

[0042] An adaptive module controls the lightweight ship detection model based on a dynamic parameter adaptive control strategy to obtain an adaptive ship detection model;

[0043] The detection module uses the adaptive ship detection model to process the land mask image and the remote sensing image to be detected to obtain a ship detection result.

[0044] To achieve the above-mentioned purpose, a third embodiment of the present application provides an electronic device, comprising: a processor, and a memory communicatively connected to the processor;

[0045] The memory stores computer-executable instructions;

[0046] The processor executes the computer-executable instructions stored in the memory to implement any of the above methods.

[0047] To achieve the above-mentioned purpose, the fourth embodiment of the present application proposes a computer-readable storage medium, including computer-executable instructions stored in the computer-readable storage medium, and the computer-executable instructions are used to implement any of the methods described above when executed by a processor.

[0048] This application provides a remote sensing image processing method based on algorithm-hardware co-optimization. By building a hybrid heterogeneous computing platform that integrates CPU, GPU, and FPGA hardware, it achieves efficient, low-power, and highly stable remote sensing image ship detection. It dynamically adjusts input resolution and parallel granularity to optimize computing resource allocation based on hardware status and reduce power consumption. A mixed-precision quantization strategy optimizes model complexity and detection accuracy, maintaining the model's powerful feature expression capabilities. Through an algorithm-hardware co-optimization strategy, the characteristics of the heterogeneous hardware platform are fully utilized to achieve efficient, low-power, and highly stable remote sensing image ship detection. This improves detection efficiency and accuracy, reduces power consumption, enhances system stability, and provides a flexible and efficient solution for different application scenarios.

[0049] Additional aspects and advantages of the present application will be given in part in the description below, and in part will become apparent from the description below, or will be learned through practice of the present application. BRIEF DESCRIPTION OF THE DRAWINGS

[0050] The above and / or additional aspects and advantages of the present application will become apparent and easily understood from the following description of the embodiments in conjunction with the accompanying drawings, in which:

[0051] Figure 1 This is a flowchart of a first specific embodiment of a remote sensing image processing method based on algorithm and hardware collaborative optimization provided by the present invention;

[0052] Figure 2 This is a flow chart of the remote sensing image processing method based on domestic heterogeneous hardware platform based on algorithm-hardware collaborative optimization;

[0053] Figure 3 This is a schematic diagram of the system framework of the ship detection method;

[0054] Figure 4 Schematic diagram designed for FPGA module;

[0055] Figure 5 This is a flow chart of the dynamic adaptive control method based on hardware perception;

[0056] Figure 6 This is a schematic diagram of ship detection results;

[0057] Figure 7 This is a structural block diagram of a remote sensing image processing device based on algorithm-hardware collaborative optimization provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0058] The core of the present invention is to provide a remote sensing image processing method, device, electronic device and storage medium based on algorithm-hardware collaborative optimization. Through the algorithm-hardware collaborative optimization strategy, the characteristics of heterogeneous hardware platforms are fully utilized to achieve high-efficiency, low-power consumption and strong stability of remote sensing image ship detection, thereby improving detection efficiency and accuracy.

[0059] In order to enable those skilled in the art to better understand the present invention, the present invention is further described in detail below with reference to the accompanying drawings and specific embodiments. Obviously, the embodiments described are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative work are within the scope of protection of the present invention.

[0060] Please refer to Figure 1 , Figure 1This is a flowchart of a first specific embodiment of a remote sensing image processing method based on algorithm and hardware collaborative optimization provided by the present invention; the specific operating steps are as follows:

[0061] Step S101: using a pre-built hybrid heterogeneous computing model to perform land and sea segmentation processing on the remote sensing image to be detected to obtain a land mask image;

[0062] Step S102: Lightweight processing is performed on the YOLOv5s architecture based on the mixed precision quantization strategy and the dynamic precision matching mechanism to obtain a lightweight model for ship detection;

[0063] Step S103: regulating the lightweight ship detection model based on the dynamic parameter adaptive control strategy to obtain an adaptive ship detection model;

[0064] Step S104: Processing the remote sensing image to be detected using the land mask image and the adaptive ship detection model to obtain a ship detection result.

[0065] Based on the above embodiment, this embodiment describes step S101 in detail:

[0066] The hardware architecture design of the heterogeneous computing framework utilizes a self-controlled domestically produced processor cluster, integrating three core computing units: a central processing unit (CPU), a graphics processing unit (GPU), and a field-programmable gate array (FPGA), to create a highly efficient processing architecture with complementary functions. The CPU, as the host control module, is responsible for system-level task scheduling, logical decision-making, and post-detection processing (such as non-maximum suppression), ensuring the coordination of system operations and the reliability of output results. The FPGA, leveraging the advantages of its hardware pipeline architecture, implements pixel-level land and sea segmentation driven by the NDWI algorithm. Its seamless "transmission-computation" feature compresses the latency of single-frame image segmentation to milliseconds, providing real-time preprocessing support for ship detection. The GPU deploys the YOLOv5s core detection network, leveraging its powerful parallel computing capabilities and floating-point precision to match the computational requirements of convolutional neural networks and ensure efficient and accurate target detection.

[0067] Specifically, a remote sensing image to be detected is obtained; the remote sensing image to be detected is parsed based on a network protocol stack to obtain a parsed remote sensing image; green light band data and near-infrared band data in the parsed remote sensing image are extracted; a normalized ratio is calculated based on the green light band data and the near-infrared band data; a segmentation threshold trained in advance using historical data is compared with the normalized ratio to obtain a binary land mask.

[0068] The FPGA uses a hardware-level pipeline design, reconstructing the NDWI algorithm calculation process through hardware description language to eliminate the redundant power consumption generated by data movement in traditional solutions. In each calculation cycle, the pipeline simultaneously completes data reading, exponential calculations, and result return, while improving energy efficiency while meeting the power consumption constraint of less than 5W for the module's on-orbit computing equipment. The GPU-CPU collaborative inference framework uses a domestically produced GPU to execute the quantized YOLOv5s inference network. The CPU dynamically schedules the algorithm's parallelism and input image slice size based on the hardware operating status, while leveraging the GPU's throughput and computing advantages while maintaining the high energy efficiency of the inference process and the stability of the system operation.

[0069] In addition, domestic processors ensure detection security from the chip level and operating system level, eliminate potential risks of imported equipment, and provide a reliable safety barrier for ship detection missions.

[0070] Based on the above embodiment, this embodiment describes step S102 in detail:

[0071] Based on the computing characteristics of the processor, the YOLOv5s architecture is quantized by combining integer precision and half-floating-point precision to obtain two quantization accuracy results. Based on the two quantization accuracy results, a mathematical correlation model between quantization accuracy and model complexity is constructed, and a dynamic matching mechanism is added to obtain a lightweight model for ship detection.

[0072] Specifically, we conducted algorithm lightweight research on the YOLOv5s architecture, achieved coordinated optimization of model complexity and detection accuracy through a mixed precision quantization strategy, and dynamically selected detection models with different quantization precisions according to the scenario.

[0073] Mixed-precision quantization strategy: Based on the computational characteristics of domestic processors, a quantization method combining INT8 integer precision and FP16 half-floating-point precision is adopted. Through systematic comparative experiments, a comprehensive evaluation was conducted on the effectiveness of the two quantization precisions in maintaining the model ship detection capability, as well as their quantitative impact on video memory usage and inference speed, providing data support for subsequent precision configuration.

[0074] Dynamic precision configuration mechanism: By constructing a mathematical correlation model between quantization accuracy and model complexity, intelligent perception of application scenario requirements is achieved. In scenarios with limited resources or high real-time requirements, such as the initial screening of large-scale ships, the system automatically prioritizes the INT8 quantization solution to significantly reduce power consumption and increase detection speed. In scenarios with strict requirements on detection accuracy, such as those requiring high-precision ship identification, it automatically switches to FP16 quantization mode to ensure that the neural network can fully retain more information, thereby maintaining the overall accuracy of the ship detection task. This dynamic adaptation mechanism not only achieves the optimal configuration of computing resources, but also finds the best balance between model complexity and detection performance, providing flexible and efficient solutions for different application scenarios.

[0075] Based on the above embodiment, this embodiment describes step S103 in detail:

[0076] Acquire hardware status data; evaluate the hardware status data based on the noise suppression mechanism and weighted status assessment method to obtain the hardware data health index; dynamically adjust the input resolution based on the hardware data health index, optimize the parallel computing granularity, and construct an adaptive adjustment strategy; adjust the lightweight ship detection model based on the adaptive adjustment strategy to obtain an adaptive ship detection model.

[0077] Specifically,

[0078] This paper proposes a dynamic adaptive reasoning framework and optimization method based on multi-dimensional hardware perception. To address the dynamic fluctuation of hardware resources faced by domestic edge computing platforms in real-time target detection tasks, a closed-loop feedback control mechanism is used to achieve a dynamic balance between computing resources and detection performance. The specific method is as follows:

[0079] Multi-dimensional hardware status monitoring system;

[0080] Build a collaborative perception network for GPU memory, temperature, and power consumption to break through the limitations of traditional single-indicator monitoring.

[0081] Multi-source data collection. First, periodically sample hardware status indicators, and collect the peak memory usage value M at the current time t every 5 inference batches. t , GPU temperature T t and instantaneous power consumption B t .

[0082] Noise suppression mechanism. Sliding window filtering is used to smooth the collected hardware status indicators to suppress instantaneous fluctuation interference:

[0083]

[0084] Among them, P t is the current state parameter, In order to smooth the estimated value, the smoothing factor α is dynamically adjusted to achieve smoothing of the current state, effectively suppressing the false triggering problem of the subsequent processing process caused by the instantaneous fluctuation of hardware indicator parameters.

[0085] Weighted status assessment: Design a hardware health index that integrates multi-dimensional indicators and assigns differentiated weights.

[0086]

[0087] Among them, w i This is a weighting factor for different hardware operating states, assigned based on the characteristics of the actual hardware platform. For example, temperature-sensitive scenarios may be assigned a larger value for w2.

[0088] Dynamic parameter adaptive control strategy

[0089] Real-time optimization of model parameters based on hardware health index:

[0090] Dynamic adjustment of input resolution. Preset multiple levels of input image slice size (such as 640×640, 1280×1280), the adjustment strategy strictly follows the hardware adaptability principle, switch:

[0091] Low load mode: Enables small-size input (640×640), reduces video memory usage, power consumption, and detection accuracy.

[0092] High load mode: Switching to large-size input (1280×1280) increases video memory usage and power consumption, but improves detection accuracy.

[0093] Considering that high-resolution image input increases video memory usage and power consumption, the system only enables large-size slicing mode under specific conditions: when hardware resources are sufficient and the inspection task requires higher accuracy rather than inference speed. In routine inspection scenarios, the system prioritizes low-resolution image slices as model input. This strategy effectively reduces video memory pressure and energy consumption, enabling more optimal hardware resource allocation. By dynamically adjusting slice size, the system achieves an intelligent balance between computational accuracy and resource consumption.

[0094] Parallel computing granularity optimization;

[0095] Dynamically adjust the number of parallel processing B parallel With batch size B size . parallel and B size From small to large, the parallel granularity set B={B1(1,1),B2(1,2),...,B max (16,16)}, adjust the parallel granularity according to the current hardware health index:

[0096]

[0097] The concurrent scale of the algorithm execution is controlled by the dual threshold parameter α b and β b Collaborative control: The system dynamically adjusts the granularity of parallel computing by sensing the hardware load status in real time, ensuring the efficient use of computing resources while maintaining the long-term stable operation of the hardware system. The specially designed dual-threshold parameters have significant intervals. This differentiated setting effectively avoids the problem of hardware critical state oscillation caused by frequent switching of algorithm parallelism. When hardware resources are sufficient, the system enables high parallelism mode to improve processing efficiency; when resource constraints are detected, the parallelism is automatically reduced to ensure system stability. This mechanism achieves a smooth transition of parallel strategy switching through the buffer zone formed by parameter intervals, significantly reducing the risk of hardware status fluctuations.

[0098] The algorithm adaptively adjusts the decision-making strategy;

[0099] A full-link optimization system of "perception-decision-execution" has been built. The system consists of three core modules: the state perception layer, the decision engine layer, and the execution adaptation layer, forming a closed-loop control process. The state perception layer deeply connects to the underlying hardware through the embedded driver interface, and obtains multi-dimensional operating data such as video memory occupancy, temperature, and power consumption in real time to establish an accurate portrait of the hardware status. The decision engine layer has a built-in fuzzy logic rule library to perform pattern recognition and trend prediction on the perceived hardware status, and dynamically generate adjustment instructions for key parameters such as input resolution and parallelism. The execution adaptation layer reconfigures the model calculation graph structure in real time according to the decision instructions. By dynamically loading operator libraries of different precisions and adaptively adjusting the data pipeline, it achieves seamless switching between input slice size and parallel strategy, ensuring that the inference process always runs within the optimal configuration range of hardware resources.

[0100] This embodiment provides a remote sensing image processing method based on algorithm-hardware co-optimization. By constructing a hybrid heterogeneous computing platform that integrates CPU, GPU, and FPGA hardware, it achieves efficient, low-power, and highly stable remote sensing image ship detection. It dynamically adjusts input resolution and parallel granularity to optimize computing resource allocation based on hardware status and reduce power consumption. A mixed-precision quantization strategy is used to optimize model complexity and detection accuracy, maintaining the model's powerful feature expression capabilities. Through an algorithm-hardware co-optimization strategy, the characteristics of the heterogeneous hardware platform are fully utilized to achieve efficient, low-power, and highly stable remote sensing image ship detection. This improves detection efficiency and accuracy, reduces power consumption, enhances system stability, and provides a flexible and efficient solution for different application scenarios.

[0101] Based on the above embodiment, this embodiment describes a remote sensing image processing method based on algorithm hardware collaborative optimization, such as Figure 2 As shown, the details are as follows:

[0102] Hardware platform construction:

[0103] This method is based on a domestically produced heterogeneous hardware platform, specifically using Milink's MK7325FC FPGA development board and V-KOOL's "Smart Box 100" to deploy an efficient ship detection method. Figure 3 The MLK MK7325FC FPGA development board is a high-performance development platform based on FPGA chip design, with broad application value in high-speed communications, video processing, industrial control, and other fields. The "Smart Box 100" (AIEdge-MRU) is a domestically produced edge intelligent computing product independently developed by V-KOOL Information. Tables 1 and 2 show the configurations of the two aforementioned hardware components, respectively.

[0104] Table 1 Basic configuration of MK7325FC FPGA development board

[0105] name Specific parameters FPGA FPGA XC7K325T-2FFG900I size Core board 68 (mm) × 80 (mm) × 9.5 (mm), baseboard 200 (mm) × 115.5 (mm) Block RAM 16MB Operating frequency 125MHz interface PCIE, SFP+, SATA, HDMI, USB, Ethernet power supply DC-12V / 5A

[0106] Table 2 Basic configuration of "Smart Box 100"

[0107] name Specific parameters size 130mm×130mm×56mm CPU Feiteng E2000Q quad-core (2×FTC664+2×FTC310) GPU Tianshu Zhixin's domestically produced GPGPU RevMRU Video Memory 16 GB Power consumption Maximum power: ≤85W; Idle power: ≤30W Data Interface USB, HDMI, Gigabit Ethernet, serial port operating system openKylin 1.0

[0108] In the domestically produced heterogeneous hardware platform architecture proposed in this example, the MLK MK7325FC FPGA development board and the "Smart Box 100" are connected using a Gigabit Ethernet cable, while the CPU and GPU within the "Smart Box 100" are connected via PCIe. Furthermore, both hardware components can exchange data with external devices using the UDP / IP network protocol over Ethernet.

[0109] The MLK MK7325FC FPGA development board is used to build the NDWI algorithm-driven land-sea segmentation module, performing data preprocessing and laying the foundation for subsequent ship detection. The "Smart Box 100" is responsible for executing the host software, data flow, and the YOLOv5s-based ship detection module. By leveraging the processor's built-in hardware status monitoring component, it monitors hardware status in real time and dynamically adjusts the input image slice size and parallel computing granularity to control the computational load, ensuring efficient and stable operation of the detection model.

[0110] In summary, this method achieves the efficient operation of the ship detection system by building a domestic heterogeneous processing platform and rationally allocating computing tasks to different hardware components.

[0111] Development of land and sea segmentation module:

[0112] The FPGA development board uses a gigabit network cable to receive data from the sensor, parses the image data packets using the UDP / IP network protocol stack, extracts green and near-infrared (NIR) band data pixel by pixel, and then calculates the NDWI in a pipeline. The calculation results are saved as 8-bit fixed-point numbers:

[0113]

[0114] The segmentation threshold is trained in advance using historical data, and the NDWI is compared with the threshold pixel by pixel to output a binary land mask (1 for land and 0 for ocean).

[0115] The module processor is implemented in the hardware description language Verilog, using the Xilinx Vivado Design Suite 2023.1 development tool. After the Verilog code is implemented, a post-synthesis simulation verification strategy is implemented to ensure the correctness of the functional modules. Figure 4 The system architecture design of the FPGA module was demonstrated. Thanks to the FPGA's parallel computing advantages, this system achieved an ultra-low processing latency of 0.2ms. Compared to the traditional CPU / GPU architecture's "store first, then process" model, this model adopts a "transmit and process" pipeline architecture. This not only shortens end-to-end response time to milliseconds, but also significantly reduces memory usage requirements, providing an ideal solution for on-orbit computing scenarios with limited storage resources.

[0116] Quantitative optimization of ship detection model:

[0117] In view of the limited storage resources of domestic edge computing platforms, this method conducts in-depth quantization research on the original YOLOv5s object detection model. The original model uses the full-precision floating-point 32-bit (FP32) parameter format. Although its detection accuracy is high, it occupies a large amount of video memory and is difficult to adapt to the limited memory resources of domestic processors. To overcome this limitation, this paper introduces a mixed-precision quantization strategy:

[0118] Analysis of the FP16 quantization scheme. When using half-precision floating-point 16-bit (FP16) quantization, the model maintains detection performance that is highly consistent with the original FP32 model. In the comparative evaluation of four core indicators including precision, recall, F1 score and AP@0.5 value, the fluctuation range of each indicator of the quantized model is controlled within 0.5%. This quantization method effectively reduces the memory usage while maintaining the model's powerful feature expression capabilities. Energy efficiency tests show that the FP16 model exhibits significant advantages over the original model: single-frame processing time is shortened by 21%, and GPU power consumption is reduced by 14%. More noteworthy is that its energy efficiency is improved by 52% compared to the original model, which has significant engineering application value in continuously operating ship detection systems.

[0119] Evaluation of the INT8 quantization scheme. Further adopting 8-bit integer (INT8) quantization, the model demonstrated extreme computational efficiency. Experimental data showed that INT8 quantization increased single-frame processing speed by 40% and reduced power consumption by up to 45%. However, this aggressive quantization strategy resulted in excessive compression of model feature information, manifested in a 20.7% increase in the missed detection rate of target ships and a 6.2% decrease in AP@0.5 compared to the baseline FP32 model. This contradiction between accuracy loss and efficiency improvement indicates that the INT8 quantization scheme must be strictly limited to specific application scenarios and is particularly unsuitable for detection tasks with stringent requirements for detection accuracy.

[0120] Comprehensive experimental results demonstrate a clear boundary for selecting quantization schemes in ship detection systems: when detection accuracy is the system priority, FP16 quantization provides an ideal balance between accuracy and power consumption; when the detection task allows for a certain loss of accuracy in exchange for extreme energy efficiency, INT8 quantization significantly improves system throughput. This quantization decision framework provides flexible configuration options for different application scenarios, effectively expanding the applicability of the YOLOv5s model on domestic hardware platforms.

[0121] Development of dynamic adaptive control methods based on hardware perception:

[0122] In order to achieve stable and efficient operation of domestic heterogeneous computing platforms under complex working conditions, the present invention constructs a dynamic adaptive control system of the algorithm of "perception-evaluation-control", such as Figure 5 As shown, its core consists of three collaborative modules:

[0123] Hardware status awareness module: Utilize the built-in hardware status monitoring component "ixsmi" of the "Smart Box 100" to obtain GPU memory usage, power consumption, and temperature in real time.

[0124] Data processing and status assessment module: First, the collected multi-dimensional hardware operation status data values ​​are pre-processed and smoothed using sliding window filtering to suppress the false triggering problem of subsequent processing caused by instantaneous fluctuations in hardware indicator parameters.

[0125]

[0126] Calculate the hardware health index using the hardware operating status:

[0127]

[0128] Given the extreme temperature sensitivity of domestic processors, a higher weight (0.6) was assigned to the temperature indicator to prevent overheating, which could lead to slow performance and system crashes. Each indicator was normalized to a maximum memory capacity of 16,000MB; a room temperature of 25°C; the maximum safe operating temperature for the "Smart Box 100" was 90°C; and a maximum power consumption of 85W.

[0129] The algorithm's dynamic adaptive control module realizes real-time optimization of model parameters based on hardware health index and dynamically adjusts the resolution of input image slices. score <0.2 (hardware resources are in surplus state), and the detection task has low accuracy requirements and focuses more on inference speed, the large-size slice mode will be activated. In conventional detection scenarios, the system prioritizes low-resolution image slices as model input. Parallel computing granularity optimization, dynamic adjustment of the number of parallel processing B parallel With batch size B size . parallel ∈{1,2,3,...,16} and B size ∈{1,2,4,8,16} is composed of a parallel granularity set B from small to large:

[0130] B={B1(1,1),B2(1,2),B3(1,4),B4(2,2),B5(2,4),B6(2,8),...,B max (16,16)}

[0131] Adjust the parallel granularity based on the current hardware health index:

[0132]

[0133] The system senses the hardware load status in real time and dynamically adjusts the number of parallel task units, ensuring efficient use of computing resources and maintaining long-term and stable operation of the hardware system.

[0134] Test verification and system integration:

[0135] To fully validate the system's performance, this method uses an authoritative public ship detection dataset for model training and testing. This dataset contains 13,435 high-precision aerial remote sensing images with 26,048 annotated ship target instances, covering a range of scales from yachts to large cargo ships. All annotations use the YOLO standard bounding box format, ensuring that the detection algorithm effectively captures the spatial distribution characteristics of the targets. Of particular note is that the dataset, acquired through multi-source sensors, encompasses complex scenes with varying lighting conditions and sea state levels. This diversity enables the trained model to possess excellent generalization capabilities, enabling it to accurately identify ship targets of various types and sizes within the images.

[0136] Model performance is evaluated using a multi-dimensional indicator system: precision (PC) measures the proportion of correct detections, recall (RC) reflects the degree of target coverage, the F1 score combines the performance of the first two, and AP@0.5 focuses on the average detection accuracy when the intersection over union (IOU) threshold is 0.5.

[0137]

[0138] TP represents the number of correctly identified ships, FP records the number of falsely detected non-ship targets, and FN counts the number of real ships that were missed.

[0139] In terms of hardware characteristics, the focus is on computing speed, resource utilization, and energy efficiency. The computing speed test uses a standard test set to measure frames per second (FPS); resource utilization covers video memory and power consumption; and energy efficiency uses the image processing frames per second per unit power consumption (FPS / W) as a core indicator to comprehensively quantify the computing characteristics of the hardware platform.

[0140] During the system integration phase, the land-sea segmentation module and the ship detection model were deployed on a domestically produced heterogeneous hardware platform. During the joint debugging and testing phase, automated test scripts were developed to simulate continuous detection tasks in high-concurrency scenarios and verify the stability of the system under long-term operation. Figure 6 Through algorithm-hardware co-optimization, this method fully leverages the architectural advantages of domestic hardware platforms, achieving an average power consumption of less than 30W while maintaining detection accuracy (less than 2% loss compared to pre-optimization). This provides technical support for building a low-power, intelligent ocean detection system, demonstrating its application value in the fields of maritime rights protection and national defense security.

[0141] Please refer to Figure 7 , Figure 7 This is a block diagram of a remote sensing image processing device based on algorithm-hardware collaborative optimization provided by an embodiment of the present invention; the specific device may include:

[0142] The image processing module 100 uses a pre-built hybrid heterogeneous computing model to perform land and sea segmentation processing on the remote sensing image to be detected to obtain a land mask image;

[0143] The lightweight processing module 200 performs lightweight processing on the YOLOv5s architecture based on a mixed precision quantization strategy and a dynamic precision matching mechanism to obtain a lightweight model for ship detection;

[0144] An adaptive module 300 controls the lightweight ship detection model based on a dynamic parameter adaptive control strategy to obtain an adaptive ship detection model;

[0145] The detection module 400 processes the remote sensing image to be detected using the land mask image and the adaptive ship detection model to obtain a ship detection result.

[0146] A remote sensing image processing device based on algorithm-hardware collaborative optimization of this embodiment is used to implement the aforementioned remote sensing image processing method based on algorithm-hardware collaborative optimization. Therefore, the specific implementation method of a remote sensing image processing device based on algorithm-hardware collaborative optimization can be seen in the embodiment part of a remote sensing image processing method based on algorithm-hardware collaborative optimization in the previous text. For example, the image processing module 100, the lightweight processing module 200, the adaptive module 300, and the detection module 400 are respectively used to implement steps S101, S102, S103, and S104 in the aforementioned remote sensing image processing method based on algorithm-hardware collaborative optimization. Therefore, its specific implementation method can refer to the description of the corresponding embodiments of each part, and will not be repeated here.

[0147] In order to implement the above embodiments, the present application also proposes an electronic device, comprising: a processor, and a memory communicatively connected to the processor; the memory stores computer-executable instructions; the processor executes the computer-executable instructions stored in the memory to implement the method provided by the above embodiments.

[0148] In order to implement the above embodiments, the present application also proposes a computer-readable storage medium, in which computer-executable instructions are stored. When the computer-executable instructions are executed by a processor, they are used to implement the methods provided by the above embodiments.

[0149] In order to implement the above embodiments, the present application also proposes a computer program product, including a computer program, which implements the methods provided by the above embodiments when executed by a processor.

[0150] The collection, storage, use, processing, transmission, provision and disclosure of user personal information involved in this application are in compliance with relevant laws and regulations and do not violate public order and good morals.

[0151] It is important to note that personal information collected from users should be used for legitimate and reasonable purposes and should not be shared or sold beyond these legitimate uses. Furthermore, such collection / sharing should be conducted only after receiving the user's informed consent, including but not limited to notifying the user to read the user agreement / user notice and sign an agreement / authorization that includes the relevant user information before using the feature. Furthermore, any necessary steps must be taken to safeguard and secure access to such personal information and ensure that others with access to personal information comply with its privacy policy and procedures.

[0152] This application contemplates providing implementations that allow users to selectively block the use or access of personal information data. Specifically, this disclosure contemplates providing hardware and / or software to prevent or block access to such personal information data. Risks can be minimized by limiting data collection and deleting data once it is no longer needed. Furthermore, where applicable, such personal information can be de-identified to protect user privacy.

[0153] In the descriptions of the foregoing embodiments, the reference terms "one embodiment", "some embodiments", "example", "specific example", or "some examples" mean that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present application. In this specification, the schematic expressions of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described may be combined in any one or more embodiments or examples in a suitable manner. In addition, those skilled in the art may combine and combine the different embodiments or examples described in this specification and the features of the different embodiments or examples, unless they are mutually inconsistent.

[0154] Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of the technical features being referred to. Thus, a feature defined as "first" or "second" may explicitly or implicitly include at least one of such features. Throughout the description of this application, "plurality" means at least two, for example, two, three, etc., unless otherwise specifically defined.

[0155] Any process or method description in a flowchart or otherwise described herein may be understood to represent a module, segment or portion of code comprising one or more executable instructions for implementing the steps of a custom logical function or process, and the scope of the preferred embodiments of the present application includes alternative implementations in which functions may be performed out of the order shown or discussed, including performing functions in a substantially simultaneous manner or in the reverse order depending on the functions involved, which should be understood by those skilled in the art to which the embodiments of the present application belong.

[0156] The logic and / or steps represented in the flowcharts or otherwise described herein, for example, can be considered as a sequenced list of executable instructions for implementing the logical functions, and can be embodied in any computer-readable medium for use by, or in conjunction with, an instruction execution system, apparatus, or device (e.g., a computer-based system, a system including a processor, or other system that can fetch and execute instructions from an instruction execution system, apparatus, or device). For purposes of this specification, a "computer-readable medium" can be any device that can contain, store, communicate, propagate, or transport a program for use by, or in conjunction with, an instruction execution system, apparatus, or device. More specific examples (a non-exhaustive list) of computer-readable media include the following: an electrical connection with one or more wires (electronic devices), a portable computer disk cartridge (magnetic device), random access memory (RAM), read-only memory (ROM), erasable and programmable read-only memory (EPROM or flash memory), fiber optic devices, and a portable compact disc read-only memory (CDROM). Furthermore, the computer-readable medium may even be paper or other suitable medium on which the program is printed, since the program may be obtained electronically, for example, by optically scanning the paper or other medium and then editing, interpreting or processing it in another suitable manner if necessary, and then storing it in a computer memory.

[0157] It should be understood that various parts of the present application can be implemented using hardware, software, firmware, or a combination thereof. In the above embodiments, multiple steps or methods can be implemented using software or firmware stored in a memory and executed by a suitable instruction execution system. For example, if implemented using hardware, as in another embodiment, any one of the following technologies known in the art or a combination thereof can be used to implement: a discrete logic circuit having a logic gate circuit for implementing a logic function on a data signal, an application-specific integrated circuit having a suitable combination of logic gate circuits, a programmable gate array (PGA), a field programmable gate array (FPGA), etc.

[0158] Those skilled in the art will understand that all or part of the steps in the method of the above embodiment can be completed by instructing related hardware through a program, and the program can be stored in a computer-readable storage medium. When the program is executed, it includes one or a combination of the steps of the method embodiment.

[0159] In addition, the functional units in the various embodiments of the present application may be integrated into a processing module, or each unit may exist physically separately, or two or more units may be integrated into a module. The above-mentioned integrated module may be implemented in the form of hardware or in the form of a software functional module. If the integrated module is implemented in the form of a software functional module and sold or used as an independent product, it may also be stored in a computer-readable storage medium.

[0160] The storage medium mentioned above may be a read-only memory, a magnetic disk, or an optical disk, etc. Although the embodiments of the present application have been shown and described above, it is understood that the above embodiments are exemplary and should not be construed as limiting the present application. Persons skilled in the art may make changes, modifications, substitutions, and variations to the above embodiments within the scope of the present application.

Claims

1. A remote sensing image processing method based on algorithm and hardware collaborative optimization, characterized in that: include: Use the pre-built hybrid heterogeneous computing model to perform land and sea segmentation on the remote sensing image to be detected and obtain the land mask image; The YOLOv5s architecture is lightweighted based on the mixed precision quantization strategy and dynamic precision matching mechanism to obtain a lightweight model for ship detection. The ship detection lightweight model is regulated based on a dynamic parameter adaptive control strategy to obtain an adaptive ship detection model; The remote sensing image to be detected is processed using the land mask image and the adaptive ship detection model to obtain a ship detection result.

2. The remote sensing image processing method based on algorithm and hardware collaborative optimization according to claim 1, characterized in that: The method of performing land-sea segmentation processing on the remote sensing image to be detected by using the pre-built hybrid heterogeneous computing model to obtain the land mask image includes: Acquire the remote sensing image to be detected; parsing the remote sensing image to be detected based on a network protocol stack to obtain a parsed remote sensing image; Extracting green light band data and near infrared band data from the analyzed remote sensing image; Calculating a normalized ratio based on the green light band data and the near infrared band data; The segmentation threshold trained in advance using historical data is compared with the normalized ratio to obtain a binary land mask.

3. The remote sensing image processing method based on algorithm and hardware collaborative optimization according to claim 1, characterized in that: The lightweight model for ship detection is obtained by lightweight processing the YOLOv5s architecture based on the mixed precision quantization strategy and dynamic precision matching mechanism, including: Based on the computing characteristics of the processor, the YOLOv5s architecture is quantized by combining integer precision and half-floating point precision to obtain two quantization accuracy results; Based on the two quantization accuracy results, a mathematical correlation model between quantization accuracy and model complexity is constructed, and a dynamic matching mechanism is added to obtain a lightweight model for ship detection.

4. The remote sensing image processing method based on algorithm and hardware collaborative optimization according to claim 1, characterized in that: The controlling the lightweight ship detection model based on the dynamic parameter adaptive control strategy to obtain the adaptive ship detection model includes: Get hardware status data; Evaluate the hardware status data based on a noise suppression mechanism and a weighted status evaluation method to obtain a hardware data health index; Dynamically adjust input resolution based on the hardware data health index, optimize parallel computing granularity, and build an adaptive adjustment strategy; The ship detection lightweight model is adjusted based on the adaptive adjustment strategy to obtain an adaptive ship detection model.

5. The remote sensing image processing method based on algorithm and hardware collaborative optimization according to claim 4, characterized in that: The obtaining of hardware status data includes: A three-state collaborative perception network for graphics processor memory, temperature, and power consumption is constructed, and the hardware status indicators are periodically sampled using the three-state collaborative perception network.

6. The remote sensing image processing method based on algorithm and hardware collaborative optimization according to claim 4, characterized in that: The noise suppression mechanism includes: using sliding window filtering to smooth the collected hardware status data to suppress instantaneous fluctuation interference. The calculation formula is: Among them, S t is the current state parameter, is the estimated value after smoothing, and α is the dynamically adjusted smoothing factor.

7. The remote sensing image processing method based on algorithm and hardware collaborative optimization according to claim 4, characterized in that: The parallel computing granularity optimization includes: Dynamically adjust the number of parallel processing and batch size, and organize the number of parallel processing and batch size into a set of parallel granularities from small to large; Based on the current hardware health index, adjust the parallel granularity. The calculation formula is: Where B={B1(1,1),B2(1,2),...,B max (16,16)} is the parallel granularity set, α b and β b is the dual threshold parameter.

8. A remote sensing image processing device based on algorithm and hardware collaborative optimization, characterized in that: include: The image processing module uses a pre-built hybrid heterogeneous computing model to perform land and sea segmentation on the remote sensing image to be detected and obtain a land mask image; The lightweight processing module performs lightweight processing on the YOLOv5s architecture based on a mixed-precision quantization strategy and a dynamic precision matching mechanism to obtain a lightweight model for ship detection. An adaptive module controls the lightweight ship detection model based on a dynamic parameter adaptive control strategy to obtain an adaptive ship detection model; The detection module processes the remote sensing image to be detected using the land mask image and the adaptive ship detection model to obtain a ship detection result.

9. An electronic device, characterized in that: include: a processor, and a memory communicatively connected to the processor; The memory stores computer-executable instructions; The processor executes the computer-executable instructions stored in the memory to implement the method according to any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that The computer-readable storage medium stores computer-executable instructions, which are used to implement the method according to any one of claims 1 to 7 when executed by a processor.

Images