Remote sensing image thumbnail processing method, system and device and storage medium

Abstract

The invention relates to the technical field of data processing, and particularly provides a remote sensing image thumbnail processing method, system and device and a storage medium, and the method comprises the steps: generating a thumbnail for a remote sensing image original file; packaging the original remote sensing image file and the thumbnail into a file in a self-defined format; obtaining basic information of an original file of the remote sensing image and the thumbnail, and constructing a file header structure based on the basic information; uploading the file to an object storage service; and querying the thumbnail from the object storage service based on the file header structure. According to the method, the ultra-high-definition remote sensing image Tiff file thumbnail can be quickly displayed on the browser side.

Description

Technical Field

[0001] The present invention belongs to the technical field of data processing, and in particular relates to a remote sensing image thumbnail processing method, system, device and storage medium. Background Art

[0002] Tiff (Tag Image File Format) image file is one of the commonly used formats in graphic image processing. Its image format is very complex, but it is widely used because it can store image information flexibly and variably, supports many color systems, and is independent of the operating system. In various geographic information systems, photogrammetry and remote sensing applications, images are required to have geographic coding information, such as the coordinate system of the image, scale, coordinates of points on the image, longitude and latitude, length units and angle units, etc.

[0003] In systems involving ultra-high-definition remote sensing images, the size of remote sensing image Tiff files is usually several hundred MB, and can be as large as 4 GB. The display of Tiff files has always been a difficulty and pain point. On the one hand, when displaying Tiff files on the Web, the browser does not support the display of Tiff files, and usually needs to be converted to Jpeg files in real time for display. Due to the large size of the files, it consumes more memory. On the other hand, since Tiff files are hundreds of MB to several GB, network transmission also takes a considerable amount of time. Especially when displaying remote sensing image lists on the Web, the above problems are more prominent. Summary of the invention

[0004] In view of the above-mentioned deficiencies in the prior art, the present invention provides a remote sensing image thumbnail processing method, system, device and storage medium to solve the above-mentioned technical problems.

[0005] In a first aspect, the present invention provides a remote sensing image thumbnail processing method, comprising: Generate thumbnails for remote sensing image raw files; Encapsulating the remote sensing image original file and the thumbnail into a file of a custom format; Obtaining basic information of the remote sensing image original file and the thumbnail, and constructing a file header structure based on the basic information; Upload the file to the object storage service; The thumbnail is queried from the object storage service based on the file header structure.

[0006] In an optional implementation, generating a thumbnail for an original remote sensing image file includes: Use the image processing library to load remote sensing image raw files; Preprocessing the image, including denoising, correcting color balance, and enhancing contrast, and retaining metadata of geographic coordinate information during the preprocessing stage; Determine the final size of the thumbnail based on the application scenario and storage limitations; Apply an image scaling algorithm to scale the original image to the specified thumbnail size; Perform color space conversion on the scaled image to ensure color consistency.

[0007] In an optional implementation, packaging the remote sensing image original file and the thumbnail into a file in a custom format includes: Initialize a Blob object, wherein the Blob object includes the remote sensing image original file and the thumbnail; Read the data of the Blob object, use the image processing function to fill the read data with zeros and encapsulate it, and convert the processed data back to the Blob object; Extract the thumbnail from the Blob object through a predefined function and process the boundary value of the thumbnail.

[0008] In an optional implementation, basic information of the remote sensing image original file and the thumbnail is obtained, and a file header structure is constructed based on the basic information, including: Write the file size, occupied space, start byte, end byte, and file MIME type in the file header structure.

[0009] In an optional implementation, uploading the file to an object storage service includes: For the data stream of uploading the file to the object storage service, the first 10 bytes and the last 10 bytes are extracted for every MB, and the MD5 value of the extracted bytes is calculated, and the integrity of the file is verified based on the MD5 value.

[0010] In a second aspect, the present invention provides a remote sensing image thumbnail processing system, comprising: A thumbnail generation module is used to generate thumbnails for original remote sensing image files; A file packaging module, used for packaging the remote sensing image original file and the thumbnail into a file of a custom format; A file header construction module, used to obtain basic information of the remote sensing image original file and the thumbnail, and construct a file header structure based on the basic information; A file upload module, used to upload the file to the object storage service; An object query module is used to query the thumbnail from the object storage service based on the file header structure.

[0011] In an optional implementation, the thumbnail generation module includes: An image loading unit, used to load the original remote sensing image file using an image processing library; A preprocessing unit, used to preprocess the image, wherein the preprocessing includes denoising, correcting color balance, and enhancing contrast, and metadata of geographic coordinate information is retained during the preprocessing stage; A size determination unit, used to determine the final size of the thumbnail according to the application scenario and storage limitations; A resizing unit, for applying an image resizing algorithm to resize the original image to a specified thumbnail size; The color conversion unit is used to perform color space conversion on the scaled image to ensure color consistency.

[0012] In an optional implementation, the file encapsulation module includes: An initialization module, used for initializing a Blob object, wherein the Blob object includes the original remote sensing image file and the thumbnail; The file processing module is used to read the data of the Blob object, and use the image processing function to fill the read data with zeros and encapsulate it, and convert the processed data back to the Blob object; The boundary processing module is used to extract a thumbnail from a Blob object through a predefined function and process a boundary value of the thumbnail.

[0013] In a third aspect, a device is provided, including: A memory, used for storing a remote sensing image thumbnail processing program; The processor is used to implement the steps of the remote sensing image thumbnail processing method provided in the first aspect when executing the remote sensing image thumbnail processing program.

[0014] In a fourth aspect, a computer-readable storage medium is provided, on which a remote sensing image thumbnail processing program is stored. When the remote sensing image thumbnail processing program is executed by a processor, the steps of the remote sensing image thumbnail processing method provided in the first aspect are implemented.

[0015] The beneficial effects of the present invention lie in that the remote sensing image thumbnail processing method, system, device and storage medium provided by the present invention use JavaScript, file stream processing, HTTP Range, custom file format and other related technologies to quickly display the implementation method of ultra-high-definition remote sensing image Tiff file thumbnails, which can quickly display ultra-high-definition remote sensing image Tiff file thumbnails on the browser side, while the code logic is simpler and clearer, the decoupling is strong, the development cycle is shorter, and resources are saved.

[0016] In addition, the invention has a reliable design principle, a simple structure and a very broad application prospect. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, for ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0018] Figure 1 is a schematic flow chart of a method according to an embodiment of the present invention.

[0019] Figure 2 It is a schematic diagram of the file structure of a method according to an embodiment of the present invention.

[0020] Figure 3 is another schematic flow chart of a method according to an embodiment of the present invention.

[0021] Figure 4 is a schematic block diagram of a system according to an embodiment of the present invention.

[0022] Figure 5 A schematic diagram of the structure of a device provided in an embodiment of the present invention. DETAILED DESCRIPTION

[0023] In order to enable those skilled in the art to better understand the technical solutions in the present invention, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present invention.

[0024] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as those commonly understood by those skilled in the art of the present invention. The terms used in the specification of the present invention herein are only for the purpose of describing specific embodiments and are not intended to limit the present invention.

[0025] The remote sensing image thumbnail processing method provided in the embodiment of the present invention is executed by a computer device, and accordingly, the remote sensing image thumbnail processing system runs in the computer device.

[0026] Figure 1 is a schematic flow chart of a method according to an embodiment of the present invention. Figure 1 The execution subject may be a remote sensing image thumbnail processing system. According to different requirements, the order of the steps in the flow chart may be changed, and some may be omitted.

[0027] like Figure 1 As shown, the method includes: S1. Generate thumbnails for remote sensing image raw files.

[0028] First, read the original file of the remote sensing image, which may contain high-resolution surface images taken by satellites or drones. Use image processing algorithms, such as downsampling, image compression or scaling technology, to reduce the size of the original image to a size that is more convenient for fast preview and loading, thereby generating a thumbnail file. This thumbnail file retains the key visual features of the original image but takes up less storage space.

[0029] S2. Encapsulating the remote sensing image original file and the thumbnail into a file in a custom format.

[0030] The generated thumbnail file is packaged together with the original remote sensing image file into a file of a custom format through a specific packaging process. This process may include adding specific metadata tags to the two files, encrypting or compressing them, and organizing them according to a specific file structure (such as header information, index table, and data block) to ensure the integrity and security of the files and facilitate subsequent processing and access.

[0031] S3. Obtain basic information of the remote sensing image original file and the thumbnail, and construct a file header structure based on the basic information.

[0032] Extract key information from remote sensing image original files and thumbnail files, such as image size, resolution, color mode, shooting time, geographic location coordinates and other basic information. This information is used to build a file header structure, which contains metadata for identifying, retrieving and managing these files. The file header structure is designed to improve file readability and processing efficiency while ensuring the accuracy and integrity of the information.

[0033] S4. Upload the file to the object storage service.

[0034] After completing the encapsulation and file header structure, the next step is to upload the file in the custom format to the object storage service. The object storage service is an efficient and scalable cloud storage solution that can handle large amounts of unstructured data, such as images, videos, and documents. During the upload process, you may need to use an API or SDK to interact with the object storage service and ensure that the file is correctly stored and indexed for quick access later.

[0035] S5. Query the thumbnail from the object storage service based on the file header structure.

[0036] Based on the information in the previously constructed file header structure, the required thumbnail files are efficiently retrieved and accessed through the query interface provided by the object storage service. This step may involve parsing the metadata in the file header structure and using appropriate query parameters (such as file name, tag, or index) to locate the thumbnail file stored in the object storage service. Once found, the thumbnail can be downloaded or directly displayed on the client so that users can quickly browse and identify the content of the remote sensing image.

[0037] In an embodiment of the present invention, based on step S1, a possible embodiment is given below to illustrate its specific implementation scheme in a non-limiting manner.

[0038] S101. Preprocess Tiff file: Before starting to generate thumbnails, the Tiff file is preprocessed, such as checking the file integrity, reading the file header information, etc., to ensure the smooth progress of subsequent processing.

[0039] If the Tiff file contains multiple pages (multi-page Tiff), you need to selectively read the pages for which thumbnails need to be generated, or generate thumbnails for all pages.

[0040] S102. Optimize Canvas usage: When using Canvas for image processing, set the size of the Canvas appropriately to avoid unnecessary memory consumption.

[0041] When drawing Tiff pages to Canvas, you can scale the Tiff pages to a smaller size before drawing them to Canvas to reduce memory usage.

[0042] S103.Memory Management: In the process of generating thumbnails, periodically release memory resources that are no longer in use, such as releasing the Canvas context that has been drawn.

[0043] If you are processing large Tiff files, you can consider splitting the processing into multiple small tasks and releasing the memory after each task is completed to avoid memory overflow.

[0044] S104. Error handling: Add error handling logic to the script, such as capturing and handling file reading errors, Canvas drawing errors, etc., to improve the robustness of the script.

[0045] S105. Performance optimization: If possible, use Web Workers to process image data in parallel to increase the speed of thumbnail generation.

[0046] Batch the drawing operations of Canvas to reduce the number of times the browser redraws and improve performance.

[0047] The following is an example of a custom script that generates Tiff thumbnails: / / Assume that Tiffz-js library is loaded and available / / Import the Tiffz-js library (here we assume that <script>标签引入） / / <script src="path / to / tiffz-js.js">< / script> / / Custom function: Generate Tiff thumbnail function generateTiffThumbnail(tiffFile,outputCanvasId, callback) { / / Create a FileReader object to read the Tiff file var reader = new FileReader(); / / Event handler triggered when file reading is complete reader.onload = function(event) { var arrayBuffer =event.target.result; / / Use Tiffz-js library to parse Tiff file Tiff.decode(arrayBuffer, function(tiff){ / / Assume that only the first page of the Tiff file is processed var firstPage = tiff.directory.entries[0]; / / Create a new Canvas element var canvas = document.getElementById(outputCanvasId); var ctx = canvas.getContext('2d'); / / Set the size of the Canvas (here we assume that the thumbnail size is 1 / 10 of the original size) varthumbnailWidth = firstPage.width / 10; var thumbnailHeight = firstPage.height / 10; canvas.width = thumbnailWidth; canvas.height = thumbnailHeight; / / Use the Tiffz-js library to draw Tiff pages onto Canvas (here the Tiffz-js library needs to support this function) / / Note: drawPageToCanvas here is a hypothetical function. When actually used, it needs to be replaced with the corresponding function provided by the Tiffz-js library / / The Tiffz-js library may not support drawing Tiff pages directly onto Canvas, so some additional processing is required here / / For example, you can convert the Tiff pages into ImageData objects first, and then draw them on Canvas / / drawPageToCanvas(ctx, firstPage, 0, 0, thumbnailWidth, thumbnailHeight); / / Since the Tiffz-js library may not support direct drawing, here is an alternative: / / Use other libraries (such as pdf-lib or similar image processing libraries) to convert Tiff pages to ImageData / / Or convert Tiff to Png / Jpeg and other formats first, and then use Canvas to draw / / var imageData = convertTiffPageToImageData(firstPage, thumbnailWidth, thumbnailHeight); / / ctx.putImageData(imageData, 0, 0); / / Assume that the Tiff pages have been successfully drawn on the Canvas / / Convert Canvas to Jpeg thumbnail (optional) / / Here you can use the toDataURL method to convert the Canvas content into a Data URL in Jpeg format varthumbnailDataUrl = canvas.toDataURL('image / jpeg', 0.8); / / 0.8 represents JPEG quality (between 0 and 1) / / Call the callback function (if any) and pass it the thumbnail's Data URL if (callback) { callback (thumbnailDataUrl);}}, function (error) { console.error ('Error decoding TIFF:', error);});}; / / Read Tiff file as ArrayBuffer reader.readAsArrayBuffer(tiffFile);} / / Assume there is a <input type="file"> Element is used to select Tiff files / / and a <canvas>The element is used to display thumbnails / / <input type="file" id="tiffFileInput"> / / <canvas id="thumbnailCanvas">< / canvas> document.getElementById('tiffFileInput').addEventListener('change', function(event) { var tiffFile =event.target.files[0]; if (tiffFile&&tiffFile.type === 'image / tiff') {generateTiffThumbnail(tiffFile, 'thumbnailCanvas', function(thumbnailDataUrl){ / / Here you can use thumbnailDataUrl to display the thumbnail / / For example, set it to a element's src attribute console.log('Thumbnailgenerated:', thumbnailDataUrl);});} else { console.warn('Please select a valid TIFF file.');}}).

[0048] The toDataURL method is used in the script example above to convert the Canvas content into a DataURL in Jpeg format, which can be used to display the thumbnail on a web page or send it to a server for storage. If you need to save the thumbnail as a file, you may need to use other methods (such as the File API) to create a Blob object and trigger the file download.

[0049] In order to improve the quality of thumbnails, the following thumbnail generation method is used: Load remote sensing image raw files: Use appropriate image processing libraries to load remote sensing image files (such as GeoTIFF, JPEG2000, etc.).

[0050] Make sure the library can read and parse the geo-reference information and other metadata in the image files.

[0051] Image preprocessing: Denoising: Apply denoising algorithms (such as Gaussian filtering, median filtering, etc.) to reduce the noise in the image.

[0052] Correct color balance: Adjust the brightness, contrast, and saturation of an image to ensure color authenticity and consistency. This may require the use of specific color correction algorithms or reference data.

[0053] Enhance contrast: Use techniques such as histogram equalization and adaptive contrast enhancement to improve the visual effect of the image.

[0054] Preserve Geo-reference Information: During pre-processing, ensure that geo-reference information and other metadata are correctly preserved and associated with the image.

[0055] Determine thumbnail size: Determine the final size of the thumbnail based on the application scenario (such as quick browsing, map service, data analysis, etc.) and storage limitations (such as file size, loading speed, etc.).

[0056] Choose an appropriate zoom factor, taking into account the resolution and geographic coverage of the image.

[0057] Apply the image scaling algorithm: Scales the original image to the specified thumbnail size using a high-quality image scaling algorithm (such as bilinear interpolation, bicubic interpolation, Lanczos resampling, etc.).

[0058] Ensure that the scaled image maintains clarity and geographic accuracy.

[0059] Color space conversion: Convert the scaled image from its original color space (such as RGB, CMYK, etc.) to a color space more suitable for display and storage (such as sRGB, JPEG color space, etc.).

[0060] This helps ensure that colors are consistent across devices and browsers.

[0061] Generate and save thumbnails: Save the preprocessed and scaled image in a thumbnail format (such as JPEG, PNG, etc.).

[0062] Associate geographic coordinate information and other metadata with thumbnail files, if possible, or save them in a separate metadata file.

[0063] In an embodiment of the present invention, based on step S2, a possible embodiment is given below to illustrate its specific implementation scheme in a non-limiting manner.

[0064] S201. File packaging process Blob binary file stream processing: Use Tiffz-js library or custom script to read Blob binary file stream.

[0065] Parse the file stream to identify its format and content.

[0066] Zero padding operation: In some cases, you may need to pad file data with zeros to ensure that the file meets certain storage or transmission requirements.

[0067] Zero padding typically involves adding extra zero bytes to the end of a file to reach a desired file size or to meet specific alignment requirements.

[0068] Packaging operation: Encapsulate the processed file data into Tiffz format (if it is not already in this format).

[0069] The Tiffz format is an extension of the Tiff format that supports storing multiple Tiff images as part of a single file while retaining the independence and integrity of each image.

[0070] Specifically, the process includes the following: 1. Initialize the Blob file First, you need a Blob object, which contains the TIFF file data to be processed.

[0071] / / Assume there is already a Blob object let blob = ...; / / Blob object of TIFF file 2. Read Blob Data Use FileReader to read Blob data for subsequent processing.

[0072] let reader = new FileReader(); reader.onload = function(event) { letarrayBuffer = event.target.result; processTiffData(arrayBuffer);};reader.readAsArrayBuffer(blob); 3. Processing TIFF Data Define a function processTiffData to process TIFF data. This function will include operations such as zero padding and encapsulation.

[0073] function processTiffData(arrayBuffer) { let uint8Array = newUint8Array(arrayBuffer); let tiffHeader = parseTiffHeader(uint8Array); / / Assume tiffHeader contains the necessary TIFF header information / / Perform zero padding (if necessary) let paddedArray= padArray(uint8Array, tiffHeader); / / Perform encapsulation operations (e.g. add custom tags, modify directories, etc.) let encapsulatedArray = encapsulateTiff(paddedArray, tiffHeader); / / Special case handling: calculate thumbnail size and handle boundaries let thumbnailData = extractThumbnail(encapsulatedArray); let resizedThumbnail = handleThumbnailBoundaries(thumbnailData); / / Convert the processed data back to a Blob object let newBlob = new Blob([resizedThumbnail], { type: 'image / tiff'}); / / Upload uploadFile(newBlob);} 4. Parsing TIFF header information Define a function parseTiffHeader to parse the header information of the TIFF file.

[0074] function parseTiffHeader(uint8Array) { / / Assume that the header information of the TIFF file is located in the first few bytes of the array / / This is just a simple example, the actual TIFF header parsing will be more complicated lettiffHeader = { byteOrder: uint8Array[0] === 0x49&&uint8Array[1] === 0x49 ? 'II' : 'MM', offset: (uint8Array[2]<<8) + uint8Array[3]}; return tiffHeader;} 5. Zero padding operation Define a function padArray to perform zero padding (if necessary).

[0075] function padArray(uint8Array, header) { / / According to the TIFF specification, some fields may require specific alignment / / This is just a simple example, the actual zero-padding logic will be more complicated let paddingSize = 2 - (uint8Array.length % 2); / / Assume that 2-byte alignment is required let paddedArray = newUint8Array(uint8Array.length + paddingSize); paddedArray.set(uint8Array); for(let i = 0; i <paddingSize; i++) { paddedArray[uint8Array.length + i] = 0;}return paddedArray;} 6. Packaging operation Define a function encapsulateTiff to perform the encapsulation operation of TIFF files.

[0076] function encapsulateTiff(uint8Array, header) { / / Add or modify directory entries, tags, etc. according to the TIFF specification / / This is just a simple example, the actual encapsulation logic will be more complicated / / For example, you can add custom tags or modify existing tag values ​​return uint8Array; / / Assume no actual modifications are made, just return directly} 7. Extract and process thumbnails Define a function extractThumbnail to extract thumbnail data, and define a function handleThumbnailBoundaries to handle the boundary value of the thumbnail.

[0077] function extractThumbnail(uint8Array) { / / Assume the thumbnail data is at a specific location in the TIFF file / / This is just a simple example, the actual extraction logic will be more complicated letthumbnailStart = 100; / / Assume the thumbnail starts at the 100th byte let thumbnailLength = 1024; / / Assume the thumbnail is 1024 bytes long let thumbnailData = uint8Array.subarray(thumbnailStart, thumbnailStart + thumbnailLength); return thumbnailData;}function handleThumbnailBoundaries(thumbnailData) { / / Resize the thumbnail as needed, handle boundary values, etc. / / For example, if the thumbnail size exceeds a certain limit, scale it / / This is just a simple example, the actual processing logic will be more complicated / / Assume no resizing is required, just return returnthumbnailData;} S202. Special case logic processing Special handling of boundary values ​​when calculating thumbnail size: Before generating thumbnails, you need to calculate the size of the thumbnail.

[0078] For some special cases (such as the original image size is not an integer multiple of the thumbnail size), special boundary value processing may be required to ensure the quality and integrity of the thumbnail.

[0079] This may involve cropping, padding or scaling the image to fit the target thumbnail size.

[0080] The encapsulated file structure is as follows Figure 2 shown.

[0081] In an embodiment of the present invention, based on step S3, a possible embodiment is given below to illustrate its specific implementation scheme in a non-limiting manner.

[0082] S301. Determine the format of the file header structure: The file header structure is usually a fixed-length binary data structure, and its format and field definition need to be determined according to the specific file format and application requirements.

[0083] For Tiff format files, the file header structure may contain a series of tag and value pairs, which are used to identify various properties and metadata of the file.

[0084] Define the fields of the file header structure: File size: Indicates the total number of bytes in the file, including the file header and the file data itself.

[0085] Occupied space size: Sometimes also referred to as the size of the file on the disk, which may include file data, file header, and additional space occupied due to file system block size alignment.

[0086] Start byte: indicates the starting position of the file data in the file (usually after the file header).

[0087] End byte: Indicates the end position of the file data (usually the file size minus 1).

[0088] File MIME type: Indicates the media type of a file (such as image / tiff), which is used to correctly identify and display files on the Web.

[0089] Assign a binary representation to a field: Each field requires a fixed binary length to represent its value. For example, the file size might be represented using a 64-bit unsigned integer to support storage of large files.

[0090] The binary representation of a field needs to follow a specific byte order (such as big endian or little endian) to ensure correct parsing on different platforms.

[0091] S302. Construct binary data of file header structure: According to the defined fields and binary representation, the value of each field is converted into binary data and they are concatenated together in the order of the fields to form the binary data of the file header structure.

[0092] During the construction process, attention should be paid to the alignment and padding between fields to ensure the correctness and consistency of the file header structure.

[0093] Add the file header structure to the beginning of the file: Add the constructed file header structure binary data to the beginning of the file before the file data.

[0094] Thus, when a file is read, the file header structure is first parsed to obtain various information and metadata about the file.

[0095] Verify the correctness of the file header structure: After building the file header structure, you can write some validation code to check the correctness of the file header structure. For example, you can verify that the value of the file size field matches the actual size of the file.

[0096] In an embodiment of the present invention, based on step S4, a possible embodiment is given below to illustrate its specific implementation scheme in a non-limiting manner.

[0097] S401. File segmentation processing: Before uploading a file, it is first divided into chunks per MB. If the file size is not an integer multiple of MB, the last chunk may be less than 1MB.

[0098] Each block is processed, including extracting specific bytes and calculating MD5 values.

[0099] S402. Extract specific bytes: For each data block, extract its first 10 bytes and last 10 bytes.

[0100] These bytes are chosen based on their location within the data block, which are typically the most sensitive locations for detecting data corruption or tampering.

[0101] S403. Calculate the MD5 value: Calculate the MD5 hash of the extracted bytes (i.e. the first 10 bytes and the last 10 bytes of each block).

[0102] MD5 is a widely used hash function that can generate a 128-bit hash value (usually represented as a 32-bit hexadecimal number) for verifying data integrity.

[0103] S404. Store MD5 value: Store the calculated MD5 value in a secure location, such as in metadata uploaded with the file, or in a database.

[0104] These MD5 values ​​will be used for subsequent file integrity verification.

[0105] S405. Upload files: Upload the file (including all parts) to the object storage service.

[0106] Ensure that proper error handling and retry mechanisms are used during uploads to improve upload reliability.

[0107] S406. Verify file integrity: After the file upload is complete, you can download the file (or just specific bytes for verification) from the object storage service.

[0108] For each downloaded data block, extract the first 10 bytes and the last 10 bytes and calculate the MD5 value.

[0109] The calculated MD5 value is compared with the previously stored MD5 value.

[0110] If the MD5 values ​​of all blocks match, the file is considered intact; if they do not, the file may be corrupted or tampered with in transit.

[0111] In an embodiment of the present invention, based on step S5, a possible embodiment is given below to illustrate its specific implementation in a non-limiting manner.

[0112] S501. File header structure analysis Metadata extraction: First, metadata related to the thumbnail needs to be extracted from the file header structure. This may include key information such as the thumbnail file name, storage path, size, format, creation time, etc. This information is embedded in the file header structure when the file is uploaded to the object storage service and is stored with the file.

[0113] Parsing tools: To parse the file header structure, you may need to use specific parsing tools or libraries. These tools can identify and extract metadata fields in the file header and provide necessary information for subsequent query operations.

[0114] 502. Query interface usage Query parameter preparation: Prepare appropriate query parameters based on the extracted metadata. For example, if the file header structure contains the file name of the thumbnail, the file name can be used as one of the query parameters. In addition, the query conditions can be further refined based on other metadata such as tags, indexes, or timestamps.

[0115] API call: Use the query API provided by the object storage service to submit the prepared query parameters to the service. The API may return a list of thumbnail files that match the query criteria, as well as detailed information about each file (such as storage location, size, access permissions, etc.).

[0116] S503. Thumbnail file access File location: Based on the results returned by the query interface, locate the required thumbnail file. This usually involves parsing the returned storage path or URL so that the file can be accessed directly.

[0117] Download or Display: Once the thumbnail file is located, you can choose to download the file or display the file content directly on the client. Downloading the file may involve obtaining the file data from the object storage service using HTTP or HTTPS protocols and saving it to a local or specified location. Displaying the file content directly on the client may involve loading the file data into an image viewer or browser so that the user can quickly browse and identify the content of the remote sensing image.

[0118] S504. Performance optimization Cache mechanism: In order to improve the access efficiency of thumbnail files, a cache mechanism can be introduced. For example, frequently used thumbnail files can be cached in a local or network location so that they can be quickly accessed when needed.

[0119] Parallel processing: If you need to access multiple thumbnail files at the same time, you can consider using parallel processing technology to speed up the file access process. This can be achieved through multithreading, asynchronous I / O or other concurrent programming techniques.

[0120] S505. Security and Rights Management Access control: When accessing thumbnail files, you need to ensure that only authorized users or systems can access the file content. This can be achieved by implementing appropriate access control strategies, such as using authentication, authorization codes, or encryption technology to protect the security of files.

[0121] Logging: To track and monitor the access to thumbnail files, you can enable logging. This helps to promptly identify potential security issues or performance bottlenecks and provide data support for subsequent optimization and improvement.

[0122] In order to further illustrate the remote sensing image thumbnail processing method provided by the present invention, another embodiment is given, please refer to Figure 3 ,include: Construct the file header structure: Constructing the file header structure is an important part of the file processing process. It contains the file header, thumbnail, Tiff original file related information, etc. The file header usually contains information such as the file size, occupied space, start byte, end byte, file MIME type, etc. This information is crucial for file parsing, processing and transmission. By constructing the file header structure, you can better manage and operate files and improve the efficiency and accuracy of file transmission.

[0123] Generate Tiff thumbnails using the custom script Tiffz-js: In order to better display Tiff files, a custom script Tiffz-js is used to generate Tiff thumbnails. In this process, the browser Canvas canvas is mainly used to collect Tiff paging samples and draw them into Jpeg. At the same time, the Canvas API is used to reduce the resolution to achieve the thumbnail effect. Since Tiff files are usually large and the compression process consumes more memory, it is necessary to reasonably handle memory usage in the process of generating thumbnails to avoid memory overflow problems.

[0124] Calculate the MD5 value check code: In order to ensure the integrity of the file, it is necessary to calculate the MD5 value check code. For the original remote sensing image file stream, extract the first 10 bytes and the last 10 bytes for each MB, and extract the last 10 bytes for the last 1MB (if not enough, extract the remaining part) to calculate the MD5 value. In this way, the integrity of the file can be effectively verified to ensure that the file will not be damaged or tampered with during transmission and processing.

[0125] Custom Tiffz format file packaging: Before uploading files, you need to package and encapsulate them. Use the Tiffz-js custom script to process the Blob binary file stream and perform zero padding and encapsulation operations. In this process, you need to perform logical processing on special cases, such as special processing of boundary values ​​when calculating the thumbnail size. In this way, you can ensure that the file will not fail to upload due to format or size issues during the upload process.

[0126] Upload the file to S3 object storage: Upload files to S3 object storage according to the S3 protocol or S3 SDK. S3 object storage is a high-performance, scalable object storage service that can effectively store and retrieve any type of data. By uploading files to S3 object storage, you can achieve persistent storage of files, which is convenient for subsequent file processing and analysis.

[0127] Display Tiff thumbnails: Finally, you need to display the Tiff thumbnail. In this operation, use the custom script Tiffz-js to quickly obtain the thumbnail, or use the tag to quickly display it. In this way, you can quickly display the thumbnail of the Tiff file of ultra-high-definition remote sensing images, provide users with a better browsing experience, and provide strong support for the processing and analysis of remote sensing images.

[0128] The beneficial effects of the above technical methods include: Simplify code complexity and simplify development. With the help of JavaScript's easy-to-learn, easy-to-use and rich API, a large amount of business-irrelevant code will be simplified, and the code logic will be simpler and clearer, which is conducive to the development work of coders.

[0129] Supports fast display of ultra-clear remote sensing image thumbnails. Through JavaScript API, through custom file formats, thumbnails and remote sensing image Tiff files are encapsulated into a new Tiffz file format; according to the HTTP range header, "Range: bytes=start-end", remote sensing image thumbnails can be quickly obtained for display.

[0130] Save performance, reduce memory usage, and reduce network transmission consumption. Solve the problem that the browser consumes more memory and the network transmission takes a long time due to the large files of hundreds of MB to several GB.

[0131] Versatility and scalability. This implementation method can support a variety of business scenarios and can be extended to more types of files, with good versatility and scalability.

[0132] In some embodiments, the remote sensing image thumbnail processing system may include a plurality of functional modules composed of computer program segments. The computer programs of the various program segments in the remote sensing image thumbnail processing system may be stored in a memory of a computer device and executed by at least one processor to perform (see Figure 1 Description) Remote sensing image thumbnail processing functions.

[0133] In this embodiment, the remote sensing image thumbnail processing system can be divided into multiple functional modules according to the functions it performs, such as Figure 4 As shown. The functional modules of the system 400 may include: a thumbnail generation module 410, a file encapsulation module 420, a file header construction module 430, a file upload module 440 and an object query module 450. The module referred to in the present invention refers to a series of computer program segments that can be executed by at least one processor and can complete fixed functions, which are stored in a memory. In this embodiment, the functions of each module will be described in detail in subsequent embodiments.

[0134] A thumbnail generation module is used to generate thumbnails for original remote sensing image files; A file packaging module, used for packaging the remote sensing image original file and the thumbnail into a file of a custom format; A file header construction module, used to obtain basic information of the remote sensing image original file and the thumbnail, and construct a file header structure based on the basic information; A file upload module, used to upload the file to the object storage service; An object query module is used to query the thumbnail from the object storage service based on the file header structure.

[0135] Optionally, as an embodiment of the present invention, the thumbnail generation module includes: An image loading unit, used to load the original remote sensing image file using an image processing library; A preprocessing unit, used to preprocess the image, wherein the preprocessing includes denoising, correcting color balance, and enhancing contrast, and metadata of geographic coordinate information is retained during the preprocessing stage; A size determination unit, used to determine the final size of the thumbnail according to the application scenario and storage limitations; A resizing unit, for applying an image resizing algorithm to resize the original image to a specified thumbnail size; The color conversion unit is used to perform color space conversion on the scaled image to ensure color consistency.

[0136] Optionally, as an embodiment of the present invention, the file encapsulation module includes: An initialization module, used for initializing a Blob object, wherein the Blob object includes the original remote sensing image file and the thumbnail; The file processing module is used to read the data of the Blob object, and use the image processing function to fill the read data with zeros and encapsulate it, and convert the processed data back to the Blob object; The boundary processing module is used to extract a thumbnail from a Blob object through a predefined function and process a boundary value of the thumbnail.

[0137] Figure 5 The remote sensing image thumbnail processing method provided for the embodiment of the present application can be applied to equipment. It will be appreciated by those skilled in the art that the equipment structure involved in the embodiment of the present invention does not constitute a limitation on the equipment, and the equipment may include more or less components than shown, or combine certain components, or arrange different components. In an embodiment of the present invention, the equipment includes but is not limited to laptop computers, desktop computers, workbenches, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The equipment may also represent various forms of mobile devices, such as personal digital processing, cellular phones, smart phones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples, and are not intended to limit the implementation of the embodiments of the present application described and / or required herein.

[0138] The device 500 may include: a processor 510, a memory 520 and a communication unit 530. These components communicate via one or more buses. Those skilled in the art will appreciate that the server structure shown in the figure does not limit the present invention, and it may be a bus structure or a star structure, and may include more or fewer components than shown in the figure, or combine certain components, or arrange components differently.

[0139] The memory 520 may be used to store the execution instructions of the processor 510, and the memory 520 may be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, magnetic disk or optical disk. When the execution instructions in the memory 520 are executed by the processor 510, the device 500 is enabled to perform some or all of the steps in the following method embodiments.

[0140] The processor 510 is the control center of the storage device, and uses various interfaces and lines to connect various parts of the entire electronic device. It runs or executes software programs and / or modules stored in the memory 520, and calls data stored in the memory to perform various functions of the electronic device and / or process data. The processor can be composed of an integrated circuit (IC), for example, it can be composed of a single packaged IC, or it can be composed of a plurality of packaged ICs with the same or different functions. For example, the processor 510 can include only a central processing unit (CPU). In an embodiment of the present invention, the CPU can be a single computing core or multiple computing cores.

[0141] The communication unit 530 is used to establish a communication channel so that the storage device can communicate with other devices, receive user data sent by other devices or send user data to other devices.

[0142] The present invention also provides a computer storage medium, wherein the computer storage medium may store a program, and when the program is executed, the program may include some or all of the steps in each embodiment provided by the present invention. The storage medium may be a magnetic disk, an optical disk, a read-only memory (ROM) or a random access memory (RAM).

[0143] Those skilled in the art can clearly understand that the technology in the embodiments of the present invention can be implemented by means of software plus a necessary general hardware platform. Based on this understanding, the technical solution in the embodiments of the present invention, in essence, or the part that contributes to the prior art, can be embodied in the form of a software product, which is stored in a storage medium such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a disk or an optical disk, and other media that can store program codes, including several instructions for enabling a computer device (which can be a personal computer, a server, or a second device, a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention.

[0144] In this specification, the same or similar parts between the various embodiments can be referred to each other. In particular, for the device embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the description in the method embodiment.

[0145] In the several embodiments provided by the present invention, it should be understood that the disclosed systems and methods can be implemented in other ways. For example, the system embodiments described above are only schematic. For example, the division of the modules is only a logical function division. There may be other division methods in actual implementation, such as multiple modules or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of systems or modules, which can be electrical, mechanical or other forms.

[0146] The modules described as separate components may or may not be physically separated, and the components shown as modules may or may not be physical modules, that is, they may be located in one place or distributed on multiple network modules. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0147] In addition, each functional module in each embodiment of the present invention may be integrated into one processing module, or each module may exist physically separately, or two or more modules may be integrated into one module.

[0148] Although the present invention has been described in detail with reference to the accompanying drawings and in combination with the preferred embodiments, the present invention is not limited thereto. Without departing from the spirit and essence of the present invention, a person of ordinary skill in the art may make various equivalent modifications or substitutions to the embodiments of the present invention, and these modifications or substitutions shall be within the scope of the present invention. Any person of ordinary skill in the art may easily think of changes or substitutions within the technical scope disclosed by the present invention, and these shall be within the scope of protection of the present invention.< / canvas>

Claims

1. A remote sensing image thumbnail processing method, characterized in that: include: Generate thumbnails for remote sensing image raw files; Encapsulating the remote sensing image original file and the thumbnail into a file of a custom format; Obtaining basic information of the remote sensing image original file and the thumbnail, and constructing a file header structure based on the basic information; Upload the file to the object storage service; The thumbnail is queried from the object storage service based on the file header structure.

2. The method according to claim 1, characterized in that Generate thumbnails for remote sensing image raw files, including: Use the image processing library to load remote sensing image raw files; Preprocessing the image, including denoising, correcting color balance, and enhancing contrast, and retaining metadata of geographic coordinate information during the preprocessing stage; Determine the final size of the thumbnail based on the application scenario and storage limitations; Apply an image scaling algorithm to scale the original image to the specified thumbnail size; Perform color space conversion on the scaled image to ensure color consistency.

3. The method according to claim 1, characterized in that Encapsulating the remote sensing image original file and the thumbnail into a file in a custom format includes: Initialize a Blob object, wherein the Blob object includes the remote sensing image original file and the thumbnail; Read the data of the Blob object, use the image processing function to fill the read data with zeros and encapsulate it, and convert the processed data back to the Blob object; Extract the thumbnail from the Blob object through a predefined function and process the boundary value of the thumbnail.

4. The method according to claim 1, characterized in that: Obtain basic information of the remote sensing image original file and the thumbnail, and construct a file header structure based on the basic information, including: Write the file size, occupied space, start byte, end byte, and file MIME type in the file header structure.

5. The method according to claim 1, characterized in that Upload the file to the object storage service, including: For the data stream of uploading the file to the object storage service, the first 10 bytes and the last 10 bytes are extracted for every MB, and the MD5 value of the extracted bytes is calculated, and the integrity of the file is verified based on the MD5 value.

6. A remote sensing image thumbnail processing system, characterized in that: include: A thumbnail generation module is used to generate thumbnails for original remote sensing image files; A file packaging module, used for packaging the remote sensing image original file and the thumbnail into a file of a custom format; A file header construction module, used to obtain basic information of the remote sensing image original file and the thumbnail, and construct a file header structure based on the basic information; A file upload module, used to upload the file to the object storage service; An object query module is used to query the thumbnail from the object storage service based on the file header structure.

7. The system according to claim 6, characterized in that The thumbnail generation module comprises: An image loading unit, used to load the original remote sensing image file using an image processing library; A preprocessing unit, used to preprocess the image, wherein the preprocessing includes denoising, correcting color balance, and enhancing contrast, and metadata of geographic coordinate information is retained during the preprocessing stage; A size determination unit, used to determine the final size of the thumbnail according to the application scenario and storage limitations; A resizing unit, for applying an image resizing algorithm to resize the original image to a specified thumbnail size; The color conversion unit is used to perform color space conversion on the scaled image to ensure color consistency.

8. The system according to claim 6, characterized in that The file encapsulation module comprises: An initialization module, used for initializing a Blob object, wherein the Blob object includes the original remote sensing image file and the thumbnail; The file processing module is used to read the data of the Blob object, and use the image processing function to fill the read data with zeros and encapsulate it, and convert the processed data back to the Blob object; The boundary processing module is used to extract a thumbnail from a Blob object through a predefined function and process a boundary value of the thumbnail.

9. A device, characterized in that: include: A memory, used for storing a remote sensing image thumbnail processing program; A processor is used to implement the steps of the remote sensing image thumbnail processing method as described in any one of claims 1 to 5 when executing the remote sensing image thumbnail processing program.

10. A computer-readable storage medium storing a computer program, characterized in that: The readable storage medium stores a remote sensing image thumbnail processing program, and when the remote sensing image thumbnail processing program is executed by a processor, the steps of the remote sensing image thumbnail processing method according to any one of claims 1 to 5 are implemented.

Images