Multi-source heterogeneous data coding indexing method and system in earthquake industry

Abstract

The invention provides a multi-source heterogeneous data coding indexing method and system in the earthquake industry, and relates to the technical field of big data processing. The method comprises the following steps: extracting common attributes and specific attributes of seismic data of each subject, and outputting a subject data attribute set to form a coding model basis; realizing a code designer capable of dynamically editing and expanding; the method comprises the following steps: analyzing data before data storage, selecting a code designer according to a subject, complementing file code information, and generating a file unique identifier; synchronously generating an index by using the coding identifier and the extension information, and storing the index in ElasticSearch; and multi-dimensional efficient retrieval of data is realized on the basis of an API (Application Program Interface) encapsulation data service of an ElasticSearch query engine. According to the invention, the module division and interaction process of the system is completely described, the coding scheme can be changed in time according to the development and change of the industry, and the designed code has man-made readability and is obviously better than the traditional random number scheme.

Description

Technical Field

[0001] This invention relates to the field of big data processing technology, and in particular to big data processing in the earthquake industry, specifically to a method and system for encoding and indexing multi-source heterogeneous data in the earthquake industry. Background Technology

[0002] With the development of earthquake research, the types of data disciplines used in industry applications are becoming increasingly diverse, and interdisciplinary data integration has become an important research requirement. Data from various disciplines and sources, such as seismology, strong ground motion, infrasound, water temperature, water level, satellite data, ground acoustics, and geothermal data, are often stored and analyzed by relevant experts using independent software. Convenient and efficient interdisciplinary data retrieval is one of the keys to improving research efficiency. Abstracting the common and unique attributes of each discipline, establishing a unified coding and indexing model for data in the data fusion library, and achieving efficient retrieval of massive amounts of data are major challenges in earthquake big data processing technology.

[0003] Chinese invention patent with publication number CN116156006A discloses a service cataloging and parsing method based on rule coding. The specific coding of this method consists of service type and random number, which cannot be changed in a timely manner according to the development of the industry, and the human readability of the coding is poor. Summary of the Invention

[0004] Purpose of the invention: To propose a multi-source heterogeneous data encoding and indexing method and system for the earthquake industry. This method can not only be updated in a timely manner according to the development and changes in the industry, but also the designed encoding is human-readable, which is significantly better than random number schemes.

[0005] This invention proposes a multi-source heterogeneous data encoding and indexing method for the earthquake industry, the steps of which are as follows: Step 1: Extract the common and unique attributes of earthquake data from various disciplines, output the set of discipline data attributes, and form the basis of the coding model; Step 2: Design coding rules according to subject categories, and output a set of coding rules including attribute combinations, separators, validity period and version; Step 3: Identify the subject type and date of the data to be stored, parse the data, extract the attribute information of the data based on the subject data attribute set output in Step 1, and output the data attribute extraction results; Step 4: Obtain the subject type output in Step 3, match the target coding rule and corresponding version in the coding rule set output in Step 2 that are within the validity period, and schedule the corresponding coding generation plugin. The coding generation plugin generates a unique coding identifier for the data based on the target coding rule and the data attribute extraction result output in Step 3. Step 5: Receive the index creation notification from the data entry module, extract the unique code identifier output in Step 4, the file storage path of the data, and the data attribute extraction results output in Step 3, generate an ElasticSearch index and store it; Step 6: Perform precise retrieval based on the data attribute extraction results output in Step 3, or perform fuzzy retrieval based on the unique code identifier output in Step 4, to complete the indexing.

[0006] As a preferred embodiment, in step 1, the public attributes include network code, station code, measurement item code, sampling rate, time, longitude, and latitude; the unique attributes include the completeness rate of seismic data and the number of interrupted records; and the subject data attribute set is a combination of public attributes and corresponding subject-specific attributes.

[0007] As a preferred solution, step 3 supports the recognition and parsing of minimized, renix, and json data types. By parsing the data packet header and data content, attribute information corresponding to the subject data attribute set in step 1 is extracted to form a structured data attribute extraction result.

[0008] As a preferred solution, in step 4, a plug-in scheduling method is adopted, with each subject corresponding to at least one encoding generation plug-in. When new subject data is added, the encoding of the subject data can be generated by developing the corresponding encoding generation plug-in and associating it with the encoding rules of that subject.

[0009] As a preferred embodiment, in step 5, the index creation notification is issued by the data storage module after the data is written to the specified path in the object storage. The file storage path is the unique access path for the data in the object storage. The ElasticSearch index contains a unique coded identifier, the file storage path, and the association mapping relationship of the data attribute extraction results.

[0010] As a preferred option, in step 6, the precise retrieval includes keyword matching retrieval based on the time, latitude and longitude, network code, and station code extracted from the data attribute results.

[0011] As a preferred embodiment, in step 6, the fuzzy search is based on ElasticSearch's word segmenter to perform partial or full character matching search of the uniquely encoded identifier.

[0012] Furthermore, this invention also proposes a multi-source heterogeneous data encoding and indexing system for the earthquake industry, the system comprising: The coding designer module is used to design coding rules by subject category. It supports the customization of coding attributes and separators, as well as the setting of rule validity period and version. It outputs a set of coding rules containing attribute combinations, separators, validity period and version. The data parsing module is used to receive data to be stored, identify the data type and subject, parse the data and extract the corresponding attribute information based on the subject data attribute set output by the coding designer module (generated by association from the coding model), and output the data attribute extraction results. The encoding generation plugin module contains encoding generation plugins for multiple disciplines. It is used to receive scheduling instructions from the encoding generator module, extract results based on target encoding rules and data attributes, and generate unique encoding identifiers for the data. The encoding generator module is used to obtain the subject type output by the data parsing module, match the target encoding rule and its corresponding version that are within the validity period from the encoding rule set output by the encoding designer module, and send a scheduling instruction to the encoding generation plug-in module. The data ingestion module is used to identify the subject type and date of the data to be stored, store the data in the specified bucket and path of the object storage, output the file storage path, and send an index creation notification to the data index generation module at the same time. The data index generation module receives the index creation notification from the data entry module, extracts the unique code identifier output by the code generation plugin module, the file storage path output by the data entry module, and the data attribute extraction results output by the data parsing module, and generates and stores the ElasticSearch index. The data retrieval module provides an ElasticSearch retrieval interface, supporting precise retrieval based on the data attribute extraction results output by the data parsing module, as well as fuzzy retrieval based on the unique encoding identifier output by the encoding generation plugin module, and returning the file storage path and data attribute information corresponding to the retrieval results.

[0013] As a preferred embodiment, the object storage adopts a storage architecture that supports RESTful APIs. Data is stored in the form of objects, and the metadata of each object includes the subject type, date, and file storage path information of the data. The file storage path is a unique access identifier for the object.

[0014] As a preferred embodiment, the data retrieval module is connected to an ElasticSearch cluster, which stores indexes of all subject data and supports single-subject attribute retrieval filtered by subject type, as well as cross-subject coded identifier or multi-attribute joint retrieval.

[0015] As a preferred embodiment, the encoding designer module provides a human-computer interaction page that allows users to upload subject data attribute set templates. A custom parser matches the attributes in the template with the key values ​​in the encoding rules, and generates an encoding format example by concatenating the attributes according to the attribute combination and separator rules.

[0016] As a preferred embodiment, the data storage module is configured with transmission components, transmission units, transmission strategies, and transmission tasks, supporting periodic or real-time data transmission. It automatically initiates transmission tasks and retrieves files to be stored based on task parameters, and synchronously outputs the file storage path and triggers an index creation notification after the file storage is completed.

[0017] Compared with the prior art, the present invention has the following beneficial effects: (1) The coding designer supports human-computer interaction to customize coding rules and can edit and take effect in real time, which reduces the impact of changes in coding rules caused by changes in industry standards on data entry and retrieval, enhances the flexibility of the system architecture, and improves the scalability of the system.

[0018] (2) The data file encoding generated by the encoding designer can not only be changed in a timely manner according to the development and changes of the industry, but also the designed encoding has human readability, which is different from the implementation methods such as UUID or auto-incrementing primary key. The improved readability makes it easier for researchers to use the system.

[0019] (3) Different types of encoding generation plugins are defined and differentiated data processing methods are provided. The corresponding plugins are matched according to different data types, which solves the complexity problem of generating unique encoding identifiers for multi-disciplinary data. At the same time, when new disciplinary data is accessed, only new plugins need to be developed and called by the encoding generator, realizing the decoupling of system functions and the customized development of new disciplinary data encoding, which can significantly reduce the R&D investment in new data entry and retrieval.

[0020] (4) It supports filtering and searching by subject attributes to meet the inherent single-subject data analysis needs of the industry. It can also achieve convenient cross-subject data analysis through joint retrieval of latitude and longitude attributes such as seismic networks and stations. At the same time, it can achieve fast fuzzy matching of concatenated fields such as encoding based on word segmentation characteristics. This provides more options for the retrieval of earthquake industry data and improves retrieval efficiency. Attached Figure Description

[0021] Figure 1 This is a diagram showing the software module relationships for the encoded index in the embodiment.

[0022] Figure 2 This is a flowchart of the encoding index in the embodiment. Detailed Implementation

[0023] In the following description, numerous specific details are set forth in order to provide a more thorough understanding of the invention. However, it will be apparent to those skilled in the art that the invention can be practiced without one or more of these details. In other instances, certain technical features well-known in the art have not been described in order to avoid obscuring the invention.

[0024] Before describing the embodiments, some specific terms that may appear in the following text will be explained.

[0025] Data ingestion refers to the process of transferring data from a source system to a destination system. This process involves extracting data from the data source, transforming the data to conform to the format and structure of the target database, and loading the data into the target database. It is a key step in data warehousing, big data analytics, data science, and other applications that require persistent data storage.

[0026] Object storage is a data storage architecture that manages data as objects. Each object typically contains the data itself, metadata (information about the data, such as name, size, type, etc.), and a unique identifier (usually a URI or URL). Object storage is designed to efficiently handle large amounts of unstructured data, which often exists as files such as images, videos, and log files. Object storage typically follows a set of standard interfaces to ensure interoperability between different systems and services. Object storage systems usually provide a series of RESTful API-based interfaces for managing and accessing data in the storage. These interfaces allow users to perform various operations, such as uploading, downloading, listing, and deleting objects, as well as managing buckets and object metadata.

[0027] Elasticsearch is an open-source, distributed search engine built on Lucene. It provides full-text search and analytics capabilities and is typically used to implement complex search functions and manipulate large datasets. Elasticsearch is widely used in various scenarios such as log analysis, full-text search, security analysis, and business analysis. It can handle structured, semi-structured, and even unstructured data, making it one of the most important tools for big data processing and analysis.

[0028] To address the shortcomings of existing technical solutions, this application proposes a multi-source heterogeneous data encoding and indexing method and system for the earthquake industry, specifically including: 1. Abstracting common and unique attributes of various disciplines to form the basis of the encoding model; 2. Implementing a dynamically editable and expandable encoding designer; 3. Parsing the data before data storage, selecting the encoding designer according to the discipline, completing the file encoding information, and generating a unique file identifier; 4. Synchronously generating an index with the encoding identifier and extended information, and storing it in Elasticsearch; 5. Encapsulating data services based on the Elasticsearch query engine's API to achieve multi-dimensional and efficient data retrieval.

[0029] The specific details of the technical solution of this invention are disclosed below. A multi-source heterogeneous data encoding and indexing system for the earthquake industry, the software module relationship diagram of the data encoding and indexing is as follows. Figure 1As shown, it consists of a data entry module, an encoding designer module, a data parsing module, an encoding generator module, an encoding generation plugin module, a data index generation module, and a data retrieval module.

[0030] The data import module identifies the data files to be stored by subject and date, and stores them in the designated bucket of object storage. During data storage, it calls the data parsing module, encoding generator module, and encoding generation plugin module to obtain the encoding identifier of the data files and related attributes such as time, latitude and longitude, network code, and station code. It then sends a file index creation event to notify the data index generation module to generate and store the index.

[0031] The coding designer module designs coding rules based on the subject classification of the data. It extracts common attributes of the subject data as the basis for coding design, allowing customization of attributes and separators used in the coding. For example, the default rule for seismic surveying is "network.station.measuring point.channel.quality.data format". Based on changes in industry standards, the common attribute "network.station" is retained, and a custom attribute "channel.quality.sampling rate.data format" is added, forming a new attribute "network.station.channel.quality.sampling rate.data format". It also supports coding validity period and version management.

[0032] Data parsing module. The data parsing module supports data recognition and parsing of different data types such as minimized, renix, and json. It can extract the main attributes of the data for encoding generation and identify the subject type to which the data belongs.

[0033] Encoding Generator Module. To ensure the uniqueness of data encoding identifiers, the encoding generator module supports the scheduling of different encoding generation plugins in a plug-in manner to generate codes designed for the corresponding discipline. The encoding generator module will match the corresponding encoding generation plugin based on the discipline identified by the data parsing module, the validity period of the encoding rules, the version of the encoding rules, etc., and the encoding generation plugin will then perform the encoding generation.

[0034] Encoding generation plugin module. Different encoding generation plugins are designed and implemented for different disciplines. The encoding generation plugin obtains the encoding rules selected by the encoding generator module, retrieves the data attributes extracted by the data parsing module according to the encoding rules, and generates a unique encoding identifier for the data according to the encoding rules.

[0035] The data index generation module receives index creation notifications when data is imported into the database. It extracts relevant attributes from the notification, such as the data file's encoding identifier, file storage path and time, latitude and longitude, network code, and station code. It then generates an ElasticSearch index creation statement and stores the index in ElasticSearch for later querying and retrieval.

[0036] The data retrieval module provides an ElasticSearch-based file retrieval interface, supporting rapid data retrieval and application based on dimensions such as time, latitude and longitude, network code, and station code. The data index generation module stores retrieval information for all data files across various disciplines in a unified ElasticSearch index. Queries can not only perform precise keyword searches using attributes such as latitude and longitude, network code, and station code, but also conduct fuzzy searches based on ElasticSearch's tokenizer and full-text search of some or all information in the encoded identifiers. High-efficiency retrieval of earthquake big data is achieved based on a high-performance ElasticSearch cluster.

[0037] Data encoding index timing such as Figure 2 As shown, the process is described below: S1. Abstracting Subject Data Attributes. Based on industry standards and the analysis of example data for each subject, abstract the common attributes of the corresponding subject data, such as network code, station code, measurement item code, sampling rate, time, longitude, latitude, etc., and then abstract the unique attributes of the corresponding subject data, such as the completeness rate of seismic data and the number of records interrupted.

[0038] S2. Design Coding Rules. Based on industry standards, generate seismic coding rules through a human-computer interaction interface using the coding designer page. Use a custom parser to parse the MSEed data packet, extracting the basic business attributes from the packet header. Use a custom JSON parser to extract data attributes. After parsing, match the corresponding attributes with the key values ​​in the rules, and then concatenate the corresponding values ​​according to the rules to generate codes, such as "network.station.measuring point.channel.quality.data format". Set the version of this rule; the default version is 1.0.0, and enable the rule.

[0039] S3. Create an inbound task.

[0040] a. Create a transport component. Based on your requirements, create the corresponding transport component. You will need to manually configure the required information, including the transport component client name, type, address, port, path, password, and username.

[0041] b. Create a transport unit. Create the corresponding transport unit according to your needs. You will need to manually configure the transport unit name, transport unit type, and select the corresponding transport component. If it is a Pulsar type, you will also need to manually configure the tenant, namespace, topic name, data tag, etc. If it is an HBase type, in addition to configuring the transport unit name, transport unit type, and selecting the corresponding transport component, you will also need to separately configure the namespace for storing data, table name, data level, transport type, storage duration, and other parameters.

[0042] c. Create a transmission strategy. Create a task strategy according to the requirements, select the source transmission unit type, select the transmission unit created in the previous step; select the destination transmission unit type and transmission unit name, and enter the data encapsulation granularity.

[0043] d. Create a transfer task. Create an inbound task, select an inbound strategy, configure task information such as: task name, execution frequency, periodic or real-time, and select the executor for the corresponding data format.

[0044] S4. Start the transfer task.

[0045] View the task list, select the task to start, and click "Start." Locate the corresponding task strategy based on the task parameters, configure the strategy to find the transmission unit information, and then use that information to find the transmission component information. Assemble the parameters for starting the transmission task based on the obtained information, and then set the task start time. Once the start time arrives, the transmission task will automatically begin.

[0046] S5. Obtain File The data import module will periodically retrieve the files to be stored from the corresponding transmission component according to the API based on the task.

[0047] S6. Parse the file The data entry module calls the data parsing module to parse the file, determines that the file is of the minimized type and identifies the discipline as seismology based on the channel code. The data parsing module extracts the main attributes of the data, such as the network code, station code, measurement point code, channel code, latitude and longitude, time, and sampling rate, for encoding generation.

[0048] S7, Encoding Generation a. If the encoding generator receives the parsed subject as seismology, it will select to call the seismology encoding generation plugin for encoding generation.

[0049] b. The seismic coding generation plugin obtains the configured coding generation rules (e.g., "network, station, measurement point, channel, quality, data format") based on the discipline, and concatenates them according to the coding generation rules based on the parsed main attribute information to generate a coding identifier.

[0050] S8, Data Ingestion and Event Notification The data entry module writes the data into the seismic data bucket of object storage based on the object storage API and issues an event notification for index generation.

[0051] S9, Data Index Generation Extract relevant attributes from the notification, such as the data file's encoding identifier, file storage path and time, latitude and longitude, network code, and station code, generate an ElasticSearch index creation statement, and store the index in ElasticSearch for later querying and retrieval.

[0052] S10, Data Retrieval and Download Based on the ElasticSearch file retrieval interface, this system supports businesses in quickly retrieving and applying data across multiple dimensions, including time, latitude and longitude, network code, and station code. The index information obtained through precise or fuzzy searches includes object storage path information, allowing users to download files from object storage based on this path.

[0053] S11, Index of Other Subject Data Taking the seismic discipline coding index in steps S1 to S10 as an example, coding rules and indexing tasks for different disciplines such as infrasound, strong ground motion, water temperature, and water level can be configured. All data files for all disciplines are indexed uniformly in Elasticsearch. When searching for files, single-discipline data analysis can be achieved by filtering by discipline attributes in the index, or cross-discipline data analysis can be achieved by using network, station, or coding attribute information in the index.

[0054] It is worth noting that the logical ideas behind the methods disclosed in the above embodiments can be implemented, in whole or in part, through software, hardware, firmware, or any other combination thereof. When implemented using software, the above embodiments can be implemented, in whole or in part, in the form of a computer program product. A computer program product includes one or more computer instructions or computer programs.

[0055] When computer instructions or computer programs are loaded or executed on a computer, all or part of the processes or functions according to the embodiments of this application are generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. Computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that includes one or more sets of available media. Available media can be magnetic media (e.g., floppy disks, hard disks, magnetic tapes), optical media (e.g., DVDs), or semiconductor media. Semiconductor media can be solid-state drives (SSDs).

[0056] Although the invention has been shown and described with reference to specific preferred embodiments, it should not be construed as limiting the invention itself. Various changes in form and detail may be made without departing from the spirit and scope of the invention as defined in the appended claims.

Claims

1. A multi-source heterogeneous data encoding indexing method in the seismic industry, characterized in that, The method comprises the following steps: Step 1, extract the common attributes and unique attributes of seismic data of each discipline, output the discipline data attribute set, and form the coding model basis; Step 2, design coding rules according to discipline classification, output coding rule set containing attribute combination, interval symbol, validity period and version; Step 3, identify the discipline type and date of the data to be stored, parse the data, extract the attribute information of the data based on the discipline data attribute set output in step 1, and output the data attribute extraction result; Step 4, obtain the discipline type output in step 3, match the target coding rule and corresponding version in the coding rule set output in step 2 corresponding to the discipline and within the validity period, and schedule the corresponding coding generation plug-in; the coding generation plug-in generates a unique coding identifier for the data based on the target coding rule and the data attribute extraction result output in step 3; Step 5, receive the index creation notification issued by the data storage module, extract the unique coding identifier output in step 4, the file storage path of the data and the data attribute extraction result output in step 3, generate an ElasticSearch index and store it; Step 6, based on the data attribute extraction result output in step 3, accurate retrieval is performed, or based on the unique coding identifier output in step 4, fuzzy retrieval is performed, and the index is completed.

2. The method of claim 1, wherein, In step 1, the common attributes include network code, station code, measurement item code, sampling rate, time, longitude and latitude; the unique attributes include the integrity of the seismic data and the number of breaks; and the discipline data attribute set is a combination set of common attributes and unique attributes corresponding to the discipline.

3. The method of claim 1, wherein, In step 3, the identification and analysis of miniseed, renix and json type data are supported; the attribute information corresponding to the discipline data attribute set in step 1 is extracted by analyzing the data packet header and data content to form a structured data attribute extraction result.

4. The method of claim 1, wherein, In step 4, a plug-in scheduling method is adopted, at least one coding generation plug-in corresponding to each discipline is provided, when new discipline data is added, the corresponding coding generation plug-in is developed and associated with the coding rule of the discipline, so that the coding generation of the discipline data is realized.

5. The method of claim 1, wherein, In step 5, the index creation notification is issued by the data storage module after the data is written to the specified path of the object storage; the file storage path is the unique access path of the data in the object storage; and the ElasticSearch index contains the associated mapping relationship of the unique coding identifier, the file storage path and the data attribute extraction result.

6. The method of claim 1, wherein, In step 6, the accurate retrieval includes keyword matching retrieval through time, longitude and latitude, network code and station code in the data attribute extraction result.

7. The method of claim 1, wherein, In step 6, the fuzzy retrieval is realized by matching part or all characters of the unique coding identifier based on the tokenizer of ElasticSearch.

8. A multi-source heterogeneous data encoding indexing system for the seismic industry, characterized in that, It comprises: A coding designer module is used for designing coding rules according to discipline classification, supporting customization of coding attributes and interval symbols, setting rule validity period and version, and outputting coding rule set containing attribute combination, interval symbol, validity period and version; The data analysis module is configured to receive data to be stored, identify the data type and the subject to which the data belongs, analyze the data and extract corresponding attribute information based on the subject data attribute set output by the coding designer module, and output the data attribute extraction result. The coding generation plug-in module includes a plurality of coding generation plug-ins corresponding to subjects, and is configured to receive a scheduling instruction from the coding generator module, generate a unique coding identifier for the data based on a target coding rule and the data attribute extraction result. The coding generator module is configured to obtain the subject type output by the data analysis module, match a target coding rule and a corresponding version in a valid period from the coding rule set output by the coding designer module, and send a scheduling instruction to the coding generation plug-in module. The data storage module is configured to identify the subject type and the date of the data to be stored, store the data in a specified bucket and path of the object storage, output the file storage path, and send an index creation notification to the data index generation module. The data index generation module is configured to receive the index creation notification sent by the data storage module, extract the unique coding identifier output by the coding generation plug-in module, the file storage path output by the data storage module, and the data attribute extraction result output by the data analysis module, generate an ElasticSearch index, and store the ElasticSearch index. The data retrieval module is configured to provide an ElasticSearch retrieval interface, support accurate retrieval based on the data attribute extraction result output by the data analysis module, and fuzzy retrieval based on the unique coding identifier output by the coding generation plug-in module, and return the file storage path and data attribute information corresponding to the retrieval result.

9. The multi-source heterogeneous data encoding indexing system of the seismic industry of claim 8, wherein, The object storage adopts a storage architecture supporting RESTful API, and data is stored in the form of objects. The metadata of each object includes the subject type, date, and file storage path information of the data, and the file storage path is a unique access identifier of the object.

10. The multi-source heterogeneous data encoding indexing system of the seismic industry of claim 8, wherein, The data retrieval module accesses an ElasticSearch cluster, the ElasticSearch cluster stores indexes of all subject data, supports single-subject attribute retrieval filtered by subject type, and cross-subject coding identifier or multi-attribute joint retrieval.

11. The multi-source heterogeneous data encoding indexing system of the seismic industry of claim 8, wherein, The coding designer module provides a human-computer interaction page, supports user uploading of a subject data attribute set template, matches the attributes in the template with key values in the coding rule through a custom parser, and splices the coding format example according to the attribute combination and interval symbol rule.

12. The multi-source heterogeneous data encoding indexing system of the seismic industry of claim 8, wherein, The data storage module is configured to configure a transmission component, a transmission unit, a transmission strategy, and a transmission task, support periodic or real-time data transmission, automatically start the transmission task and pull the file to be stored according to the task parameters, output the file storage path synchronously and trigger the index creation notification after the file storage is completed.

Images