Big data access method based on protocol adaptation and scheduling in seismological industry

Abstract

The present invention belongs to the technical field of seismic big data, and relates to a big data access method based on protocol adaption and scheduling in the seismological industry. The method comprises the following steps: performing protocol adaption; automatically pulling up client strategy scheduling; performing control by means of a client plug-in; receiving seismic data; pushing the seismic data; and performing failure recovery and smooth upgrade. Compared with the prior art in which whether there is a need to add resources can only be artificially determined without standards and strategies, the present invention involves classifying access tasks of different data types into different priorities on the basis of availability requirements of different seismic data access and availability information of each resource, and configuring a load threshold value to be 0.7 as a scheduling triggering condition on the basis of the total number of instrument terminals that need to be connected and the number of instrument terminals that are actually supported by a single client in a cloud platform, and by means of monitoring and testing and the resource consumption conditions of different protocol data, such that whether there is a need to add resources can be automatically determined.

Description

A big data access method for protocol adaptation and scheduling in the earthquake industry Technical Field

[0001] The present invention belongs to the technical field of earthquake big data, and in particular relates to a big data access method for earthquake industry protocol adaptation and scheduling. Background Art

[0002] In the seismic industry, real-time acquisition and stable access to seismic data are of great significance for subsequent data processing and analysis. The challenges of big data access in the seismic industry mainly include huge data volumes, diverse data sources and protocol differences, high requirements for data access stability, and inconsistent data formats. First, the amount of data in the seismic industry is very large, including various types of data such as earthquake observation data, status data, and control data. The huge scale of this data requires an effective and stable access method. Second, the data sources in the seismic industry are very diverse, including seismic monitoring stations, seismic instruments, satellite remote sensing and other data collection methods. The diversity of these data sources and the differences in docking protocols bring certain complexities to data access. Third, most of the data in the seismic industry is accessed in real time, which requires extremely high stability of data access to avoid data loss as much as possible. Finally, the data format in the seismic industry is also inconsistent. Different data sources use different data formats and protocols, which increases the difficulty of data access.

[0003] Traditional data access methods often only support specific protocols and data formats, and require adaptation and conversion for each protocol, which limits the selection of data sources and data access capabilities. At the same time, since traditional data access services are deployed on a single machine and are highly coupled with server environment resources, there is a risk of single point failure and data loss. Summary of the Invention

[0004] In view of the above-mentioned deficiencies in the existing technology, the purpose of the invention is to provide a big data access method for protocol adaptation and scheduling in the seismic industry. Through multi-protocol plug-in integration, it ensures support for data docking of instrument terminals of multiple protocols. According to the availability requirements of different seismic data access in the actual environment and the availability information of each resource, an availability-aware priority algorithm strategy is configured, and the load threshold is configured as a scheduling trigger condition. Kubernetes is combined to start up services and distribute scheduler tasks, and instrument terminals are allocated to different clients. Through the configuration of health checks and automatic failover in Kubernetes, the high stability of data access is ensured.

[0005] The present invention proposes a big data access method for earthquake industry protocol adaptation and scheduling, comprising the following steps:

[0006] S1. Perform protocol adaptation;

[0007] S2. Automatically start client policy scheduling;

[0008] S3. Control via client plug-in;

[0009] S4. receiving seismic data;

[0010] S5. Push the earthquake data;

[0011] S6. Perform fault recovery and smooth upgrade.

[0012] Furthermore, the protocol adaptation specifically includes the following steps:

[0013] S11. Enter the information of the access instrument terminal;

[0014] S12. Perform plug-in testing;

[0015] S13. Test by issuing protocol adaptation instructions.

[0016] Furthermore, the automatic client policy scheduling specifically includes the following steps:

[0017] S21. Group the instrument terminals of different protocol types, and the service protocols within each group belong to the same protocol type;

[0018] S22. Then, based on the availability requirements of different seismic data accesses and the availability information of each resource, the tasks are divided into different priorities;

[0019] S23. The total number of instrument terminals that need to be connected, the number of instrument terminals actually supported by a single client in the cloud platform, the number of instrument terminals passed the monitoring test, and the resource consumption of different protocol data are then set up by setting an availability-aware priority algorithm strategy to launch services and distribute tasks;

[0020] S24. Set the load threshold to 70%, and then conduct a test judgment. If the resource usage is within the 70% range, the system is judged to be running stably. When more instrument terminals need to be connected and the load threshold exceeds 70%, the client service is automatically called up and the relevant information is configured to complete the terminal data access.

[0021] Furthermore, the control via the client plug-in specifically includes the following steps:

[0022] S31. When the client service is successfully launched, the service is registered in the aggregation gateway, and its service name, IP address information, type, and externally exposed interface information in the cloud platform are stored in the database;

[0023] S32. When the client service is registered, it sends heartbeat information to the aggregation gateway, records the current client identification id, the total client load, the current client load, the number of currently connected terminals, the number of currently connected normal terminals, the load of each plug-in itself and the status;

[0024] S33. Control the service management module through actual business needs and specific client services, and control the operation of the service plug-in when connecting to the terminal.

[0025] Furthermore, the receiving of seismic data specifically includes the following steps:

[0026] S41. Establishing a network connection between the client and the server, and transmitting data via the TCP / IP protocol;

[0027] S42. Parse the data format through the protocol plug-in and divide the data into different parts according to the protocol definition;

[0028] S43. Analyze the data and verify its integrity;

[0029] S44. Decode the parsed data according to the encoding method defined in the seismic data transmission protocol, and then further clean the parsed and decoded seismic data;

[0030] S45. After the data transmission is completed, the connection between the client and the server is closed to release resources and network ports.

[0031] Furthermore, the pushing of seismic data specifically includes the following steps:

[0032] S51. Based on the specific subsystem data requirements, agree on the naming conventions for topics in Pulsar and automatically create topics using the three-level restrictions of Tenant, Namespace, and Topic.

[0033] S52. After parsing the header information of the access data, the client protocol plug-in pushes the data of different subsystems to the corresponding topics for subsequent system use.

[0034] Furthermore, the fault recovery and smooth upgrade specifically include the following steps:

[0035] S61. Define health checks in the application's Pod configuration and configure livenessProbe and readinessProbe to regularly check the container's running status.

[0036] S62. When a pod fails, Kubernetes automatically migrates the pod on that node to another healthy node.

[0037] S63. Trigger a smooth upgrade by updating the image version in the deployment resource or using the kubectl command. Kubernetes gradually starts the new version of the Pod and gradually stops the old version of the Pod.

[0038] S64. If an error or other problem is detected during the upgrade process, it will automatically roll back to the previous version. You can roll back to the old version of the deployment by updating the image version of the deployment resource or using the kubectl command. Kubernetes will start the old version of the Pod and stop the new version of the Pod, rolling back to the previous state.

[0039] Furthermore, in said S44, the cleaning process includes deduplication, sorting and formatting of the data.

[0040] Furthermore, in S43, the method for verifying the integrity of the data adopts a hash method.

[0041] Furthermore, in said S33, the operations include registration, testing, starting, and stopping.

[0042] The beneficial effects of the present invention are as follows:

[0043] 1. Compared with the existing technology that can only manually determine whether additional resources are needed without standards or strategies, the access tasks of different data types are divided into different priorities based on the availability requirements of different seismic data access and the availability information of each resource. Among them, observation data requires real-time access and has extremely high real-time requirements, so the priority is set to high; status data and control data require real-time access, but allow delays within a certain range, so the priority is set to medium; offline data does not require high real-time performance and only requires regular synchronization, so the priority is set to low. Based on the total number of instrument terminals that need to be accessed and the number of instrument terminals actually supported by a single client in the cloud platform, as well as the resources consumed by different protocol data through monitoring tests, a load threshold of 0.7 is configured as a scheduling trigger condition to automatically determine whether additional resources are needed.

[0044] 2. Compared with the existing technology, which can only start the service through background deployment when adding data access services, this system uses the set availability-aware priority algorithm strategy, uses Kubernetes to pull up the service, and the scheduler to distribute tasks, and allocates all instrument terminals to different clients, so that the number of instrument terminals connected to each client is maintained within a certain range, and the system resources consumed are maintained within a certain range. When more instrument terminals need to be connected, the client service can be automatically controlled to configure relevant information to complete terminal data access.

[0045] 3. In response to the needs of the earthquake industry, various communication protocols involved, such as SeedLink, Ntrip, HTTP, MQTT, CoAP, etc., are encapsulated in the form of plug-in integration, and the interfaces of different protocols are unified using the adapter pattern. Compared with the existing technology that requires enabling corresponding services when accessing data of different protocols, cannot be compatible with multiple types of protocols at the same time, and supports multiple data formats, it can realize data interaction and communication between different protocols, support elastic expansion, and can dynamically increase or decrease adapter instances as needed to meet different access requirements.

[0046] 4. Compared with existing technologies that often deploy services directly to servers in the form of single nodes, which require manual restart in the event of a failure and lack disaster recovery, Kubernetes defines health checks, configures livenessProbe and readinessProbe, regularly checks whether the containers are running normally, and configures automatic failover. When the client service in a Pod goes down, causing data access failure, Kubernetes automatically transfers the Pod on the node to other healthy nodes, ensuring high stability of data access. BRIEF DESCRIPTION OF THE DRAWINGS

[0047] The accompanying drawings are only for the purpose of illustrating specific embodiments and are not to be considered as limiting the present invention. Throughout the drawings, the same reference numerals represent the same components. Obviously, the drawings described below are only some of the embodiments of the present invention. Those skilled in the art can also derive other drawings based on these drawings.

[0048] FIG1 is a flow chart of an embodiment of the present invention;

[0049] FIG2 is a functional architecture diagram of the data access part according to an embodiment of the present invention;

[0050] FIG3 is a flow chart of a protocol adaptation connectivity test according to an embodiment of the present invention;

[0051] FIG4 is a flowchart of client policy scheduling and launching according to an embodiment of the present invention. DETAILED DESCRIPTION

[0052] In order to enable those skilled in the art to better understand the technical solutions in the embodiments of the present invention, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the embodiments described are part of the embodiments of the present invention, rather than all of the embodiments. It should be understood that these descriptions are merely exemplary and are not intended to limit the scope of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative work should fall within the scope of protection of the present invention.

[0053] Furthermore, in the following description, descriptions of well-known structures and technologies are omitted to avoid unnecessarily obscuring the concepts disclosed in the present invention.

[0054] Exemplary embodiments will be described in detail herein, examples of which are illustrated in the accompanying drawings. In the following description, when referring to the drawings, like numbers in different figures represent the same or similar elements unless otherwise indicated. The embodiments described in the following exemplary embodiments are not intended to represent all possible embodiments consistent with the present invention. Rather, they are merely examples of methods and systems consistent with certain aspects of the present invention, as detailed in the appended claims.

[0055] The present invention proposes a big data access method for protocol adaptation and scheduling in the earthquake industry, which is used to solve the problems faced by data access in the earthquake industry, such as large differences in protocols, diverse data formats, and high requirements for data access stability. The present invention introduces big data technology and utilizes plug-in integration to realize the encapsulation of multiple communication protocols, and uses the adapter mode to design a unified interface to realize data interaction and communication between different protocols. According to the availability requirements of data access and the availability information of each resource, an availability-aware priority algorithm strategy is made, and the load threshold is configured as a scheduling trigger condition. Kubernetes is combined to realize the automated processing of service startup and scheduler task distribution, and instrument terminals are allocated to different clients to maintain the number of instrument terminal accesses and system resource consumption of each client within a certain range. The high stability of data access is ensured through the configuration of health checks and automatic failover.

[0056] Method Example

[0057] In the present invention, as shown in FIG1 to FIG4 , a big data access method for earthquake industry protocol adaptation and scheduling is provided, comprising the following steps:

[0058] S1. Perform protocol adaptation;

[0059] S2. Automatically start client policy scheduling;

[0060] S3. Control via client plug-in;

[0061] S4. receiving seismic data;

[0062] S5. Push the earthquake data;

[0063] S6. Perform fault recovery and smooth upgrade.

[0064] Furthermore, the present invention provides stable access to large amounts of data from instrument terminals of different protocol types in the seismic industry, and has the following advantages:

[0065] ① Through multi-protocol plug-in integration, ensure support for multi-type protocol instrument terminal data docking;

[0066] ② Resource monitoring to ensure sufficient resources when launching client services;

[0067] ③ Configure the availability-aware priority algorithm strategy based on the availability requirements of different seismic data access in the actual environment and the availability information of each resource;

[0068] ④ Configure the load threshold as the scheduling trigger condition, combine it with Kubernetes to pull up services and distribute scheduler tasks, and allocate instrument terminals to different clients;

[0069] ⑤ Carry out command control for specific client services, and control the registration, testing, start, stop and other operations of the plug-in in the service when it connects to the terminal;

[0070] ⑥After parsing the packet header information of the access data, the client protocol plug-in pushes the data of different subsystems to the corresponding topic of the message middleware Pulsar according to the specifications for subsequent system use;

[0071] ⑦ Ensure high stability of data access through the configuration of health checks and automatic failover in Kubernetes.

[0072] Regarding the above-mentioned protocol adaptation, the present invention provides the following detailed process:

[0073] a. Enter the information of the access instrument terminal. According to the needs, clarify the type of seismic data transmission protocol and data format that need to be adapted, and enter the instrument terminals that need to be accessed under different protocols into the database table. The entry fields need to include basic information such as the terminal device's IP address, port, user name, password, station information, station network information, device type, device model, sub-sensor association information, observation data communication account and observation data communication password. At the same time, add a type distinction field to facilitate the client's multi-protocol plug-in recognition and activation.

[0074] b. Plug-in test: Different protocol acquisition plug-ins are selected for connectivity test based on the protocol type and communication information of the instrument terminal to ensure that the protocol parsing function of the protocol plug-in is normal.

[0075] c. Protocol adaptation instruction issuance test: Through a uniformly defined entry, the adapter mode is used to encapsulate and distribute the interfaces of different protocols, distinguish data formats, and automatically adapt protocols to perform data access connectivity tests on instrument terminals. In addition, in response to the needs of the seismic industry, multiple protocols involved, such as SeedLink, Ntrip, HTTP, MQTT, CoAP and other common protocols, are encapsulated in plug-in form. At the same time, the adapter mode is used to unify the interfaces of different protocols. By defining a unified interface and data format, data interaction and communication between different protocols can be achieved.

[0076] Regarding the above-mentioned automatic client policy scheduling, the present invention provides the following detailed process:

[0077] a. Group instrument terminals with different protocol types. Services within each group should belong to the same protocol type and provide similar functions. Perform performance stress testing on batches of 100 devices. Use the Kubernetes monitoring tool, Metrics Server, to collect metrics on single-node resource usage, including CPU utilization, memory usage, network bandwidth, and data throughput.

[0078] b. Assign tasks different priorities based on the availability requirements of different seismic data access and the availability information of each resource. For example, tasks can be divided into three priorities: high, medium, and low. Observation data requires real-time access with extremely high real-time requirements, so its priority is set to high. Status data and control data require real-time access but allow a certain range of delay, so its priority is set to medium. Offline data does not require high real-time requirements and only requires periodic synchronization, so its priority is set to low.

[0079] c. Based on the total number of instrument terminals that need to be connected and the number of instrument terminals actually supported by a single client in the cloud platform, as well as the resources consumed by different protocol data through monitoring tests, service startup and task distribution are carried out through the set availability-aware priority algorithm strategy, and all instrument terminals to be connected are allocated to different clients, so that the number of instrument terminals connected to each client is maintained within a certain range, and the system resources consumed are maintained within a certain range.

[0080] d. Currently, after testing, resource utilization is within 70%, and the system is running relatively stably, so the threshold is set to 0.7. When more instrument terminals require access and the load threshold exceeds 0.7, the client service can be automatically called up and the relevant information can be configured to complete terminal data access.

[0081] Furthermore, based on the total number of instrument terminals that need to be connected and the number of instrument terminals actually supported by a single client in the cloud platform, a scheduling policy is set to maintain the number of instrument terminals connected to each agent within a certain range. When more instrument terminals need to be connected, the newly added client service can be automatically called up and the relevant information can be configured to complete the terminal data access.

[0082] In summary, in the present invention, plug-in integration is used to implement the encapsulation of multiple communication protocols, and the adapter mode is used to design a unified interface to achieve data interaction and communication between different protocols.

[0083] In addition, various protocol plug-ins are integrated into a client, which also has the ability to push the accessed data in the form of data packets to the message middleware Pulsar in real time to complete real-time data access.

[0084] Regarding the above-mentioned control through the client plug-in, in the present invention, the following detailed process is provided:

[0085] a. After the client service is successfully launched, it needs to register the service in the aggregation gateway and store its service name, IP address information, type, and externally exposed interface information in the cloud platform into the database.

[0086] b. After the client service is registered, it will continue to send heartbeat information to the aggregation gateway, recording the current client ID, the total client load, the current client load, the number of currently connected terminals, the number of currently connected normal terminals, and the load and status of each plug-in.

[0087] c. The service management module can perform instruction control on specific client services according to actual business needs, and control the registration, testing, start, stop and other operations of the service plug-in when it connects to the terminal.

[0088] In summary, in the present invention, an availability-aware priority algorithm strategy is implemented based on the availability requirements of data access and the availability information of each resource. The load threshold is configured as the scheduling trigger condition. Kubernetes is combined to start the service and distribute the scheduler tasks. The instrument terminals are allocated to different clients to ensure that the number of instrument terminal accesses and system resource consumption of each client are within a certain range, ultimately ensuring the automatic control and high stability of the service.

[0089] The client then uses the stateless service as a carrier to create a container image containing the service code and runtime environment. This image can be built using Docker or other containerization technologies and uploaded to a container image repository.

[0090] Regarding the above-mentioned reception of seismic data, the present invention provides the following detailed process:

[0091] a. Protocol plug-in network connection. A network connection is established between the client and server, using the TCP / IP protocol for data transmission. A protocol handshake is performed between the client and server to ensure that both parties can correctly understand and parse the seismic data transmission protocol. The handshake process includes sending and receiving information such as version numbers and handshake flags.

[0092] b. The protocol plug-in parses the data format. The client parses the received data according to the format of the seismic data transmission protocol. According to the protocol definition, the data is divided into different parts, such as the data header and data body. The parsing process requires identifying and extracting different information based on the data structure and identification fields.

[0093] c. Verify data integrity. When parsing data, a hash method is used to verify data integrity. According to the protocol definition, the data contains a checksum field to ensure that the data is not damaged or lost during transmission.

[0094] d. Data decoding and processing. Decode the parsed data according to the encoding method defined in the seismic data transmission protocol. The parsed and decoded seismic data is then further cleaned and processed, including deduplication, sorting, and formatting, to meet application requirements.

[0095] f. Close the connection. After the data transfer is complete, the client and server can close the connection to release resources and network ports.

[0096] In summary, in the present invention, instruction control is performed for specific client services to control the registration, testing, starting, stopping and other operations of the plug-in in the service when it connects to the terminal, thereby ensuring the controllability of the service.

[0097] Regarding the above-mentioned pushing of seismic data, the present invention provides the following detailed process:

[0098] a. Based on the specific subsystem data requirements, agree on the naming conventions for topics in Pulsar and automatically create topics using the three-level restriction principle of Tenant, Namespace, and Topic.

[0099] b. After parsing the packet header information of the access data, the client protocol plug-in pushes the different subsystem data to the corresponding topic according to the specification for subsequent system use.

[0100] In summary, in the present invention, after parsing the header information of the access data, the client protocol plug-in pushes different subsystem data to the corresponding topic according to the specification to ensure subsequent system use.

[0101] Regarding the above-mentioned fault recovery and smooth upgrade, the present invention provides the following detailed process:

[0102] a. In the application's Pod configuration, define health checks, configure livenessProbe and readinessProbe, and regularly check whether the container is running normally.

[0103] b. Configure automatic failover. When a pod fails, Kubernetes automatically migrates the pod on that node to another healthy node. By leveraging the collaborative work of the container runtime and control plane, Kubernetes can quickly restore service after a failure.

[0104] c. Perform a smooth upgrade. Trigger a smooth upgrade by updating the image version in the deployment resource or using the kubectl command. Kubernetes will gradually start pods of the new version and stop pods of the old version. Throughout the upgrade, the cluster will remain available, with no perceived service interruption.

[0105] d. Fault tolerance and rollback. If errors or issues are detected during the upgrade process, you can roll back to the previous version. Roll back to the old version of the deployment by updating the image version of the deployment resource or using the kubectl command. Kubernetes will start the pods of the old version and stop the pods of the new version to roll back to the previous state.

[0106] Regarding the configuration of the Pod, the present invention creates a Pod description file to define the Pod configuration, including the container image name, resource requirements, environment variables, port mapping, etc. A Service object is created to expose the Pod to other services or users inside or outside the cluster. The Service can provide a stable network access address for the Pod and distribute traffic to the backend Pod based on the load balancing algorithm.

[0107] In summary, in the present invention, the high stability of data access and disaster recovery processing when failures occur are ensured through the configuration of health checks and automatic failover in Kubernetes.

[0108] Based on the same inventive concept, another embodiment of the present invention provides an electronic device, including a processor, a communication interface, a memory, and a communication bus, wherein the processor, the communication interface, and the memory communicate with each other via the communication bus.

[0109] Memory for storing computer programs;

[0110] The processor is used to implement the big data access method for earthquake industry protocol adaptation and scheduling of the present invention when executing the program stored in the memory.

[0111] The communication bus mentioned in the above terminal can be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. The communication bus can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, only one thick line is used in the figure, but it does not mean that there is only one bus or one type of bus. The communication interface is used for communication between the above terminal and other devices. The memory can include a random access memory (RAM) or a non-volatile memory, such as at least one disk storage. Optionally, the memory can also be at least one storage system located away from the aforementioned processor.

[0112] The above-mentioned processor can be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it can also be a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, and discrete hardware components.

[0113] In addition, to achieve the above-mentioned purpose, an embodiment of the present invention also proposes a computer-readable storage medium storing a computer program, which, when executed by a processor, implements the big data access method for earthquake industry protocol adaptation and scheduling of an embodiment of the present invention.

[0114] Those skilled in the art will appreciate that embodiments of the present invention may be provided as methods, systems, or computer program products. Thus, embodiments of the present invention may take the form of entirely hardware embodiments, entirely software embodiments, or embodiments combining software and hardware aspects. Furthermore, embodiments of the present invention may take the form of a computer program product implemented on one or more computer-usable devices (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0115] The embodiments of the present invention are described with reference to the flowcharts and / or block diagrams of the methods, terminal devices (systems), and computer program products according to the embodiments of the present invention. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing terminal device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing terminal device produce a system for implementing the functions specified in one or more processes in the flowchart and / or one or more boxes in the block diagram.

[0116] These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable data processing terminal device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce a manufactured product including an instruction system that implements the functions specified in one or more processes in the flowchart and / or one or more boxes in the block diagram.

[0117] These computer program instructions can also be loaded onto a computer or other programmable data processing terminal device so that a series of operating steps are executed on the computer or other programmable terminal device to produce computer-implemented processing, so that the instructions executed on the computer or other programmable terminal device provide steps for implementing the functions specified in one or more processes in the flowchart and / or one or more boxes in the block diagram.

[0118] Finally, it should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply that there is any such actual relationship or order between these entities or operations. "And / or" means that either one of the two can be selected, or both can be selected. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that the process, method, article or terminal device that includes a series of elements includes not only those elements, but also other elements that are not explicitly listed, or also includes elements that are inherent to such process, method, article or terminal device. In the absence of further restrictions, the elements defined by the sentence "including a..." do not exclude the presence of other identical elements in the process, method, article or terminal device that includes the elements.

[0119] The above are merely specific embodiments of the present invention, but the scope of protection of the present invention is not limited thereto. Any person skilled in the art can easily conceive of various equivalent modifications or substitutions within the technical scope disclosed in the present invention, and such modifications or substitutions are intended to be within the scope of protection of the present invention. Therefore, the scope of protection of the present invention shall be subject to the scope of protection of the claims.

[0120] Finally, it should be noted that the above embodiments are merely illustrative of the technical solutions of the embodiments of the present invention, and are not intended to limit them. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they may still modify the technical solutions described in the aforementioned embodiments, or replace some of the technical features therein with equivalents; and such modifications or replacements do not deviate from the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention. Any changes or replacements that can be easily conceived by those skilled in the art within the technical scope disclosed in the present invention should be covered by the scope of protection of the present invention.

Claims

1. A big data access method for earthquake industry protocol adaptation and scheduling, characterized in that: The steps include: S1. Perform protocol adaptation; S2. Automatically start client policy scheduling; S3. Control through client plug-in; S4. receiving seismic data; S5. Push the seismic data; S6. Perform fault recovery and smooth upgrade.

2. A big data access method for earthquake industry protocol adaptation and scheduling according to claim 1, characterized in that: The protocol adaptation specifically includes the following steps: S11. Enter the information of the access instrument terminal; S12. Perform plug-in testing; S13. Test by issuing protocol adaptation instructions.

3. A big data access method for earthquake industry protocol adaptation and scheduling according to claim 1, characterized in that: The automatic client policy scheduling specifically includes the following steps: S21. Grouping instrument terminals of different protocol types, where the service protocols within each group belong to the same protocol type; S22. Then, according to the availability requirements of different seismic data access and the availability information of each resource, the tasks are divided into different priorities; S23. Then the total number of instrument terminals that need to be connected, the number of instrument terminals actually supported by a single client in the cloud platform, the number of monitoring tests, and the resource consumption of different protocol data are set up by setting the availability-aware priority algorithm strategy to start the service and distribute the tasks; S24. Set the load threshold to 70%, and then conduct a test. If the resource usage is within 70%, the system is considered to be running stably. When more instrument terminals need to be connected and the load threshold exceeds 70%, the client service is automatically called up, and the relevant information is configured to complete the terminal data access.

4. A big data access method for earthquake industry protocol adaptation and scheduling according to claim 3, characterized in that: The control through the client plug-in specifically includes the following steps: S31. When the client service is successfully started, the service is registered in the aggregation gateway, and its service name, IP address information, type, and externally exposed interface information in the cloud platform are stored in the database; S32. When the client service is registered, it sends heartbeat information to the aggregation gateway, recording the current client ID, the total client load, the current client load, the number of currently connected terminals, the number of currently connected normal terminals, the load of each plug-in itself and the status; S33. Instruct the service management module to control the operation of the plug-in in the control service when it connects to the terminal according to the actual business needs and the specific client services.

5. A big data access method for earthquake industry protocol adaptation and scheduling according to claim 1, characterized in that: The receiving of seismic data specifically includes the following steps: S41. Establish a network connection between the client and the server, and transmit data via TCP / IP protocol; S42. Parse the data format through the protocol plug-in and divide the data into different parts according to the protocol definition; S43. Analyze the data and verify the integrity of the data; S44. Decoding the parsed data according to the encoding method defined in the seismic data transmission protocol, and further cleaning the parsed and decoded seismic data; S45. After the data transmission is completed, the connection between the client and the server is closed to release resources and network ports.

6. A big data access method for earthquake industry protocol adaptation and scheduling according to claim 5, characterized in that: The pushing of seismic data specifically includes the following steps: S51. According to the specific subsystem data requirements, agree on the naming convention of topics in Pulsar, and automatically create topics through the three-level restriction principle of Tenant, Namespace, and Topic; S52. After parsing the packet header information of the access data, the client protocol plug-in pushes the different subsystem data to the corresponding topic for subsequent system use.

7. A big data access method for earthquake industry protocol adaptation and scheduling according to claim 5, characterized in that: The fault recovery and smooth upgrade specifically include the following steps: S61. In the Pod configuration of the application, define health checks and configure livenessProbe and readinessProbe to regularly check the running status of the container; S62. When a Pod fails, Kubernetes will automatically transfer the Pod on the current node to other healthy nodes; S63. Trigger a smooth upgrade by updating the image version in the deployment resource or using the kubectl command, gradually start the new version of the Pod through Kubernetes, and gradually stop the old version of the Pod; S64. When errors or other problems are detected during the upgrade process, automatically roll back to the previous version. By updating the image version of the deployment resource or using the kubectl command, roll back to the old version of the deployment. Kubernetes will start the old version The original Pod and stop the new version of the Pod to roll back to the previous state.

8. A big data access method for earthquake industry protocol adaptation and scheduling according to claim 5, characterized in that: In S44, the cleaning process includes deduplication, sorting and formatting of the data.

9. A big data access method for earthquake industry protocol adaptation and scheduling according to claim 5, characterized in that: In S43, the method for verifying the integrity of the data adopts a hash method.

10. A big data access method for earthquake industry protocol adaptation and scheduling according to claim 4, characterized in that: In S33, the operations include registration, testing, starting, and stopping.

Images