Rail transit data transfer method and device

Abstract

The present invention provides a rail transit data transfer method and device, the method comprising: determining a target storage space corresponding to the target data to be transferred based on configuration information of the target data to be transferred; determining a transfer strategy corresponding to the target data to be transferred based on the type of the target storage space or the protocol corresponding to the target storage space; and transferring the target data to be transferred to the target storage space based on the transfer strategy. The rail transit data transfer method and device provided by the present invention can more simply and efficiently implement rail transit data transfer when the data source and storage space types are different, can simultaneously transfer data to multiple storage spaces, and can improve the flexibility of rail transit data transfer.

Description

Technical Field

[0001] The present invention relates to the field of rail transit technology, and in particular to a rail transit data transfer method and device. Background Art

[0002] With the development of science and technology, rail transit has greatly facilitated people's lives and has become an indispensable part of modern life.

[0003] The normal operation of rail transit relies on the support of various rail transit systems. During the actual operation of rail transit, the data generated by these systems can be called rail transit data. There is a need to transfer rail transit data between different rail transit systems.

[0004] However, existing rail transit data transfer methods are difficult to achieve simple and efficient transfer of rail transit data. Therefore, how to transfer rail transit data more simply and efficiently is a technical problem that needs to be solved in this field. Summary of the Invention

[0005] The present invention provides a rail transit data transfer method and device, which are used to solve the defect in the prior art that it is difficult to simply and efficiently transfer rail transit data, and to achieve simpler and more efficient transfer of rail transit data.

[0006] The present invention provides a rail transit data transfer method, comprising:

[0007] Get the target data to be transferred;

[0008] Determining a target storage space corresponding to the target data to be transferred based on the configuration information of the target data to be transferred;

[0009] Determining a transfer policy corresponding to the target data to be transferred based on the type of the target storage space or the protocol corresponding to the target storage space;

[0010] Based on the transfer strategy, the target data to be transferred is transferred to the target storage space.

[0011] According to a rail transit data transfer method provided by the present invention, determining a transfer strategy corresponding to the target data to be transferred based on the type of the target storage space includes:

[0012] In a case where the type of the target storage space is MySQL or SQLite, determining that the transfer strategy corresponding to the target data to be transferred includes a first transfer strategy;

[0013] Among them, the first transfer strategy includes transferring the target data to be transferred to the target storage space based on the Mybatis framework and the Springboot framework.

[0014] According to a rail transit data transfer method provided by the present invention, determining a transfer strategy corresponding to the target data to be transferred based on the type of the target storage space includes:

[0015] In a case where the type of the target storage space is a message queue, determining that the transfer strategy corresponding to the target data to be transferred includes a second transfer strategy;

[0016] Among them, the second transfer strategy includes obtaining a message queue executor based on the configuration information of the target storage space, parsing the target data to be transferred using JSONObject, and then transferring the parsed target data to be transferred to the target storage space based on the message queue executor.

[0017] According to a rail transit data transfer method provided by the present invention, determining a transfer strategy corresponding to the target data to be transferred based on a protocol corresponding to the target storage space includes:

[0018] In a case where the protocol corresponding to the target storage space is HTTP protocol, UDP protocol or TCP protocol, determining that the transfer strategy corresponding to the target data to be transferred includes a third transfer strategy;

[0019] The third transfer strategy includes transferring the target data to be transferred to the target storage space based on a protocol corresponding to the target storage space.

[0020] According to a rail transit data transfer method provided by the present invention, determining a transfer strategy corresponding to the target data to be transferred based on the type of the target storage space includes:

[0021] In a case where the type of the target storage space is ES, determining that the transfer strategy corresponding to the target data to be transferred includes a fourth transfer strategy;

[0022] The fourth transfer strategy includes obtaining an implementation class corresponding to the target storage space based on an operation type of the target storage space, and transferring the target data to be transferred to the target storage space based on the implementation class.

[0023] According to a rail transit data transfer method provided by the present invention, obtaining target data to be transferred includes:

[0024] Get the original data to be transferred;

[0025] Performing data verification on the original data to be transferred;

[0026] When the original data to be transferred passes the data verification, performing data processing on the original data to be transferred;

[0027] Determining the original data to be transferred after data processing as the target data to be transferred;

[0028] Wherein, the data verification includes data format verification; the data processing includes data filtering.

[0029] According to a rail transit data transfer method provided by the present invention, transferring the target data to be transferred to the target storage space based on the transfer strategy includes:

[0030] In a case where the target storage space passes the verification, the target data to be transferred is transferred to the target storage space based on the transfer policy.

[0031] The present invention also provides a rail transit data transfer device, comprising:

[0032] The transfer data acquisition module is used to obtain the target data to be transferred;

[0033] A storage space determining module, configured to determine a target storage space corresponding to the target data to be transferred based on configuration information of the target data to be transferred;

[0034] A transfer strategy determination module, configured to determine a transfer strategy corresponding to the target data to be transferred based on the type of the target storage space or the protocol corresponding to the target storage space;

[0035] The data transfer module is used to transfer the target data to be transferred to the target storage space based on the transfer strategy.

[0036] The present invention also provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, the rail transit data transfer method as described above is implemented.

[0037] The present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon. When the computer program is executed by a processor, the rail transit data transfer method as described in any one of the above is implemented.

[0038] The present invention also provides a computer program product, comprising a computer program, wherein when the computer program is executed by a processor, the computer program implements any of the above-mentioned rail transit data transfer methods.

[0039] The rail transit data transfer method and device provided by the present invention determine the target storage space corresponding to the target data to be transferred based on the configuration information of the target data to be transferred, and then determine the transfer strategy corresponding to the target data to be transferred based on the type or corresponding protocol of the above-mentioned target storage space. Then, based on the above-mentioned transfer strategy, the target data to be transferred is transferred to the above-mentioned target storage space. This method and device can realize the transfer of rail transit data more simply and efficiently when the data source and storage space type are different, can transfer data to multiple storage spaces synchronously, and can improve the flexibility of rail transit data transfer. BRIEF DESCRIPTION OF THE DRAWINGS

[0040] In order to more clearly illustrate the technical solutions in the present invention or the prior art, a brief introduction is given below to the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0041] Figure 1 This is one of the flow charts of the rail transit data transfer method provided by the present invention;

[0042] Figure 2 This is the second flow chart of the rail transit data transfer method provided by the present invention;

[0043] Figure 3 It is a structural schematic diagram of the rail transit data transfer device provided by the present invention;

[0044] Figure 4 It is a structural schematic diagram of the electronic device provided by the present invention. DETAILED DESCRIPTION

[0045] To make the objectives, technical solutions, and advantages of the present invention more clear, the technical solutions of the present invention will be clearly and completely described below in conjunction with the accompanying drawings. Obviously, the embodiments described are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts shall fall within the scope of protection of the present invention.

[0046] It should be noted that the data transmission method adopted by the rail transit system is usually specific, and the data transmission methods adopted by different rail transit systems may be different. The data transmission of the rail transit system has the characteristics of strong limitations, poor universality and high expansion cost.

[0047] Since it is difficult to unify the data transmission methods used by different rail transit systems, it is difficult to simply and efficiently transfer rail transit data when there is a need to transfer rail transit data between different rail transit systems, which creates obstacles to the coordination of multiple rail transit systems.

[0048] To address this issue, the present invention provides a rail transit data transfer method based on the HTTP protocol. This method can more simply and efficiently transfer data between different types of databases, allowing for simultaneous data transfer to multiple databases, thereby increasing the flexibility of rail transit data transfer.

[0049] Figure 1 This is one of the flow charts of the rail transit data transfer method provided by the present invention. Figure 1 The rail transit data transfer method of the present invention is described. Figure 1 As shown, the method includes: step 101, obtaining target data to be transferred.

[0050] It should be noted that the execution subject of the embodiment of the present invention is a rail transit data transfer device.

[0051] Specifically, in an embodiment of the present invention, the target data to be transferred is rail transit data. The target data to be transferred is the transfer target of the rail transit data transfer method provided by the present invention. The target storage space is the transfer destination of the target data to be transferred. Based on the rail transit data transfer method provided by the present invention, the target data to be transferred can be transferred to the target storage space.

[0052] It should be noted that, in the embodiments of the present invention, the data source of the target data to be transferred may be referred to as the data source to be transferred. The type of the data source to be transferred may be the same as or different from the type of the target storage space; and the protocol corresponding to the data source to be transferred may also be the same as or different from the protocol corresponding to the target storage space.

[0053] It should be noted that the number of target storage spaces in the embodiment of the present invention may be one or more. For example, the target storage space may be a MySQL database; or the target storage space may include a MySQL database, an Oracle database, and a Kafka message queue.

[0054] In embodiments of the present invention, the target data to be transferred can be obtained in a variety of ways. For example, the original data to be transferred output by the data source to be transferred can be obtained as the target data to be transferred; or, after obtaining the original data to be transferred output by the data source to be transferred, the original data to be transferred can be processed, and then the processed original data to be transferred can be determined as the target data to be transferred. The specific method for obtaining the target data to be transferred is not limited in embodiments of the present invention.

[0055] As an optional embodiment, obtaining target data to be transferred includes: obtaining original data to be transferred.

[0056] It should be noted that the rail transit data transfer method provided by the present invention is based on the HTTP protocol, and what is exposed to the outside is the HTTP interface.

[0057] Accordingly, in the embodiment of the present invention, the original data to be transferred output by the above-mentioned data source to be transferred can be obtained based on the HTTP interface.

[0058] Perform data verification on the original data to be transferred; data verification includes data format verification.

[0059] Specifically, after obtaining the original data to be transferred, data verification may be performed on the original data to be transferred.

[0060] Optionally, in an embodiment of the present invention, data verification of the original data to be transferred may include: performing format verification on the original data to be transferred based on a preset format, and performing integrity verification and anti-duplicate submission verification on the original data to be transferred.

[0061] When the original data to be transferred passes the data verification, data processing is performed on the original data to be transferred;

[0062] Among them, data processing includes data filtering.

[0063] Specifically, after data verification is performed on the original data to be transferred, if the original data to be transferred passes the data verification, data processing may be performed on the original data to be transferred.

[0064] It should be noted that, in the embodiment of the present invention, the data processing of the above-mentioned original data to be transferred may include but is not limited to data filtering.

[0065] The original data to be transferred after data processing is determined as the target data to be transferred.

[0066] Specifically, after data processing is performed on the original data to be transferred, the processed original data to be transferred may be determined as the target data to be transferred.

[0067] The embodiment of the present invention performs data verification on the original data to be transferred. If the original data to be transferred passes the data verification, the original data to be transferred is further processed, and the original data to be transferred after data processing is determined as the target data to be transferred. This can improve the security of the transferred target data to be transferred and avoid transfer failures caused by format errors / duplications and other problems.

[0068] Step 102: Determine a target storage space corresponding to the target data to be transferred based on the configuration information of the target data to be transferred.

[0069] Specifically, the configuration information of the target data to be transferred may include an interface request parameter topic_id.

[0070] After obtaining the target data to be transferred, the target storage space corresponding to the target data to be transferred can be determined based on the topic_id in the target data to be transferred.

[0071] It should be noted that after determining the target storage space corresponding to the target data to be transferred, configuration information of the target storage space may be further obtained, wherein the configuration information may include the type of the target storage space and / or the protocol corresponding to the target storage space.

[0072] It should be noted that the types of storage spaces in the embodiments of the present invention may include but are not limited to MySQL databases, Qracle databases, ES databases, and message queues; the protocols corresponding to the storage spaces may include but are not limited to HTTP protocols, UDP protocols, and TCP protocols.

[0073] It should be noted that, based on the configuration information of the target storage space, the target table structure can also be queried, and the queried target table structure can be updated.

[0074] Step 103: Determine a transfer strategy corresponding to the target data to be transferred based on the type of the target storage space or the protocol corresponding to the target storage space.

[0075] Specifically, after determining the target storage space corresponding to the target data to be transferred, the type of the target storage space or the protocol corresponding to the target storage space may be determined based on the configuration information of the target storage space.

[0076] After determining the type of the target storage space or the protocol corresponding to the target storage space, the transfer strategy corresponding to the target data to be transferred can be determined based on the correspondence between the type of storage space and the preset transfer strategy, or based on the correspondence between the protocol corresponding to the storage space and the preset transfer strategy.

[0077] It should be noted that the preset transfer strategy, the correspondence between the storage space type and the preset transfer strategy, and the correspondence between the protocol corresponding to the storage space and the preset transfer strategy can be predetermined based on prior knowledge and / or actual circumstances. In the embodiments of the present invention, the preset transfer strategy, the correspondence between the storage space type and the preset transfer strategy, and the correspondence between the protocol corresponding to the storage space and the preset transfer strategy are not specifically limited.

[0078] Step 104: Based on the transfer strategy, transfer the target data to be transferred to the target storage space.

[0079] Specifically, after determining the transfer policy corresponding to the target data to be transferred, the target data to be transferred may be transferred based on the transfer policy, and the target data to be transferred may be transferred to the target storage space.

[0080] As an optional embodiment, transferring the target data to be transferred to the target storage space based on the transfer policy includes: if the target storage space passes the verification, transferring the target data to be transferred to the target storage space based on the transfer policy.

[0081] Specifically, in an embodiment of the present invention, before executing the transfer policy corresponding to the target data to be transferred, the target storage space can be verified to determine whether the target storage space is empty, thereby avoiding the transfer failure of the target data to be transferred due to the empty target storage space.

[0082] When the target storage space passes the verification, it can be determined that the target storage space is not empty. In this case, the target data to be transferred can be transferred to the target storage space based on a transfer policy corresponding to the target data to be transferred.

[0083] It should be noted that after the target data to be transferred is transferred to the target storage space, the transfer result can be returned to the interface caller.

[0084] The embodiment of the present invention determines the target storage space corresponding to the target data to be transferred based on the configuration information of the target data to be transferred, and then determines the transfer strategy corresponding to the target data to be transferred based on the type of the above-mentioned target storage space or the corresponding protocol, and then transfers the target data to be transferred to the above-mentioned target storage space based on the above-mentioned transfer strategy. This can realize the transfer of rail transit data more simply and efficiently when the data source and storage space type are different, and can synchronously transfer data to multiple storage spaces, which can improve the flexibility of rail transit data transfer.

[0085] Based on the contents of the above embodiments, determining the transfer strategy corresponding to the target data to be transferred based on the type of the target storage space includes: when the type of the target storage space is MySQL, determining that the transfer strategy corresponding to the target data to be transferred includes a first transfer strategy;

[0086] Among them, the first transfer strategy includes transferring the target data to be transferred to the target storage space based on the Mybatis framework and the Springboot framework.

[0087] It should be noted that MySQL is a relational database management system. The MySQL database can increase data storage speed and improve data storage flexibility by saving data in different tables.

[0088] It should be noted that, when the number of target storage spaces is one, if the type of the target storage space is MySQL, it can be determined that the target storage space is a MySQL database;

[0089] In the case that there are multiple target storage spaces, if the type of any target storage space is MySQL, it can be determined that the target storage space is a MySQL database.

[0090] Specifically, when the type of the target storage space is MySQL, based on the correspondence between the type of the storage space and the preset transfer strategy, it can be determined that the transfer strategy corresponding to the target data to be transferred includes a first transfer strategy, and then based on the first transfer strategy, the target data to be transferred can be transferred to the above-mentioned MySQL database. The above-mentioned first transfer strategy includes transferring the target transfer data to the above-mentioned MySQL database based on the Mybatis framework and the Springboot framework.

[0091] Based on the above first transfer strategy, the target data to be transferred can be transferred to the above MySQL database through the DBHandler's data transfer method.

[0092] Based on the above-mentioned first transfer strategy, the specific steps of transferring the target data to be transferred to the above-mentioned MySQL database include: first, based on the Mybatis framework and the DataSourceID in the configuration information of the target storage space, the transfer path of the target data to be transferred can be switched to the above-mentioned MySQL database.

[0093] Mybatis is a persistence layer framework that supports customized SQL, stored procedures, and advanced mapping, eliminating the need for nearly all JDBC code and manual parameter setting and result set retrieval. Mybatis can also use simple XML or annotations to configure and map native information, mapping interfaces and Java POJOs (Plain Ordinary Java Objects) to records in the database.

[0094] Springboot is a framework that simplifies Spring development and provides a quick way to use Spring.

[0095] In the embodiment of the present invention, the environment architecture for transferring the target transfer data to the above-mentioned MySQL database is a Springboot framework.

[0096] It should be noted that based on the Mybatis framework and the DataSourceID in the target storage space's configuration information, the MySQL database can be retrieved from the cache. If the MySQL database is not found in the cache, the target data to be transferred can be switched to the default storage space. All storage spaces can then be traversed to retrieve a storage space that matches the DataSourceID. Once a storage space matching the DataSourceID is found, the storage space can be assigned to the dynamic storage space manager and saved to the cache.

[0097] Secondly, according to the operation type information (on Type) in the configuration information of the target data to be transferred, an SQL generation strategy corresponding to the operation type information may be generated to generate an SQL statement corresponding to the target data to be transferred.

[0098] Finally, based on the Mybatis framework, the above SQL statements can be executed to transfer the target data to the MySQL database in the target storage space.

[0099] The embodiment of the present invention determines that the transfer strategy corresponding to the target data to be transferred out includes a first transfer strategy when the type of the target storage space is MySQL. The above-mentioned first transfer strategy includes transferring the target data to be transferred to a MySQL database based on the Mybatis framework and the Springboot framework, which can transfer the target data to be transferred to the MySQL database more simply and efficiently.

[0100] It should be noted that Oracle is a relational database management system with good portability, ease of use, and powerful functions. It is suitable for all types of large, medium, and small computer environments.

[0101] It should be noted that, when the number of target storage spaces is one, if the type of the target storage space is Oracle, it can be determined that the target storage space is an Oracle database;

[0102] In the case that there are multiple target storage spaces, if the type of any target storage space is Oracle, it can be determined that the target storage space is an Oracle database.

[0103] Specifically, when the type of the target storage space includes Oracle, it can be determined based on the correspondence between the type of storage space and the preset transfer strategy that the transfer strategy corresponding to the target data to be transferred includes the first transfer strategy, and then based on the first transfer strategy, the target data to be transferred can be transferred to the above-mentioned Oracle database.

[0104] Based on the first transfer strategy, the target data to be transferred can be transferred to the Oracle database through the DBHandler data transfer method.

[0105] Based on the above-mentioned first transfer strategy, the specific steps of transferring the target data to be transferred to the above-mentioned Oracle database include: first, based on the Mybatis framework and the DataSourceID in the configuration information of the target storage space, the transfer path of the target data to be transferred can be switched to the above-mentioned Oracle database.

[0106] Secondly, according to the operation type information (on Type) in the configuration information of the target data to be transferred, an SQL generation strategy corresponding to the operation type information may be generated to generate an SQL statement corresponding to the target data to be transferred.

[0107] Finally, based on the Mybatis framework, the above SQL statements can be executed to transfer the target data to the Oracle database in the target storage space.

[0108] The embodiment of the present invention determines that the transfer strategy corresponding to the target data to be transferred out includes a first transfer strategy when the type of the target storage space is Oracle. The above-mentioned first transfer strategy includes transferring the target data to be transferred to the Oracle database based on the Mybatis framework and the Springboot framework, which can transfer the target data to be transferred to the Oracle database more simply and efficiently.

[0109] Based on the contents of the above embodiments, determining the transfer strategy corresponding to the target data to be transferred based on the type of the target storage space includes: when the type of the target storage space is a message queue, determining that the transfer strategy corresponding to the target data to be transferred includes a second transfer strategy;

[0110] Among them, the second transfer strategy includes obtaining a message queue executor based on the configuration information of the target storage space, parsing the target data to be transferred using JSONObject, and then transferring the parsed target data to be transferred to the target storage space based on the message queue executor.

[0111] It should be noted that Message Queue (MQ) usually refers to message middleware that can forward messages.

[0112] It should be noted that, when the number of target storage spaces is one, if the type of the target storage space is a message queue, it can be determined that the target storage space is a message queue;

[0113] In the case that there are multiple target storage spaces, if the type of any target storage space is a message queue, it can be determined that the target storage space is a message queue.

[0114] It should be noted that the message queue in the embodiment of the present invention may include a Kafka message queue and / or an ActiveMq message queue.

[0115] Specifically, when the type of the target storage space is a message queue, it can be determined based on the correspondence between the type of storage space and the preset transfer strategy that the transfer strategy corresponding to the target data to be transferred includes a second transfer strategy, and then based on the second transfer strategy, the target data to be transferred can be transferred to the above-mentioned message queue.

[0116] Based on the second transfer strategy, the specific steps of transferring the target data to be transferred out to the message queue in the target storage space may include: first, generating a KEY value based on the topicId and URL in the configuration information of the target storage space; obtaining a message queue executor (MqProducer) from the cache based on the KEY value;

[0117] If the message queue executor is not obtained from the cache based on the above KEY value, the type of the target storage space (MqType) can be obtained based on the code attribute information in the configuration information of the target storage space, and then the above message queue executor can be generated based on the above MqType, topicId and URL.

[0118] Secondly, use JSONObject to parse the target data to be transferred, and parse the JSON data in the target data to be transferred into a JSONObejct object.

[0119] Finally, based on the above message queue executor, the parsed target data to be transferred is transferred to the message queue.

[0120] An embodiment of the present invention determines that, when the type of the target storage space is a message queue, the transfer strategy corresponding to the target data to be transferred out includes a second transfer strategy. The above-mentioned second transfer strategy includes obtaining a message queue executor based on the configuration information of the target storage space, using JSONObject to parse the target data to be transferred, and then transferring the parsed target data to be transferred to the message queue based on the message queue executor. This can make it simpler and more efficient to transfer the target data to be transferred to the message queue.

[0121] Based on the contents of the above embodiments, determining the transfer strategy corresponding to the target data to be transferred based on the protocol corresponding to the target storage space includes: when the protocol corresponding to the target storage space is HTTP protocol, UDP protocol or TCP protocol, determining that the transfer strategy corresponding to the target data to be transferred includes a third transfer strategy;

[0122] The third transfer strategy includes transferring the target data to be transferred to the target storage space based on the protocol corresponding to the target storage space.

[0123] Specifically, when the protocol corresponding to the target storage space is the HTTP protocol, it can be determined that the transfer policy corresponding to the target data to be transferred includes a third transfer policy, and then the target transfer data can be transferred to the target storage space based on the third transfer protocol. The third transfer policy includes transferring the target data to be transferred to the target storage space based on the HTTP protocol.

[0124] In the case that the protocol corresponding to the target storage space is the HTTP protocol, based on the above-mentioned third transfer strategy, the target data to be transferred can be transferred to the target storage space through the storage data method of HttpHandler.

[0125] In the case where the protocol corresponding to the target storage space is HTTP, based on the third transfer strategy, the specific steps of transferring the target data to be transferred to the target storage space include: first, obtaining the URL and code attribute information in the configuration information of the target storage space;

[0126] Secondly, using httpclient technology, create HttpPost, RequestConfig and other objects, set corresponding properties, and execute HTTP requests to transfer the target data to the target storage space.

[0127] When the protocol corresponding to the target storage space is the UDP protocol, it can be determined that the transfer strategy corresponding to the target data to be transferred includes a third transfer strategy, and then the target transfer data can be transferred to the target storage space based on the third transfer protocol. The third transfer strategy includes transferring the target data to be transferred to the target storage space based on the UDP protocol.

[0128] In the case that the protocol corresponding to the target storage space is the UDP protocol, based on the above-mentioned third transfer strategy, the target data to be transferred can be transferred to the target storage space through the storage data method of UdpHandle.

[0129] When the protocol corresponding to the target storage space is UDP, based on the third transfer strategy, the specific steps of transferring the target data to be transferred to the target storage space include: first, obtaining the URL and topicId in the configuration information of the target storage space, and then obtaining the IP and port of the target storage space by parsing the URL;

[0130] Secondly, a DatagramPacket object is created, and the target data to be transferred is transferred to the target storage space through the DatagramSocket object, so that the target data to be transferred can be transferred to the target storage space by establishing a DatagramPacket object based on the TCP protocol.

[0131] Specifically, when the protocol corresponding to the target storage space is the TCP protocol, it can be determined that the transfer strategy corresponding to the target data to be transferred includes a third transfer strategy, and then the target transfer data can be transferred to the target storage space based on the third transfer protocol. The third transfer strategy includes transferring the target data to be transferred to the target storage space based on the TCP protocol.

[0132] In the case that the protocol corresponding to the target storage space is the TCP protocol, based on the above-mentioned third transfer strategy, the target data to be transferred can be transferred to the target storage space through the storage data method of TcpHandle.

[0133] In the case where the protocol corresponding to the target storage space is the TCP protocol, based on the above-mentioned third transfer strategy, the specific steps of transferring the target data to be transferred to the target storage space include: first, obtaining the URL and topicId in the configuration information of the target storage space, and obtaining the IP and port of the target storage space by parsing the above-mentioned URL, and parsing the target data to be transferred, parsing the JSON type data in the target data to be transferred into byte data packets;

[0134] Secondly, create a Socket object and obtain the output stream OutputStream, and transfer the target data to the target storage space through the output stream, so that the target data to be transferred can be transferred to the target storage space by establishing a socket based on the TCP protocol.

[0135] Based on the contents of the above embodiments, determining the transfer strategy corresponding to the target data to be transferred based on the type of the target storage space includes: when the type of the target storage space is ES, determining that the transfer strategy corresponding to the target data to be transferred includes a fourth transfer strategy;

[0136] The fourth transfer strategy includes obtaining an implementation class corresponding to the target storage space based on an operation type of the target storage space, and transferring the target data to be transferred to the target storage space based on the implementation class.

[0137] It should be noted that Elasticsearch (ES for short) is a distributed, highly scalable, and highly real-time search and data analysis engine that has the ability to search, analyze, and explore large amounts of data.

[0138] It should be noted that, when the number of target storage spaces is one, if the type of the target storage space is ES, it can be determined that the target storage space is an ES database;

[0139] In the case that there are multiple target storage spaces, if the type of any target storage space is ES, it can be determined that the target storage space is an ES database.

[0140] Specifically, when the type of the target storage space is ES, it can be determined based on the correspondence between the type of storage space and the preset transfer strategy that the transfer strategy corresponding to the target data to be transferred includes the fourth transfer strategy, and then based on the fourth transfer strategy, the target data to be transferred can be transferred to the above-mentioned ES database.

[0141] Based on the fourth transfer strategy, the specific steps of transferring the target data to be transferred to the ES database include: first, the operation type (opType) of the ES database can be obtained in ESContext, and the EsHandler implementation class corresponding to the ES database can be obtained according to the operation type, such as ESInsertHandler, ESUpdateHandler, ESDeleteHandler, and ESReplaceHandler; wherein ESHandler is the processor corresponding to the ES database;

[0142] Secondly, obtain the URL and topicId in the configuration information of the ES database, and obtain the IP and port of the ES database by parsing the above URL, and then initialize the RestHighLevelClient based on the IP and port of the ES database;

[0143] Again, based on the topicId in the ES database configuration information, create an IndexRequest and set the type, id, and data resource;

[0144] Finally, transmit the IndexRequest through RestHighLevelClient.

[0145] The embodiment of the present invention determines that the transfer strategy corresponding to the target data to be transferred out includes a fourth transfer strategy when the type of the target storage space is ES. The above-mentioned first transfer strategy includes the fourth transfer strategy, including an operation type based on the target storage space, obtains the implementation class corresponding to the target storage space, and transfers the target data to be transferred to the target storage space based on the implementation class. This can make it simpler and more efficient to transfer the target data to be transferred to the ES database.

[0146] In order to facilitate understanding of the rail transit data transfer method provided by the present invention, the rail transit data transfer method provided by the present invention is described below through an example.

[0147] Figure 2 This is the second flow chart of the rail transit data transfer method provided by the present invention. Figure 2 As shown, after calling the interface and obtaining the original data to be transferred, data verification can be performed on the above original data to be transferred to determine whether the above original data to be transferred has format errors and whether it includes duplicate data. If the above original data to be transferred passes the data verification, data processing can be performed on the above original data to be transferred to obtain the target data to be transferred.

[0148] In the case that the original data to be transferred fails to pass the data verification, a prompt may be given on the user interaction interface to remind the user that the format of the original data to be transferred is incorrect, incomplete or duplicated.

[0149] Based on the configuration information of the target data to be transferred, the target storage space corresponding to the target data to be transferred can be determined, and it can be determined whether the target storage space is empty.

[0150] In the case that the target storage space is empty, a prompt may be given on the user interaction interface to prompt the user that the target storage space is empty.

[0151] When the target storage space is not empty, the transfer protocol may be determined based on the type of the target storage space or the protocol corresponding to the target storage space, and the target data to be transferred may be transferred to the target storage space based on the transfer protocol.

[0152] After the target data to be transferred is transferred to the target storage space, the transfer result can be returned to the interface caller.

[0153] Figure 3 This is a schematic diagram of the structure of the rail transit data transfer device provided by the present invention. Figure 3 The rail transit data transfer device provided by the present invention is described. The rail transit data transfer device described below and the rail transit data transfer method provided by the present invention described above can be referred to each other. Figure 3 As shown, there is a transfer data acquisition module 301, a storage space determination module 302, a transfer strategy determination module 303 and a data transfer module 304.

[0154] The transfer data acquisition module 301 is used to obtain target data to be transferred;

[0155] The storage space determining module 302 is configured to determine a target storage space corresponding to the target data to be transferred based on the configuration information of the target data to be transferred;

[0156] A transfer strategy determination module 303 is configured to determine a transfer strategy corresponding to the target data to be transferred based on the type of the target storage space or the protocol corresponding to the target storage space;

[0157] The data transfer module 304 is configured to transfer the target data to be transferred to the target storage space based on the transfer strategy.

[0158] Specifically, the transfer data acquisition module 301 , the storage space determination module 302 , the transfer strategy determination module 303 and the data transfer module 304 are electrically connected.

[0159] The transfer data acquisition module 301 can obtain the target data to be transferred in a variety of ways. For example, it can obtain the original data to be transferred output by the above-mentioned data source to be transferred as the target data to be transferred; or it can obtain the original data to be transferred output by the above-mentioned data source to be transferred, perform data processing on the original data to be transferred, and then determine the processed original data to be transferred as the target data to be transferred. The specific method for obtaining the target data to be transferred is not limited in the embodiments of the present invention.

[0160] The storage space determining module 302 may determine the target storage space corresponding to the target data to be transferred based on the topic_id in the target data to be transferred.

[0161] The transfer strategy determination module 303 may determine the type of the target storage space or the protocol corresponding to the target storage space based on the configuration information of the target storage space.

[0162] After determining the type of the target storage space or the protocol corresponding to the target storage space, the transfer strategy corresponding to the target data to be transferred can be determined based on the correspondence between the type of storage space and the preset transfer strategy, or based on the correspondence between the protocol corresponding to the storage space and the preset transfer strategy.

[0163] The data transfer module 304 may transfer the target data to be transferred based on the transfer strategy, and transfer the target data to be transferred to the target storage space.

[0164] The rail transit data transfer device in the embodiment of the present invention determines the target storage space corresponding to the target data to be transferred based on the configuration information of the target data to be transferred, and then determines the transfer strategy corresponding to the target data to be transferred based on the type of the above-mentioned target storage space or the corresponding protocol, and then transfers the target data to be transferred to the above-mentioned target storage space based on the above-mentioned transfer strategy. It can realize the transfer of rail transit data more simply and efficiently when the data source and storage space type are different, and can synchronously transfer data to multiple storage spaces, which can improve the flexibility of rail transit data transfer.

[0165] Figure 4 An example of a physical structure diagram of an electronic device is shown below. Figure 4 As shown, the electronic device may include: a processor 410, a communication interface 420, a memory 430, and a communication bus 440, wherein the processor 410, the communication interface 420, and the memory 430 communicate with each other via the communication bus 440. The processor 410 may call the logic instructions in the memory 430 to execute the rail transit data transfer method, which includes: obtaining target data to be transferred; determining the target storage space corresponding to the target data to be transferred based on the configuration information of the target data to be transferred; determining the transfer strategy corresponding to the target data to be transferred based on the type of the target storage space or the protocol corresponding to the target storage space; and transferring the target data to be transferred to the target storage space based on the transfer strategy.

[0166] In addition, the logic instructions in the above-mentioned memory 430 can be implemented in the form of a software functional unit and can be stored in a computer-readable storage medium when sold or used as an independent product. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, or the part of the technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes several instructions for enabling a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk.

[0167] On the other hand, the present invention also provides a computer program product, which includes a computer program, which can be stored on a non-transitory computer-readable storage medium. When the computer program is executed by a processor, the computer can execute the rail transit data transfer method provided by the above methods, which includes: obtaining target data to be transferred; determining the target storage space corresponding to the target data to be transferred based on the configuration information of the target data to be transferred; determining the transfer strategy corresponding to the target data to be transferred based on the type of the target storage space or the protocol corresponding to the target storage space; and transferring the target data to be transferred to the target storage space based on the transfer strategy.

[0168] On the other hand, the present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon. When the computer program is executed by a processor, it is implemented to execute the rail transit data transfer method provided by the above-mentioned methods, the method including: obtaining target data to be transferred; determining the target storage space corresponding to the target data to be transferred based on the configuration information of the target data to be transferred; determining the transfer strategy corresponding to the target data to be transferred based on the type of the target storage space or the protocol corresponding to the target storage space; and transferring the target data to be transferred to the target storage space based on the transfer strategy.

[0169] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, i.e., they may be located in one location or distributed across multiple network units. Some or all of the modules may be selected based on actual needs to achieve the objectives of the present embodiment. Persons of ordinary skill in the art will be able to understand and implement the present invention without inventive effort.

[0170] Through the description of the above embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus a necessary general hardware platform, or of course, by hardware. Based on this understanding, the essence of the above technical solution or the part that contributes to the existing technology can be embodied in the form of a software product. The computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, a magnetic disk, an optical disk, etc., and includes a number of instructions for enabling a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment or certain parts of the embodiments.

[0171] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the various embodiments of the present invention.

Claims

1. A rail transit data transfer method, characterized in that: include: Acquire target data to be transferred, where the target data to be transferred is rail transit data; Determining a target storage space corresponding to the target data to be transferred based on the configuration information of the target data to be transferred; Determining a transfer policy corresponding to the target data to be transferred based on the type of the target storage space or the protocol corresponding to the target storage space; Based on the transfer strategy, the target data to be transferred is transferred to the target storage space; The step of obtaining target data to be transferred includes: Based on the HTTP interface, obtain the original data to be transferred output by the data source to be transferred; Performing data verification on the original data to be transferred; When the original data to be transferred passes the data verification, performing data processing on the original data to be transferred; Determining the original data to be transferred after data processing as the target data to be transferred; Wherein, the data verification includes data format verification; the data processing includes data filtering; The determining, based on the type of the target storage space, a transfer strategy corresponding to the target data to be transferred includes: In a case where the type of the target storage space is MySQL or SQLite, determining that the transfer strategy corresponding to the target data to be transferred includes a first transfer strategy; The first transfer strategy includes transferring the target data to be transferred to the target storage space based on the Mybatis framework and the Springboot framework; The determining, based on the type of the target storage space, a transfer strategy corresponding to the target data to be transferred includes: In a case where the type of the target storage space is a message queue, determining that the transfer strategy corresponding to the target data to be transferred includes a second transfer strategy; Among them, the second transfer strategy includes obtaining a message queue executor based on the configuration information of the target storage space, parsing the target data to be transferred using JSONObject, and then transferring the parsed target data to be transferred to the target storage space based on the message queue executor.

2. The rail transit data transfer method according to claim 1, characterized in that: The determining, based on the protocol corresponding to the target storage space, a transfer strategy corresponding to the target data to be transferred includes: In a case where the protocol corresponding to the target storage space is HTTP protocol, UDP protocol or TCP protocol, determining that the transfer strategy corresponding to the target data to be transferred includes a third transfer strategy; The third transfer strategy includes transferring the target data to be transferred to the target storage space based on a protocol corresponding to the target storage space.

3. The rail transit data transfer method according to claim 1, characterized in that: The determining, based on the type of the target storage space, a transfer strategy corresponding to the target data to be transferred includes: In a case where the type of the target storage space is ES, determining that the transfer strategy corresponding to the target data to be transferred includes a fourth transfer strategy; The fourth transfer strategy includes obtaining an implementation class corresponding to the target storage space based on an operation type of the target storage space, and transferring the target data to be transferred to the target storage space based on the implementation class.

4. The rail transit data transfer method according to any one of claims 1 to 3, characterized in that: The transferring the target data to be transferred to the target storage space based on the transfer strategy includes: In a case where the target storage space passes the verification, the target data to be transferred is transferred to the target storage space based on the transfer policy.

5. A rail transit data transfer device, characterized in that: include: A transfer data acquisition module is used to acquire target data to be transferred, wherein the target data to be transferred is rail transit data; A storage space determining module, configured to determine a target storage space corresponding to the target data to be transferred based on configuration information of the target data to be transferred; A transfer strategy determination module, configured to determine a transfer strategy corresponding to the target data to be transferred based on the type of the target storage space or the protocol corresponding to the target storage space; A data transfer module, configured to transfer the target data to be transferred to the target storage space based on the transfer strategy; The transfer data acquisition module acquires target data to be transferred, including: Based on the HTTP interface, obtain the original data to be transferred output by the data source to be transferred; Performing data verification on the original data to be transferred; When the original data to be transferred passes the data verification, performing data processing on the original data to be transferred; Determining the original data to be transferred after data processing as the target data to be transferred; Wherein, the data verification includes data format verification; the data processing includes data filtering; The transfer strategy determination module determines a transfer strategy corresponding to the target data to be transferred based on the type of the target storage space, including: In a case where the type of the target storage space is MySQL or SQLite, determining that the transfer strategy corresponding to the target data to be transferred includes a first transfer strategy; The first transfer strategy includes transferring the target data to be transferred to the target storage space based on the Mybatis framework and the Springboot framework; The transfer strategy determination module determines a transfer strategy corresponding to the target data to be transferred based on the type of the target storage space, including: In a case where the type of the target storage space is a message queue, determining that the transfer strategy corresponding to the target data to be transferred includes a second transfer strategy; Among them, the second transfer strategy includes obtaining a message queue executor based on the configuration information of the target storage space, parsing the target data to be transferred using JSONObject, and then transferring the parsed target data to be transferred to the target storage space based on the message queue executor.

6. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the program, the rail transit data transfer method according to any one of claims 1 to 4 is implemented.

7. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the rail transit data transfer method according to any one of claims 1 to 4 is implemented.

Images