Data transmission method, apparatus, computer equipment and computer-readable storage medium

Abstract

This application relates to the field of information technology and discloses a data transmission method, apparatus, computer device, and computer-readable storage medium. The data transmission method includes: acquiring a data source and a distribution expression for each target data source corresponding to the data source; acquiring data to be transmitted from the data source; if data matching the distribution expression exists in the data to be transmitted, transmitting the data matching the distribution expression to the target data source; if no data matching the distribution expression exists in the data to be transmitted, constructing a data distribution plugin for the data source and target data sources, using the data distribution plugin to filter the data to be transmitted, and transmitting the filtered data to the target data source. By using the constructed distribution expression for data distribution and dynamically extending the distribution expression using the data distribution plugin, the need to redevelop the data transmission link is avoided, thus improving the efficiency of data distribution.

Description

Technical Field

[0001] This application relates to the field of information, and more specifically to data transmission methods, apparatus, computer equipment, and computer-readable storage media. Background Technology

[0002] Data distribution refers to synchronizing data from a data source to multiple target data sources to ensure data consistency between the data source and the target data sources. On-demand data distribution, on the other hand, involves distributing data from a single data source to different target data sources based on demand, with each target data source only receiving the data it needs. To ensure near real-time performance, high performance, and high throughput of the data platform, existing technologies typically employ direct distribution or direct copying methods. However, in on-demand data distribution, the data sources are diverse and massive in quantity, and the processing logic for distributing data to different target data sources also differs.

[0003] Direct data distribution cannot quickly adapt to different business scenarios. When data from one data source needs to be distributed to multiple target data sources multiple times, the data distribution efficiency is low. Furthermore, in cases of heterogeneous data sources, it is necessary to redevelop the transmission links between the data source and target data sources, further impacting data distribution efficiency. Direct copying requires manual data distribution by operators, which lacks flexibility and is prone to distributing incorrect data to the target data source. Additionally, direct copying consumes significant manpower and resources, resulting in low data distribution efficiency. Summary of the Invention

[0004] The purpose of this invention is to provide a device for solving the problem of low data distribution efficiency.

[0005] To achieve the above objectives, in a first aspect, this application provides a data transmission method, comprising:

[0006] Obtain the data source, and the distribution expression for each target data source corresponding to the data source;

[0007] Obtain the data to be sent from the data source;

[0008] If any data in the data to be sent matches the distribution expression, the data in the data to be sent that matches the distribution expression will be sent to the target data source.

[0009] If no data matching the distribution expression exists in the data to be sent, a data distribution plugin is constructed for the data source and the target data source. The data distribution plugin is used to filter the data to be sent, and the filtered data to be sent is sent to the target data source.

[0010] In conjunction with the first aspect, in a first possible implementation, the method further includes:

[0011] Obtain the mapping rules between the data source and the target data source;

[0012] If the data to be sent conforms to the mapping rule, the data to be sent is mapped based on the mapping rule;

[0013] If the data to be sent does not conform to the mapping rules, a mapping plugin between the data source and the target data source is constructed, and the mapping function of the mapping plugin is used to map the data to be sent.

[0014] In conjunction with the first possible implementation of the first aspect, in the second possible implementation, the step of mapping the data to be sent based on the mapping rule when the data to be sent conforms to the mapping rule includes:

[0015] If the data to be sent conforms to the mapping rule, the data of the first field in the data to be sent is mapped to the data of the second field based on the mapping rule.

[0016] In conjunction with the first aspect, in a third possible implementation, the method further includes:

[0017] Obtain the conversion rules between the data source and the target data source;

[0018] If the data to be sent conforms to the conversion rules, the data to be sent is converted based on the conversion rules;

[0019] If the data to be sent does not conform to the conversion rules, a conversion plugin between the data source and the target data source is constructed, and the conversion function of the conversion plugin is used to convert the data to be sent.

[0020] In conjunction with the third possible implementation of the first aspect, in the fourth possible implementation, the step of performing data conversion on the data to be sent based on the conversion rule when the data to be sent conforms to the conversion rule includes:

[0021] If the data to be sent conforms to the conversion rule, the data with the first value in the data to be sent is converted into data with the second value based on the conversion rule.

[0022] In conjunction with the third possible implementation of the first aspect, in the fifth possible implementation, the step of performing data conversion on the data to be sent based on the conversion rule when the data to be sent conforms to the conversion rule includes:

[0023] If the data to be sent conforms to the conversion rules, the data in the first format of the data to be sent is converted into data in the second format based on the conversion rules.

[0024] In conjunction with the first aspect, in the sixth possible implementation, obtaining the data to be sent from the data source includes:

[0025] Based on the initialization functions, connection functions, data retrieval functions, and data consumption functions of the data source and the target data source, a data retrieval plugin for the data source and the target data source is constructed.

[0026] Use the data retrieval plugin to obtain the data to be sent from the data source.

[0027] Secondly, this application provides a data transmission apparatus, comprising:

[0028] The expression acquisition module is used to acquire the data source and the distribution expression for each target data source corresponding to the data source;

[0029] The data acquisition module is used to acquire the data to be sent from the data source;

[0030] An expression data sending module is used to send the data in the data to be sent that conforms to the distribution expression to the target data source when there is data in the data to be sent that conforms to the distribution expression.

[0031] The plug-in data sending module is used to construct a data distribution plug-in between the data source and the target data source when there is no data in the data to be sent that matches the distribution expression. The data distribution plug-in is used to filter the data to be sent and then send the filtered data to the target data source.

[0032] Thirdly, this application provides a computer device, the computer device including a memory and a processor, the memory storing a computer program, the computer program implementing the data transmission method as described in the first aspect when executed by the processor.

[0033] Fourthly, this application provides a computer-readable storage medium storing a computer program, which, when executed by a processor, implements the data transmission method as described in the first aspect.

[0034] This application provides a data transmission method, comprising: acquiring a data source and a distribution expression for each target data source corresponding to the data source; acquiring data to be transmitted from the data source; if data matching the distribution expression exists in the data to be transmitted, transmitting the data matching the distribution expression to the target data source; if no data matching the distribution expression exists in the data to be transmitted, constructing a data distribution plugin for the data source and the target data source, filtering the data to be transmitted using the data distribution plugin, and transmitting the filtered data to the target data source. By using the constructed distribution expression for data distribution and dynamically extending the distribution expression using the data distribution plugin, the need to redevelop the data transmission link is avoided, thus improving the efficiency of data distribution.

[0035] Other features and advantages of the present invention will be described in detail in the following detailed description section. Attached Figure Description

[0036] The accompanying drawings are provided to further illustrate the invention and form part of the specification. They are used together with the following detailed description to explain the invention, but do not constitute a limitation thereof. In the drawings:

[0037] Figure 1 A first flowchart of the data transmission method provided in an embodiment of this application is shown;

[0038] Figure 2 A second flowchart of the data transmission method provided in an embodiment of this application is shown;

[0039] Figure 3 A third flowchart of the data transmission method provided in this application embodiment is shown;

[0040] Figure 4 A schematic diagram of the structure of the data transmission device provided in an embodiment of this application is shown. Detailed Implementation

[0041] The specific embodiments of the present invention will now be described in detail with reference to the accompanying drawings. It should be understood that the specific embodiments described herein are for illustrative and explanatory purposes only and are not intended to limit the scope of the present invention.

[0042] The components of the embodiments of the invention described and illustrated herein can typically be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of the invention provided in the accompanying drawings is not intended to limit the scope of the claimed invention, but merely to illustrate selected embodiments of the invention. All other embodiments obtained by those skilled in the art based on the embodiments of the invention without inventive effort are within the scope of protection of the invention.

[0043] In the following, the terms “comprising,” “having,” and their cognates, which may be used in various embodiments of the invention, are intended only to indicate a particular feature, number, step, operation, element, component, or combination thereof, and should not be construed as excluding, firstly, the presence of one or more other features, numbers, steps, operations, elements, components, or combinations thereof, or adding the possibility of one or more features, numbers, steps, operations, elements, components, or combinations thereof.

[0044] Furthermore, the terms "first," "second," and "third" are used only to distinguish descriptions and should not be interpreted as indicating or implying relative importance.

[0045] Unless otherwise specified, all terms used herein (including technical and scientific terms) shall have the same meaning as commonly understood by one of ordinary skill in the art to which the various embodiments of the invention pertain. Terms (such as those defined in commonly used dictionaries) shall be interpreted as having the same meaning as in their contextual meaning in the relevant technical field and shall not be interpreted as having an idealized or overly formal meaning, unless clearly defined in the various embodiments of the invention.

[0046] Example 1

[0047] Please see Figure 1 , Figure 1 A first flowchart of the data transmission method provided in an embodiment of this application is shown. Figure 1 The data transmission methods in the document include:

[0048] S110, Obtain the data source and the distribution expression for each target data source corresponding to the data source.

[0049] The data transmission method is applied to a computer device. For ease of understanding, in the embodiments of this application, the computer device is an application server PC (Personal Computer). The application server PC is used to deploy a data distribution platform. Simultaneously, this embodiment also deploys a relational database, a non-relational database, and a message broker. The relational database is used for data persistence, the non-relational database is used for the persistence of rules and expressions, and the message broker is used to provide an asynchronous data distribution mechanism.

[0050] When data needs to be distributed from a data source to a target data source, data distribution is configured. This configuration includes specifying the data source from which data is to be retrieved, the target data source from which data is to be consumed, mapping rules, transformation rules, and distribution expressions. The data source and the distribution expression for each target data source are then obtained to filter the data and achieve on-demand data distribution.

[0051] S120, Obtain the data to be sent from the data source.

[0052] A data source refers to the database or database server used by a database application. The data source stores all information related to establishing database connections. On-demand data distribution refers to distributing data from one data source to different target data sources based on demand. Each target data source only retrieves the data it needs from the data source it requires. The process involves retrieving all data from the data source that needs to be distributed to each target data source, resulting in the data source's pending distribution. It's important to understand that during the data distribution configuration process, one data source and multiple target data sources are configured. The number of target data sources is set according to actual needs and is not limited here.

[0053] As an example, obtaining the data to be sent from the data source includes:

[0054] Based on the initialization functions, connection functions, data retrieval functions, and data consumption functions of the data source and the target data source, a data retrieval plugin for the data source and the target data source is constructed.

[0055] Use the data retrieval plugin to obtain the data to be sent from the data source.

[0056] For ease of understanding, in the embodiments of this application, a basic API (Application Program Interface) is used to construct the data source. The basic API includes an initialization function, a connection function, a data retrieval function, and a data consumption function. Specifically, the initialization function is the `init` function, the connection function is the `connect` function, the data retrieval function is the `pull` function, and the data consumption function is the `push` function.

[0057] The initialization function provides startup parameters and initializes the data source and / or target data source. The connection function establishes a connection between the data source and the target data source. The data fetch function retrieves data from the data source, and the data consumption function consumes data sent by the target data source. The built data fetch plugin retrieves data from the data source to obtain the data to be sent required by the target data source.

[0058] S130, if there is data in the data to be sent that matches the distribution expression, the data in the data to be sent that matches the distribution expression is sent to the target data source.

[0059] If data matching the distribution expression exists within the data to be sent, the data is filtered based on the distribution expression to obtain the data that matches the expression, and then sent to the target data source. For example, if the expression is A>100&&X=='shanghai, the data to be sent is filtered to obtain data where A>100&&X=='shanghai, and this data is then sent to the target data source. By using the distribution expression to filter the data to be sent from the data source, it ensures that each target data source receives only the required data, thus achieving on-demand data distribution.

[0060] S140, if there is no data in the data to be sent that matches the distribution expression, construct a data distribution plugin for the data source and the target data source, use the data distribution plugin to filter the data to be sent, and send the filtered data to the target data source.

[0061] When it's impossible to filter the data needed by the target data source based on the distribution expression, directly adding a distribution expression usually requires redeveloping the transmission link between the data source and the target data source, leading to data distribution interruptions and inefficient data distribution. A solution is to use an API to build a data distribution plugin for both the data source and the target data source, and then use this plugin to dynamically extend the distribution expression.

[0062] Specifically, the data distribution plugin's API includes a distribution function, which in this embodiment is a check function. Without requiring the redevelopment of the transmission link, the distribution function of the data distribution plugin filters data that meets the distribution conditions and sends the filtered data to the target data source, achieving on-demand data distribution. Data distribution is performed using a constructed distribution expression, and the data distribution plugin dynamically extends the distribution expression, avoiding the need to redevelop the data transmission link and improving the efficiency of data distribution. Simultaneously, data mapping and transformation rules are constructed, and the plugin extends these rules, improving the ease of use and adaptability to different scenarios.

[0063] Please see Figure 2 , Figure 2 A second flowchart of the data transmission method provided in an embodiment of this application is shown.

[0064] As an example, the method also includes:

[0065] S121, Obtain the mapping rules between the data source and the target data source.

[0066] During data distribution, the data to be sent from the data source needs to be mapped and transformed to obtain the fields, values, and formats required by the target data source. Constructing the mapping rules between the data source and the target data source requires understanding that these rules are configured based on the field types required by the target data source; this will not be elaborated upon here.

[0067] S122, if the data to be sent conforms to the mapping rule, perform data mapping on the data to be sent based on the mapping rule.

[0068] If the data to be sent conforms to the mapping rules, the data to be mapped in the data to be sent is obtained based on the mapping rules, and data mapping is performed on the data to be mapped based on the mapping rules. For ease of understanding, in the embodiments of this application, the data to be mapped in the data to be sent is the data of field A. Based on the mapping rules, the data of field A is converted into data of other fields.

[0069] S123, if the data to be sent does not conform to the mapping rules, construct a mapping plugin between the data source and the target data source, and use the mapping function of the mapping plugin to perform data mapping on the data to be sent.

[0070] If the data to be sent does not conform to the mapping rules, directly modifying the existing mapping rules usually requires redeveloping the transmission link between the data source and the target data source, leading to data distribution interruption and low data distribution efficiency. This paper addresses this issue by using an API to build a mapping plugin between the data source and the target data source, which dynamically extends the mapping rules. Specifically, the API of the mapping plugin includes mapping functions; in this embodiment, the mapping function is a mapping function. Without needing to redevelop the transmission link, the mapping function of the mapping plugin maps the data to be sent, mapping data in field A to data in other fields.

[0071] In an optional example, the step of mapping the data to be sent based on the mapping rule when the data to be sent conforms to the mapping rule includes:

[0072] If the data to be sent conforms to the mapping rule, the data of the first field in the data to be sent is mapped to the data of the second field based on the mapping rule.

[0073] The mapping rule is a simple field mapping. If the data to be sent conforms to the mapping rule, the data of the first field in the data to be sent is obtained, and based on the mapping rule, the data of the first field is mapped to the data of the second field. The first and second fields are set according to actual needs and are not limited here. For example, mapping the data of the first field to the data of the second field could be converting the data of field A to the data of field X, where field A is the first field and field X is the second field. Mapping the data of the first field to the data of the second field could also be converting the data of field B to the data of field Y, where field B is the first field and field Y is the second field.

[0074] Please see Figure 3 , Figure 3 A third flowchart of the data transmission method provided in the embodiments of this application is shown.

[0075] As an example, the method also includes:

[0076] S124, Obtain the conversion rules between the data source and the target data source.

[0077] During data distribution, the data to be sent from the data source needs to be mapped and transformed to obtain the fields, values, and formats required by the target data source. Constructing the transformation rules between the data source and the target data source requires understanding that these rules are configured based on the format and value types required by the target data source; this will not be elaborated upon here.

[0078] S125, if the data to be sent conforms to the conversion rule, perform data conversion on the data to be sent based on the conversion rule.

[0079] If the data to be sent conforms to the conversion rules, the data to be converted is obtained from the data to be sent based on the conversion rules, and the data to be converted is then converted based on the conversion rules. Specifically, the data conversion includes numerical conversion and format conversion, which will not be elaborated here.

[0080] S126, if the data to be sent does not conform to the conversion rules, construct a conversion plugin between the data source and the target data source, and use the conversion function of the conversion plugin to perform data conversion on the data to be sent.

[0081] If the data to be sent does not conform to the conversion rules, directly modifying the existing conversion rules usually requires redeveloping the transmission link between the data source and the target data source, leading to data distribution interruption and low data distribution efficiency. This paper addresses this issue by using an API to build conversion plugins for the data source and target data source, and then dynamically extending the conversion rules using these plugins. Specifically, the API of the conversion plugin includes conversion functions; in this embodiment, the conversion function is the `transform` function. Without needing to redevelop the transmission link, the conversion functions of the conversion plugin perform numerical and / or format conversions on the data to be sent.

[0082] In an optional example, the step of performing data transformation on the data to be sent based on the transformation rules when the data to be sent conforms to the transformation rules includes:

[0083] If the data to be sent conforms to the conversion rule, the data with the first value in the data to be sent is converted into data with the second value based on the conversion rule.

[0084] If the data to be sent conforms to the conversion rules, the first value in the data to be sent is obtained, and based on the conversion rules, the first value is converted into a second value. The first and second values ​​are set according to actual needs and are not limited here. For example, converting the first value to the second value could be converting a value of X to a value of 1, where X is the first value and 1 is the second value. Converting the first value to the second value could also be converting a value of Y to a value of 2, where Y is the first value and 2 is the second value.

[0085] In an optional example, the step of performing data transformation on the data to be sent based on the transformation rules when the data to be sent conforms to the transformation rules includes:

[0086] If the data to be sent conforms to the conversion rules, the data in the first format of the data to be sent is converted into data in the second format based on the conversion rules.

[0087] If the data to be sent conforms to the conversion rules, the data in the first format is retrieved, and based on the conversion rules, the data in the first format is converted to the data in the second format. The first and second formats are set according to actual needs and are not limited here. For example, converting data in the first format to data in the second format could mean converting data in the format 2022-12-29 to data in the format 20221229 or 12292022, where 2022-12-29 is the data in the first format, and 20221229 or 12292022 is the data in the second format. It is important to understand that during the data distribution process, the data from the data source is mapped and converted before the data from the data source is distributed.

[0088] This application provides a data transmission method, comprising: acquiring a data source and a distribution expression for each target data source corresponding to the data source; acquiring data to be transmitted from the data source; if data matching the distribution expression exists in the data to be transmitted, transmitting the data matching the distribution expression to the target data source; if no data matching the distribution expression exists in the data to be transmitted, constructing a data distribution plugin for the data source and the target data source, filtering the data to be transmitted using the data distribution plugin, and transmitting the filtered data to the target data source. By using the constructed distribution expression for data distribution and dynamically extending the distribution expression using the data distribution plugin, the need to redevelop the data transmission link is avoided, thus improving the efficiency of data distribution.

[0089] Example 2

[0090] Please see Figure 4 , Figure 4 A schematic diagram of the structure of the data transmission device provided in an embodiment of this application is shown. Figure 4 The data transmission device 200 in the middle includes:

[0091] The expression acquisition module 210 is used to acquire the data source and the distribution expression of each target data source corresponding to the data source;

[0092] The data acquisition module 220 is used to acquire the data to be sent from the data source;

[0093] The expression data sending module 230 is used to send the data in the data to be sent that conforms to the distribution expression to the target data source when there is data in the data to be sent that conforms to the distribution expression.

[0094] The plug-in data sending module 240 is used to construct a data distribution plug-in between the data source and the target data source when there is no data matching the distribution expression in the data to be sent, to filter the data to be sent using the data distribution plug-in, and to send the filtered data to the target data source.

[0095] As an example, the data transmission device 200 further includes:

[0096] The mapping rule acquisition module is used to acquire the mapping rules between the data source and the target data source;

[0097] The rule mapping data module is used to map the data to be sent based on the mapping rule when the data to be sent conforms to the mapping rule.

[0098] The plugin mapping data module is used to construct a mapping plugin between the data source and the target data source when the data to be sent does not conform to the mapping rules, and to use the mapping function of the mapping plugin to perform data mapping on the data to be sent.

[0099] In an optional example, the rule mapping data module is further configured to, based on the mapping rule, map the data of the first field in the data to be sent to the data of the second field when the data to be sent conforms to the mapping rule.

[0100] As an example, the data transmission device 200 further includes:

[0101] The conversion rule acquisition module is used to acquire the conversion rules between the data source and the target data source;

[0102] The rule-based data conversion module is used to convert the data to be sent based on the conversion rules when the data to be sent conforms to the conversion rules.

[0103] The plugin conversion data module is used to construct a conversion plugin between the data source and the target data source when the data to be sent does not conform to the conversion rules, and to use the conversion function of the conversion plugin to convert the data to be sent.

[0104] In an optional example, the rule-based data conversion module is further configured to convert a first numerical value in the data to be sent into a second numerical value based on the conversion rule, provided that the data to be sent conforms to the conversion rule.

[0105] In an optional example, the rule-based data conversion module is further configured to convert data in a first format in the data to be sent into data in a second format based on the conversion rules, provided that the data to be sent conforms to the conversion rules.

[0106] As an example, data acquisition module 220 includes:

[0107] The data retrieval plugin construction submodule is used to construct the data retrieval plugin for the data source and the target data source based on the initialization function, connection function, data retrieval function and data consumption function of the data source and the target data source;

[0108] The plugin data acquisition submodule is used to retrieve the data to be sent from the data source using the data retrieval plugin.

[0109] The data transmission device 200 is used to execute the corresponding steps in the data transmission method described above. The specific implementation of each function will not be described in detail here. In addition, the optional examples in Embodiment 1 are also applicable to the data transmission device 200 in Embodiment 2.

[0110] This application also provides a computer device, which includes a memory and a processor. The memory stores a computer program, and when the processor executes the computer program, it implements the data transmission method as described in Embodiment 1.

[0111] In this embodiment, the expression acquisition module 210, data acquisition module 220, expression data sending module 230, and plug-in data sending module 240 are all stored as program units in the memory, and the processor executes the above-mentioned program units stored in the memory to realize the corresponding functions.

[0112] A processor contains a kernel, which retrieves the corresponding program units from memory. One or more kernels can be configured, and the problem of inefficient data distribution can be addressed by adjusting kernel parameters.

[0113] The memory may include non-permanent memory in computer-readable media, such as random access memory (RAM) and / or non-volatile memory, such as read-only memory (ROM) or flash RAM, and the memory includes at least one memory chip.

[0114] This application also provides a computer-readable storage medium storing a computer program, which, when executed by a processor, implements the data transmission method as described in Embodiment 1.

[0115] Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application can take the form of a computer program product embodied on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0116] This application is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this application. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart... Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0117] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0118] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0119] In a typical configuration, a computing device includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.

[0120] Memory may include non-persistent memory in computer-readable media, such as random access memory (RAM) and / or non-volatile memory, such as read-only memory (ROM) or flash RAM. Memory is an example of computer-readable media.

[0121] Computer-readable storage media include both permanent and non-permanent, removable and non-removable media that can store information by any method or technology. Information can be computer-readable instructions, data structures, program modules, or other data. Examples of computer storage media include, but are not limited to, phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, CD-ROM, digital versatile optical disc (DVD) or other optical storage, magnetic tape, magnetic magnetic disk storage or other magnetic storage devices, or any other non-transferable medium that can be used to store information accessible by a computing device. As defined herein, computer-readable media does not include transient computer-readable media, such as modulated data signals and carrier waves.

[0122] It should also be noted that the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such process, method, article, or apparatus. Unless otherwise specified, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes that element.

[0123] The above are merely embodiments of this application and are not intended to limit the scope of this application. Various modifications and variations can be made to this application by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the scope of the claims of this application.

Claims

1. A data transmission method, characterized in that, include: Obtain the data source, and the distribution expression for each target data source corresponding to the data source; Obtain the data to be sent from the data source; If any data in the data to be sent matches the distribution expression, the data in the data to be sent that matches the distribution expression will be sent to the target data source. If no data matching the distribution expression exists in the data to be sent, a data distribution plugin is constructed for the data source and the target data source. The data distribution plugin is used to filter the data to be sent, and the filtered data to be sent is sent to the target data source.

2. The data transmission method according to claim 1, characterized in that, The method further includes: Obtain the mapping rules between the data source and the target data source; If the data to be sent conforms to the mapping rule, the data to be sent is mapped based on the mapping rule; If the data to be sent does not conform to the mapping rules, a mapping plugin between the data source and the target data source is constructed, and the mapping function of the mapping plugin is used to map the data to be sent.

3. The data transmission method according to claim 2, characterized in that, The step of mapping the data to be sent based on the mapping rule when the data to be sent conforms to the mapping rule includes: If the data to be sent conforms to the mapping rule, the data of the first field in the data to be sent is mapped to the data of the second field based on the mapping rule.

4. The data transmission method according to claim 1, characterized in that, The method further includes: Obtain the conversion rules between the data source and the target data source; If the data to be sent conforms to the conversion rules, the data to be sent is converted based on the conversion rules; If the data to be sent does not conform to the conversion rules, a conversion plugin between the data source and the target data source is constructed, and the conversion function of the conversion plugin is used to convert the data to be sent.

5. The data transmission method according to claim 4, characterized in that, The step of performing data transformation on the data to be sent based on the transformation rules when the data to be sent conforms to the transformation rules includes: If the data to be sent conforms to the conversion rule, the data with the first value in the data to be sent is converted into data with the second value based on the conversion rule.

6. The data transmission method according to claim 4, characterized in that, The step of performing data transformation on the data to be sent based on the transformation rules when the data to be sent conforms to the transformation rules includes: If the data to be sent conforms to the conversion rules, the data in the first format of the data to be sent is converted into data in the second format based on the conversion rules.

7. The data transmission method according to claim 1, characterized in that, The process of obtaining the data to be sent from the data source includes: Based on the initialization functions, connection functions, data retrieval functions, and data consumption functions of the data source and the target data source, a data retrieval plugin for the data source and the target data source is constructed. Use the data retrieval plugin to obtain the data to be sent from the data source.

8. A data transmission device, characterized in that, include: The expression acquisition module is used to acquire the data source and the distribution expression for each target data source corresponding to the data source; The data acquisition module is used to acquire the data to be sent from the data source; An expression data sending module is used to send the data in the data to be sent that conforms to the distribution expression to the target data source when there is data in the data to be sent that conforms to the distribution expression. The plug-in data sending module is used to construct a data distribution plug-in between the data source and the target data source when there is no data in the data to be sent that matches the distribution expression. The data distribution plug-in is used to filter the data to be sent and then send the filtered data to the target data source.

9. A computer device, characterized in that, The computer device includes a memory and a processor. The memory stores a computer program, which, when executed by the processor, implements the data transmission method as described in any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program, which, when executed by a processor, implements the data transmission method as described in any one of claims 1 to 7.

Images