Data transmission method and device, storage medium and electronic equipment

Abstract

The invention provides a data transmission method and device, a storage medium and electronic equipment, and the method comprises the steps: obtaining target data from an upstream database, and generating at least one to-be-transmitted message based on the target data; determining transmission indication information of each to-be-transmitted message in the at least one to-be-transmitted message, and determining M priority messages from the at least one to-be-transmitted message based on the transmission indication information of each to-be-transmitted message; and sending the M priority messages to an output component according to a sequence of the transmission priorities from high to low, and writing the received priority messages into a target downstream database through the output component. The embodiment of the invention can effectively improve the data transmission efficiency.

Description

Technical Field

[0001] The present invention relates to the field of computer technology, and in particular to a data transmission method, device, storage medium and electronic equipment. Background Art

[0002] At present, Data Transmission Service (DTS) is a general database data transmission solution with functions such as data migration and data synchronization, and has good transmission efficiency and synchronization reliability. However, related technologies usually send messages to downstream databases in the order of messages in the queue, and retransmit timed data according to a certain retransmission strategy, which may cause serious delays in critical data due to insufficient bandwidth resources, resulting in low transmission efficiency. Based on this, there is currently no good solution to improve data transmission efficiency. Summary of the invention

[0003] In view of this, an embodiment of the present invention provides a data transmission method, device, storage medium and electronic device to solve the problem of low data transmission efficiency caused by related technologies; that is, an embodiment of the present invention can send the determined priority messages in order of transmission priority from high to low, thereby avoiding low-priority data affecting the transmission efficiency of high-priority data, which can effectively improve the data transmission efficiency, and further effectively improve the transmission efficiency of critical data (i.e., priority messages with higher transmission priority).

[0004] According to one aspect of an embodiment of the present invention, a data transmission method is provided, the method comprising:

[0005] Acquire target data from an upstream database, and generate at least one message to be transmitted based on the target data;

[0006] Determine transmission indication information of each message to be transmitted in the at least one message to be transmitted, and determine M priority messages from the at least one message to be transmitted based on the transmission indication information of each message to be transmitted, where M is a non-negative integer;

[0007] The M priority messages are sent to the output component in order of transmission priority from high to low, and the received priority messages are written into the target downstream database through the output component.

[0008] According to another aspect of an embodiment of the present invention, a data transmission device is provided, the device comprising:

[0009] An acquisition unit, used for acquiring target data from an upstream database;

[0010] a processing unit, configured to generate at least one message to be transmitted based on the target data;

[0011] The processing unit is further configured to determine transmission indication information of each message to be transmitted in the at least one message to be transmitted, and determine M priority messages from the at least one message to be transmitted based on the transmission indication information of each message to be transmitted, where M is a non-negative integer;

[0012] The processing unit is also used to send the M priority messages to the output component in order of transmission priority from high to low, and write the received priority messages into the target downstream database through the output component.

[0013] According to another aspect of an embodiment of the present invention, a data transmission system is provided, the data transmission system comprising at least one upstream database, at least one downstream database and a DTS service device; wherein the DTS service device can be used to execute the above-mentioned method.

[0014] According to another aspect of an embodiment of the present invention, an electronic device (i.e., a DTS service device) is provided, which includes a processor and a memory for storing a program; wherein the program includes instructions, which, when executed by the processor, enable the processor to execute the above-mentioned method.

[0015] According to another aspect of an embodiment of the present invention, a non-transitory computer-readable storage medium storing computer instructions is provided, wherein the computer instructions are used to enable a computer to execute the above-mentioned method.

[0016] The embodiment of the present invention can obtain target data from an upstream database and generate at least one message to be transmitted based on the target data. Then, the transmission indication information of each message to be transmitted in the at least one message to be transmitted can be determined, and based on the transmission indication information of each message to be transmitted, M priority messages can be determined from the at least one message to be transmitted, where M is a non-negative integer. Further, the M priority messages can be sent to the output component in order of transmission priority from high to low, and the received priority messages can be written into the target downstream database through the output component. It can be seen that the embodiment of the present invention can send the determined priority messages in order of transmission priority from high to low, thereby avoiding low priority data affecting the transmission efficiency of high priority data, effectively improving data transmission efficiency, and further effectively improving the transmission efficiency of critical data (i.e., priority messages with higher transmission priority). BRIEF DESCRIPTION OF THE DRAWINGS

[0017] Further details, features and advantages of the invention are disclosed in the following description of exemplary embodiments in conjunction with the accompanying drawings, in which:

[0018] Figure 1 A schematic flow chart of a data transmission method according to an exemplary embodiment of the present invention is shown;

[0019] Figure 2 A schematic diagram of a data transmission system according to an exemplary embodiment of the present invention is shown;

[0020] Figure 3 A schematic flow chart of another data transmission method according to an exemplary embodiment of the present invention is shown;

[0021] Figure 4 A schematic diagram showing a data transmission device according to an exemplary embodiment of the present invention is shown;

[0022] Figure 5 A block diagram of an exemplary electronic device that can be used to implement an embodiment of the present invention is shown. DETAILED DESCRIPTION

[0023] Embodiments of the present invention will be described in more detail below with reference to the accompanying drawings. Although certain embodiments of the present invention are shown in the accompanying drawings, it should be understood that the present invention can be implemented in various forms and should not be construed as being limited to the embodiments described herein, which are instead provided for a more thorough and complete understanding of the present invention. It should be understood that the drawings and embodiments of the present invention are only for exemplary purposes and are not intended to limit the scope of protection of the present invention.

[0024] It should be understood that the various steps described in the method embodiments of the present invention may be performed in different orders and / or in parallel. In addition, the method embodiments may include additional steps and / or omit the steps shown. The scope of the present invention is not limited in this respect.

[0025] The term "including" and its variations used in this document are open inclusions, that is, "including but not limited to". The term "based on" means "based at least in part on". The term "one embodiment" means "at least one embodiment"; the term "another embodiment" means "at least one other embodiment"; the term "some embodiments" means "at least some embodiments". Relevant definitions of other terms will be given in the following description. It should be noted that the concepts of "first", "second", etc. mentioned in the present invention are only used to distinguish different devices, modules or units, and are not used to limit the order or interdependence of the functions performed by these devices, modules or units.

[0026] It should be noted that the modifications of "one" and "plurality" mentioned in the present invention are illustrative rather than restrictive, and those skilled in the art should understand that, unless otherwise clearly indicated in the context, it should be understood as "one or more".

[0027] The names of the messages or information exchanged between multiple devices in the embodiments of the present invention are only used for illustrative purposes, and are not used to limit the scope of these messages or information.

[0028] It should be noted that the embodiments of the present invention may involve a data transmission system, which may include but is not limited to: at least one upstream database (sender of data, responsible for providing data to be transmitted), at least one downstream database (such as a target downstream database, receiver of data, responsible for receiving and processing such data) and a DTS service device (also referred to as a dts-server node), etc., which is not limited in the embodiments of the present invention. Optionally, the number of DTS service devices may be one or more, which is not limited in the embodiments of the present invention; when the number of DTS service devices is multiple, distributed data transmission can be achieved. Optionally, a DTS service device may be an electronic device, that is, the DTS service device in the data transmission system may include one or more electronic devices; based on this, the execution subject of the data transmission method proposed in the embodiments of the present invention may be one or more electronic devices, that is, one or more electronic devices constituting the DTS service device, and so on.

[0029] Optionally, the electronic device may be a terminal (i.e., a client) or a server. The terminals mentioned here may include, but are not limited to, smart phones, tablet computers, laptop computers, desktop computers, smart watches, intelligent voice interaction devices, smart home appliances, vehicle-mounted terminals, and the like. The server mentioned here may be an independent physical server, or a server cluster or distributed system consisting of multiple physical servers, or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, CDN (Content Delivery Network), and big data and artificial intelligence platforms, and the like.

[0030] Based on the above description, an embodiment of the present invention proposes a data transmission method, which can be applied to a DTS service device in a data transmission system, that is, the data transmission method can be executed by the electronic device mentioned above (that is, it can be executed by the electronic device constituting the DTS service device), that is, it can be executed by the DTS service device; for the convenience of explanation, the following description will be given by taking the electronic device executing the data transmission method as an example; Figure 1 As shown, the data transmission method may include the following steps S101-S103:

[0031] S101, obtaining target data from an upstream database, and generating at least one message to be transmitted based on the target data.

[0032] Optionally, the upstream database may be any database, which is not limited in the embodiment of the present invention.

[0033] Optionally, the data transmission system may further include a data transmission service interaction device (also referred to as a dts-server-api component, which is a response module for accepting user requests); in the embodiment of the present invention, a user may initiate a data transmission request, and the data transmission service interaction device may respond to the data transmission request initiated by the user, and save and / or convert (such as modify) the DTS task configuration requested by the user, and then save the DTS task configuration in a database (here, the DTS metadata database). Optionally, the data transmission request may carry a DTS task configuration, and a DTS task configuration may include but is not limited to: an upstream database address (used to indicate the location of an upstream database for obtaining data), a downstream database address (used to indicate the location of a downstream database for writing data (such as the target downstream database described below)), a user name, a user password, and data indication information, etc., which are not limited in the embodiment of the present invention. Optionally, the data indication information may include but is not limited to at least one of the following: a data name, a database name, and a data table name, etc., which are not limited in the embodiment of the present invention.

[0034] Further, the data transmission service interaction device can send DTS task indication data to the data transmission service device (i.e., DTS service device, also referred to as dts-server) through DTS task configuration; optionally, a DTS task indication data may include, but is not limited to: upstream database address, downstream database address, and data indication information, etc., which are not limited in the embodiment of the present invention. Based on this, the data transmission service device can obtain target data (i.e., data indicated by the DTS task indication data) from the upstream database (i.e., the upstream database indicated by the DTS task indication data) based on the DTS task indication data, etc. Among them, the data transmission service device can be an execution module of the DTS task, which can be used to execute and process the DTS task, and can use the Raft protocol (a consistency protocol) to ensure the high availability of the dts-server, and can be responsible for scheduling different river tasks (also referred to as dts-river, the smallest unit of DTS task execution) to different dts-server nodes. Among them, dts-river can be used to specifically execute the full / incremental synchronization process, report the execution progress to the dts-server, and update the write location and other information to the DTS metadata database. Based on this, the data transmission service device may include at least one dts-river.

[0035] Optionally, the electronic device (i.e., the data transmission service device) may include an Input component (a component for obtaining upstream data), that is, the dts-river may include an Input component, which may be a component in the dts-river for obtaining upstream data; based on this, the target data can be obtained from the upstream database through the Input component. Optionally, the target data can be any data, that is, the target data can be data required to be obtained by any DTS task, and the embodiment of the present invention does not limit this. Optionally, the Input may also be responsible for converting different types of upstream data (such as target data) into a core.Msg message type (a message type) uniformly transmitted in the DTS; and / or, it may also be responsible for persisting the synchronization progress of the DTS, that is, it may also update the location of the corresponding message in the DTS metadata database (i.e., update the storage location of the corresponding message in the downstream database) when receiving the message confirmation information sent by the Output component, and so on. Based on this, the electronic device can convert the target data into at least one message to be processed through Input, and the message type of a message can be core.Msg message type; then correspondingly, the electronic device can generate at least one message to be transmitted based on at least one message to be processed, so as to realize the above-mentioned generation of at least one message to be transmitted based on the target data.

[0036] Optionally, the electronic device may also include an Emitter component (also referred to as a transmitter module), which is a key component in DTS that transmits messages from the Input to the Dispatcher cache queue (i.e., the core.msg message cache module). Optionally, the Emitter may also be responsible for adding control data and / or message filtering operations to the message, such as executing the message filtering operations indicated by the DTS task indication data. Accordingly, the Input component may transmit at least one message to be transmitted to the Emitter component.

[0037] Based on this, the electronic device can generate at least one message to be transmitted based on at least one message to be processed through the Emitter component; exemplarily, at least one message to be processed can be used as at least one message to be transmitted, or at least one message to be processed can be filtered according to the information filtering operation indicated by the corresponding DTS configuration indication data to obtain at least one message to be transmitted, and so on. Optionally, the electronic device can also generate a message sequence number (also referred to as a sequence number) for each message to be transmitted in at least one message to be transmitted through the Emitter.

[0038] Optionally, the electronic device may also include a scheduler component (also referred to as a Scheduler component), which is a component in DTS responsible for efficiently processing messages in the cache queue, performing message confirmation, and ensuring the order of message processing; based on this, the electronic device can send at least one message to be transmitted and the sequence number of each message to be transmitted to the Scheduler component through the Emitter, so that at least one message to be transmitted and the sequence number are added to the message queue (i.e., the core.Msg message queue) through the message cache module in the Scheduler component, so that at least one message to be transmitted is sent to the output component through the consumption and writing modules in the Scheduler component. Among them, Output can be a component in DTS responsible for writing core.Msg messages to different downstream databases. It has a high degree of scalability and can flexibly write messages to a variety of different downstream systems and databases to meet the needs of different scenarios.

[0039] S102, determining transmission indication information of each message to be transmitted in at least one message to be transmitted, and based on the transmission indication information of each message to be transmitted, determining M priority messages from at least one message to be transmitted, where M is a non-negative integer.

[0040] Optionally, the electronic device may determine the transmission indication information of each message to be transmitted through the Scheduler component, and determine M priority messages from at least one message to be transmitted based on the transmission indication information of each message to be transmitted. Optionally, the transmission indication information of a message to be transmitted may be used to indicate the transmission level of the corresponding message to be transmitted.

[0041] In one embodiment, when determining the transmission indication information of each message to be transmitted among at least one message to be transmitted, for any message to be transmitted among at least one message to be transmitted, the electronic device can determine whether the pending confirmation time of any message to be transmitted is greater than a preset pending confirmation time threshold; if the pending confirmation time of any message to be transmitted is greater than the preset pending confirmation time threshold, the priority indication information corresponding to any message to be transmitted can be determined, and the priority indication information corresponding to any message to be transmitted can be used as the transmission indication information of any message to be transmitted, and a message whose transmission indication information is one priority indication information is treated as a priority message. Optionally, the confirmation time of a message to be transmitted may refer to the time during which the corresponding message to be transmitted is sent to the output component but the message confirmation information returned by the output component is not received, that is, the confirmation time of a message to be transmitted may refer to the difference between the current system time and the time when the corresponding message to be transmitted is sent to the output component, and the corresponding message to be transmitted has not been confirmed at the current system time (that is, the message confirmation information of the corresponding message to be transmitted has not been received at the current system time); optionally, when the message confirmation information of any message to be transmitted is received, the confirmation time of any message to be transmitted may be empty or 0, etc., and the embodiment of the present invention does not limit this. Optionally, the preset confirmation time threshold may be set according to experience or according to actual needs, and the embodiment of the present invention does not limit this. In this case, whether any message to be transmitted is a priority message can be judged based on the confirmation time of any message to be transmitted.

[0042] In another embodiment, the electronic device can determine whether the number of messages in the queue where any message to be transmitted is located (i.e., the message queue) exceeds the queue cache threshold and whether any message to be transmitted is located within a specified position range in the corresponding queue (i.e., the queue where any message to be transmitted is located); if the number of messages in the queue where any message to be transmitted is located exceeds the queue cache threshold and any message to be transmitted is located within the specified position range in the corresponding queue, the priority indication information corresponding to any message to be transmitted can be determined, and the priority indication information corresponding to any message to be transmitted can be used as the transmission indication information of any message to be transmitted. Optionally, the queue cache threshold can be set according to experience or actual needs, or it can be determined according to a preset cache ratio threshold (such as the product of the preset cache ratio threshold and the total amount of queue cache), etc.; this is not limited in the embodiments of the present invention. Optionally, the specified position range (which may refer to the first Q positions) may be set according to experience or actual needs (in this case, Q may be set according to experience or actual needs, and Q is a positive integer), or may be determined based on the number of messages in the queue where any message to be transmitted is located and the queue cache threshold (such as Q may be equal to the difference between the number of messages in the queue where any message to be transmitted is located and the queue cache threshold), etc., and the embodiments of the present invention do not limit this. In this case, whether any message to be transmitted is a priority message may be determined based on the number of messages in the queue where any message to be transmitted is located and the position of any message to be transmitted in the corresponding queue.

[0043] In another implementation, the electronic device may determine the data type of any message to be transmitted; if the data type of any message to be transmitted is a user data type (i.e., any message to be transmitted is user data, not table structure data, etc.), then the priority indication information corresponding to any message to be transmitted may be determined, and the priority indication information corresponding to any message to be transmitted may be used as the transmission indication information of any message to be transmitted, etc. In this case, whether any message to be transmitted is a priority message may be determined according to the data type of any message to be transmitted.

[0044] Optionally, when determining the priority indication information corresponding to any message to be transmitted, the electronic device may determine the priority indication information corresponding to any message to be transmitted according to the waiting time of any message to be transmitted; optionally, the waiting time of a message to be transmitted may refer to the time between the time when the Scheduler component receives the corresponding message to be transmitted and the current system time, that is, it may refer to the time between the time when the corresponding message to be transmitted arrives at the Scheduler component and the current system time; or, it may refer to the time between the time when the corresponding message to be transmitted is added to the message queue and the current system time, and so on; the embodiment of the present invention does not limit this. Optionally, when determining the priority indication information corresponding to any message to be transmitted according to the waiting time of any message to be transmitted, the waiting time range where the waiting time of any message to be transmitted is located may be determined from multiple waiting time ranges, and the priority indication information corresponding to the waiting time range where the waiting time of any message to be transmitted is located is used as the priority indication information corresponding to any message to be transmitted; wherein, one waiting time range may correspond to one priority indication information. Optionally, multiple ranges of time to be transmitted may be set based on experience or actual needs, which is not limited in the embodiments of the present invention. Optionally, a priority indication information may be used to indicate a transmission level with priority; optionally, the greater the time indicated by the range of time to be transmitted, the higher the transmission level indicated by the corresponding priority indication information (i.e., the higher the priority of the transmission level indicated by the corresponding priority indication information). Optionally, the priority indication information corresponding to a range of time to be transmitted may be set based on experience or actual needs, which is not limited in the embodiments of the present invention.

[0045] Optionally, when the number of messages in the queue where any message to be transmitted is located exceeds the queue cache threshold and any message to be transmitted is located within a specified position range in the corresponding queue (i.e., when judging whether any message to be transmitted is a priority message according to the number of messages in the queue where any message to be transmitted is located and the position where any message to be transmitted is located), the electronic device may also determine the priority indication information corresponding to any message to be transmitted according to the position of any message to be transmitted in the corresponding queue; specifically, the message position range where the position of any message to be transmitted is located may be determined from multiple message position ranges, and the priority indication information corresponding to the message position range where the position of any message to be transmitted is located may be used as the priority indication information corresponding to any message to be transmitted; wherein, one message position range may correspond to one priority indication information. Optionally, multiple message position ranges may be set according to experience or according to actual needs, and the embodiment of the present invention does not limit this. Optionally, the closer the position indicated by the message position range is, the higher the transmission level indicated by the corresponding priority indication information (i.e., the higher the priority of the transmission level indicated by the corresponding priority indication information). Optionally, the electronic device may also perform weighted summation on the priority indication information corresponding to the range of the waiting time for transmission of any message to be transmitted and the priority indication information corresponding to the message position range where the position of any message to be transmitted is located, to obtain the priority indication information corresponding to any message to be transmitted, and so on. Optionally, the priority indication information corresponding to a message position range may be set according to experience or according to actual needs, which is not limited in the embodiments of the present invention.

[0046] Optionally, when judging whether any message to be transmitted is a priority message according to the data type of any message to be transmitted, when the data type of any message to be transmitted is a user data type, the designated priority indication information may be used as the priority indication information corresponding to any message to be transmitted; or, the designated priority indication information may be weightedly summed with the priority indication information corresponding to the range of the waiting time for transmission of any message to be transmitted and / or the priority indication information corresponding to the message position range of the position of any message to be transmitted, to obtain the priority indication information corresponding to any message to be transmitted, and so on; the embodiments of the present invention are not limited to this. Optionally, the designated priority indication information may be set according to experience or according to actual needs, and the embodiments of the present invention are not limited to this.

[0047] Optionally, when any message to be transmitted is not a priority message, that is, does not have corresponding priority indication information, no priority indication information can be used as the transmission indication information of any message to be transmitted, and no priority indication information can be used to indicate that the corresponding message to be transmitted does not have priority; or, at this time, the transmission indication information of any message to be transmitted can be 0 or empty, etc.; the embodiment of the present invention is not limited to this. Correspondingly, the electronic device can determine M priority messages whose transmission indication information is priority indication information (that is, any priority indication information) from at least one message to be transmitted, that is, all messages to be transmitted whose transmission indication information is any priority indication information can be added to the M priority messages; illustratively, when the priority indication information is greater than 0, M priority messages whose transmission indication information is greater than 0 can be determined from at least one message to be transmitted, and so on. Optionally, the priority indication information used to indicate a transmission level with priority can be set according to experience or according to actual needs, and the embodiment of the present invention is not limited to this.

[0048] It should be understood that, after receiving the message confirmation information of any message to be transmitted, the message to be transmitted can be removed from at least one message to be transmitted, that is, the message that has been written into the downstream database can be removed from at least one message to be transmitted. Based on this, the M priority messages can all be messages that have not been written into the downstream database, that is, all are messages that have not received the corresponding message confirmation information.

[0049] S103, sending the M priority messages to the output component in descending order of transmission priority, and writing the received priority messages into the target downstream database through the output component.

[0050] Optionally, the electronic device may further include a priority confirmation component (also referred to as a PriorityConfirm component), such as Figure 2 As shown; Based on this, the electronic device can receive M priority messages through the priority confirmation component, that is, receive M priority messages (i.e. messages with priority) from the Scheduler. And, the M priority messages can be added to the message priority queue (i.e. the core.Msg message priority queue). Among them, Figure 2 The RPC (Remote Procedure Call) in it is a remote procedure call protocol, DML can be the abbreviation of Data Manipulation Language, AckMsg can represent a confirmation message, and Ack can represent confirmation.

[0051] Furthermore, the electronic device may send the M priority messages to the output component in descending order of transmission priority through the priority confirmation component.

[0052] Optionally, the target downstream database may be any database, which is not limited in the embodiment of the present invention; illustratively, the target downstream database may be any downstream database indicated by the corresponding DTS task indication data, and so on.

[0053] The embodiment of the present invention can obtain target data from an upstream database and generate at least one message to be transmitted based on the target data. Then, the transmission indication information of each message to be transmitted in the at least one message to be transmitted can be determined, and based on the transmission indication information of each message to be transmitted, M priority messages can be determined from the at least one message to be transmitted, where M is a non-negative integer. Further, the M priority messages can be sent to the output component in order of transmission priority from high to low, and the received priority messages can be written into the target downstream database through the output component. It can be seen that the embodiment of the present invention can send the determined priority messages in order of transmission priority from high to low, thereby avoiding low priority data affecting the transmission efficiency of high priority data, effectively improving data transmission efficiency, and further effectively improving the transmission efficiency of critical data (i.e., priority messages with higher transmission priority).

[0054] Based on the above description, the embodiment of the present invention further proposes a more specific data transmission method, which can be applied to the DTS service device in the data transmission system, that is, the data transmission method can be executed by the electronic device mentioned above (that is, it can be executed by the electronic device constituting the DTS service device), that is, it can be executed by the DTS service device; for the convenience of explanation, the data transmission method executed by the electronic device is taken as an example for explanation; please refer to Figure 3 , the data transmission method may include the following steps S301-S306:

[0055] S301, obtaining target data from an upstream database, and generating at least one message to be transmitted based on the target data.

[0056] S302, determining transmission indication information of each message to be transmitted in at least one message to be transmitted, and based on the transmission indication information of each message to be transmitted, determining M priority messages from at least one message to be transmitted, where M is a non-negative integer.

[0057] S303: Determine the transmission priority of each priority message based on the transmission indication information of each priority message in the M priority messages.

[0058] In an embodiment of the present invention, an electronic device may determine the sequence number of each priority message among M priority messages; and determine the transmission priority of each priority message based on the sequence number and transmission indication information of each priority message; wherein, if the transmission level indicated by the transmission indication information of the first priority message among the M priority messages is equal to the transmission level indicated by the transmission indication information of the second priority message and the sequence number of the first priority message is less than the sequence number of the second priority message, or the transmission level indicated by the transmission indication information of the first priority message is higher than the transmission level indicated by the transmission indication information of the second priority message, then the transmission priority of the first priority message is higher than the transmission priority of the second priority message. wherein, the transmission indication information of a message may be used to indicate the transmission level of the corresponding message, and the transmission priority of a message may be determined based on the transmission level and sequence number of the corresponding message; optionally, the first priority message and the second priority message may be any priority message among the M priority messages, respectively.

[0059] Based on this, for the first priority message and the second priority message, if the transmission level indicated by the transmission indication information of the first priority message is equal to the transmission level indicated by the transmission indication information of the second priority message and the sequence number of the first priority message is greater than the sequence number of the second priority message, or the transmission level indicated by the transmission indication information of the first priority message is lower than the transmission level indicated by the transmission indication information of the second priority message, then the transmission priority of the first priority message is lower than the transmission priority of the second priority message.

[0060] In an embodiment of the present invention, the message priority queue is located in a priority confirmation component (i.e., PriorityConfirm component); based on this, the electronic device can receive M priority messages through the priority confirmation component, i.e., receive messages with priority indication information (i.e., the transmission indication information is the priority indication information) from the Scheduler; and through the priority confirmation component, determine the transmission priority of each priority message based on the transmission indication information of each priority message in the M priority messages.

[0061] Optionally, in other embodiments, the electronic device may also determine the transmission priority of each priority message based on the transmission indication information of each priority message in the M priority messages through the Scheduler component, and send each priority message and the corresponding transmission priority to the priority confirmation component, etc.; the present invention is not limited to this.

[0062] S304: Store the M priority messages in a message priority queue in descending order of transmission priority.

[0063] In an embodiment of the present invention, the electronic device can store M priority messages in a message priority queue in the order of transmission priority from high to low through a priority confirmation component, that is, add M priority messages to the message priority queue in the order of transmission priority from high to low, so that the message priority queue is sorted from high to low according to the transmission priority, that is, messages with high transmission priority are prioritized in the queue. Based on this, the embodiment of the present invention can ensure that during the transmission process, messages with high transmission priority are preferentially sent to the downstream through a message priority queue. Based on this, the transmission priority of the previous priority message in the message priority queue is higher than the transmission priority of the next priority message, that is, the transmission level of the previous priority message in the message priority queue is higher than or equal to the transmission level of the next priority message, and when the transmission level of the previous priority message is equal to the transmission level of the next priority message, the sequence number of the previous priority message is less than the sequence number of the next priority message.

[0064] Optionally, when the priority confirmation component receives a new priority message, the message priority queue can also be dynamically adjusted through the priority confirmation component. That is, the new priority message can be added to the message priority queue according to the transmission priority of the new priority message, so that the transmission priority of the message before the new priority message in the message priority queue is higher than the transmission priority of the new priority message, and the transmission priority of the message after the new priority message is lower than the transmission priority of the new priority message. In other words, if a new high-priority message appears, the order of the message priority queue will be automatically adjusted to put the high-priority message in front. Based on this, the embodiment of the present invention can ensure that each message sent to the Output has the highest priority, ensuring that important data is transmitted first.

[0065] S305, sending the priority messages in the message priority queue to the output component in sequence, so as to send the M priority messages to the output component in the order of transmission priority from high to low.

[0066] In the embodiment of the present invention, PriorityConfirm may send the priority messages to the Output component one by one according to the transmission priority; that is, M priority messages are sent to the output component through the priority confirmation component.

[0067] Optionally, for any priority message among the M priority messages, after any priority message is written into the target downstream database, the message confirmation information of any priority message can be returned to the priority confirmation component through the output component; Optionally, after the output component writes the high priority message (i.e., writes any priority message into the downstream database), the output component can call the HighAck interface (a message confirmation interface) to confirm the message for any priority message, and the message confirmation information of any priority message can be sent to the priority confirmation component. Among them, the HighAck interface is a fast confirmation channel designed for high priority messages, which can immediately confirm that the message has been successfully written to the downstream, so as to ensure that the high priority message can be processed quickly through the confirmation mechanism, without waiting for the traditional confirmation process with the low priority message, thus ensuring the processing timeliness of the high priority message. Optionally, the message confirmation information of a message may include but is not limited to at least one of the following: the location information of the corresponding message (i.e., the storage location in the downstream database), the information identifier of the corresponding information (such as a sequence number, etc.) and the written indication identifier, etc., which are not limited in the embodiment of the present invention. Optionally, the written indication identifier can be set according to experience or according to actual needs, which are not limited in the embodiment of the present invention. Optionally, the output component may also send message confirmation information of any message to the input component, so that the location information of any message is added to the DTS metadata database through the input component.

[0068] Based on this, the electronic device can confirm through the priority confirmation component that any priority message has been written into the target downstream database based on the message confirmation information of any priority message, and continue to send the next priority message of any priority message to the output component in order of transmission priority from high to low. In other words, the next priority message of any priority message in the message priority queue (i.e., the first priority message in the current queue) can be sent to the output component. Based on this, PriorityConfirm can process messages according to the transmission level and the order of arrival to ensure that messages at the same transmission level are written in order to avoid disorder in message processing. This mechanism maintains the order of data at the same transmission level while ensuring priority processing of priority messages, ensuring consistency and integrity during data transmission.

[0069] Optionally, for any priority message among the M priority messages, based on the transmission indication information of any priority message, determine the retry strategy and retry time threshold corresponding to any priority message; optionally, the retry strategy and retry time threshold corresponding to any priority message may be determined through the priority confirmation component based on the transmission indication information of any priority message. Optionally, a transmission level may correspond to a retry strategy and a retry time threshold, then the retry strategy corresponding to the transmission level indicated by the transmission indication information of any priority message may be used as the retry strategy corresponding to any priority message, and the retry time threshold corresponding to the transmission level indicated by the transmission indication information of any priority message may be used as the retry time threshold corresponding to any priority; optionally, the retry strategy and retry time threshold corresponding to a transmission level may be set according to experience or according to actual needs, and the embodiments of the present invention are not limited to this.

[0070] Correspondingly, if the waiting retry time of any priority message exceeds the retry time threshold corresponding to any priority message, any priority message will be retried according to the retry strategy corresponding to any priority message until any priority message reaches the stop retry condition. Optionally, the waiting retry time of a message may refer to the difference between the current system time and the time when the corresponding message was last sent to the output component (i.e., the most recent time). Optionally, a retry strategy may include but is not limited to a retry count threshold, a retry failure record method identifier, etc., which is not limited in the embodiment of the present invention. Optionally, when the number of retries of any priority message reaches the retry count threshold indicated by the retry strategy corresponding to any priority message, it can be determined that any priority message reaches the stop retry condition; and / or, when the message confirmation information of any priority message is received, it can be determined that any priority message reaches the stop retry condition, etc.; this is not limited in the embodiment of the present invention.

[0071] Based on this, if an exception occurs when the message sent by PriorityConfirm is written downstream by Output (such as connection failure, write error, etc.), the retry mechanism will be triggered; and policies such as the retry count threshold can be set according to the transmission level, and messages with high transmission levels will have higher retry priorities (such as the retry count threshold corresponding to messages with high transmission levels can be higher, and the retry duration threshold can be shorter). Accordingly, if multiple retries fail (that is, the corresponding retry count threshold is reached), the error status can be recorded, or the backup processing flow can be turned to; that is, it can be processed according to the retry failure recording method indicated by the corresponding retry failure recording method identifier. Optionally, a retry failure recording method identifier can be used to indicate a retry failure recording method; optionally, the retry failure recording method identifier of a retry failure recording method can be set according to experience or actual needs, and the embodiments of the present invention are not limited to this.

[0072] Optionally, after PriorityConfirm receives the message confirmation information returned by the HighAck interface, it can also persist the confirmed message. This persistence operation records the processing progress and status of the message to ensure that important data is not lost in the event of a system failure or unexpected interruption. The persistent confirmation state also supports the fault recovery mechanism of DTS, that is, after recovery, the confirmed messages can be skipped based on the persistent record to avoid repeated processing. Based on this, the embodiment of the present invention is optimized for high-priority messages to ensure that the system gives priority to unconfirmed high-priority data during fault recovery.

[0073] S306, writing the received priority message into the target downstream database through the output component.

[0074] Optionally, the Output component can be responsible for writing the received core.Msg message to different downstream systems (i.e., downstream databases, file storage, etc.), and can support writing of multiple downstream types to meet different business needs.

[0075] Based on this, the PriorityConfirm component can act as a priority confirmation queue in DTS, aiming to ensure that high-priority messages are written to the downstream system first, while ensuring the processing order and reliability of the messages, and can improve the scalability of data transmission services by introducing a priority mechanism. In addition, the timeout retransmission mechanism introduced in the embodiment of the present invention can ensure that when a message is lost during transmission or is not confirmed after a timeout, retransmission is automatically initiated according to different retransmission strategies, etc., to avoid serious delays in critical data due to insufficient bandwidth resources when the network is congested, which can greatly improve the stability of message processing and the overall performance of the system, and can improve the reliability and efficiency of data transmission.

[0076] The embodiment of the present invention can obtain target data from an upstream database and generate at least one message to be transmitted based on the target data. Then, the transmission indication information of each message to be transmitted in the at least one message to be transmitted can be determined, and based on the transmission indication information of each message to be transmitted, M priority messages can be determined from the at least one message to be transmitted, where M is a non-negative integer. Furthermore, the transmission priority of each priority message can be determined based on the transmission indication information of each priority message in the M priority messages; and the M priority messages can be stored in a message priority queue in order of transmission priority from high to low. Based on this, the priority messages in the message priority queue can be sent to the output component in sequence to realize sending the M priority messages to the output component in order of transmission priority from high to low; and the received priority messages can be written into the target downstream database through the output component. It can be seen that the embodiment of the present invention can implement a message confirmation mechanism based on transmission priority through the PriorityConfirm component, which can ensure that high-priority messages are given priority during the transmission process, and confirm the high-priority messages immediately after writing through the fast confirmation interface (HighAck), so as to avoid sharing the standard confirmation process with low-priority messages (such as messages without priority) with higher latency; and the PriorityConfirm component introduces a message priority queue to sort the transmission priority from high to low and manage the messages received from the Scheduler (such as the M priority messages mentioned above), and can dynamically adjust the queue order to ensure the priority processing status of high-priority messages in the queue message priority column, even when new high-priority messages appear, the message priority queue can be re-sorted to ensure the timely transmission of important data; based on this, the embodiment of the present invention can effectively improve the transmission efficiency of high-priority messages, thereby effectively improving the data transmission efficiency.

[0077] Based on the description of the above-mentioned data transmission method, the present invention also provides a data transmission device; Figure 4 As shown, the data transmission device may include an acquisition unit 401 and a processing unit 402. The data transmission device may execute Figure 1 or Figure 3 The data transmission method shown, that is, the data transmission device can run the above-mentioned units:

[0078] An acquisition unit 401 is used to acquire target data from an upstream database;

[0079] A processing unit 402 is configured to generate at least one message to be transmitted based on the target data;

[0080] The processing unit 402 is further configured to determine transmission indication information of each message to be transmitted in the at least one message to be transmitted, and determine M priority messages from the at least one message to be transmitted based on the transmission indication information of each message to be transmitted, where M is a non-negative integer;

[0081] The processing unit 402 is further configured to send the M priority messages to the output component in descending order of transmission priority, and write the received priority messages into a target downstream database through the output component.

[0082] In one implementation, when determining the transmission indication information of each message to be transmitted in the at least one message to be transmitted, the processing unit 402 may be specifically configured to:

[0083] For any of the at least one message to be transmitted, determine whether the waiting confirmation time of any of the message to be transmitted is greater than a preset waiting confirmation time threshold, and / or determine whether the number of messages in the queue in which any of the message to be transmitted is located exceeds a queue cache threshold and whether any of the message to be transmitted is located within a specified position range in the corresponding queue;

[0084] If the pending confirmation time of any message to be transmitted is greater than the preset pending confirmation time threshold, and / or the number of messages in the queue where any message to be transmitted is located exceeds the queue cache threshold and any message to be transmitted is located within the specified position range in the corresponding queue, then the priority indication information corresponding to any message to be transmitted is determined, and the priority indication information corresponding to any message to be transmitted is used as the transmission indication information of any message to be transmitted. A message with one transmission indication information as one priority indication information is treated as a priority message.

[0085] In another implementation, the processing unit 402 may also be used to:

[0086] Determine the transmission priority of each priority message based on the transmission indication information of each priority message in the M priority messages;

[0087] The M priority messages are stored in a message priority queue in descending order of transmission priority;

[0088] When the processing unit 402 sends the M priority messages to the output component in the order of transmission priority from high to low, it can be specifically used to:

[0089] The priority messages in the message priority queue are sent to the output component in sequence, so as to send the M priority messages to the output component in the order of transmission priority from high to low.

[0090] In another implementation manner, when the processing unit 402 determines the transmission priority of each priority message based on the transmission indication information of each priority message in the M priority messages, it can be specifically used to:

[0091] Determining the sequence number of each priority message in the M priority messages;

[0092] Based on the sequence number and transmission indication information of each priority message, the transmission priority of each priority message is determined; wherein, if the transmission level indicated by the transmission indication information of the first priority message among the M priority messages is equal to the transmission level indicated by the transmission indication information of the second priority message and the sequence number of the first priority message is less than the sequence number of the second priority message, or the transmission level indicated by the transmission indication information of the first priority message is higher than the transmission level indicated by the transmission indication information of the second priority message, then the transmission priority of the first priority message is higher than the transmission priority of the second priority message.

[0093] In another implementation, the message priority queue is located in the priority confirmation component, and the processing unit 402 may also be used to:

[0094] Receiving the M priority messages through the priority confirmation component;

[0095] When the processing unit 402 determines the transmission priority of each priority message based on the transmission indication information of each priority message in the M priority messages, it can be specifically used to:

[0096] The priority confirmation component determines the transmission priority of each priority message based on the transmission indication information of each priority message in the M priority messages.

[0097] In another implementation, the M priority messages are sent to the output component via a priority confirmation component, and the processing unit 402 may also be used to:

[0098] For any priority message among the M priority messages, after the any priority message is written into the target downstream database, the message confirmation information of the any priority message is returned to the priority confirmation component through the output component;

[0099] Through the priority confirmation component, based on the message confirmation information of any priority message, it is confirmed that any priority message has been written into the target downstream database, and the next priority message of any priority message is sent to the output component in order of transmission priority from high to low.

[0100] In another implementation, the processing unit 402 may also be used to:

[0101] For any priority message among the M priority messages, based on the transmission indication information of the any priority message, determine a retry strategy and a retry duration threshold corresponding to the any priority message;

[0102] If the waiting retry time of any priority message exceeds the retry time threshold corresponding to any priority message, any priority message will be retried according to the retry strategy corresponding to any priority message until any priority message reaches the stop retry condition.

[0103] According to one embodiment of the present invention, Figure 4 Each unit in the data transmission device shown can be separately or completely combined into one or several other units to form, or one (some) of the units can be further divided into multiple functionally smaller units to form, which can achieve the same operation without affecting the realization of the technical effects of the embodiments of the present invention. The above-mentioned units are divided based on logical functions. In practical applications, the function of one unit can also be implemented by multiple units, or the functions of multiple units can be implemented by one unit. In other embodiments of the present invention, any data transmission device may also include other units. In practical applications, these functions can also be implemented with the assistance of other units, and can be implemented by the collaboration of multiple units.

[0104] According to another embodiment of the present invention, the program can be executed by running a program on a general electronic device such as a computer including a central processing unit (CPU), a random access memory medium (RAM), a read-only memory medium (ROM), and other processing elements and storage elements. Figure 1 or Figure 3 A computer program (including program code) for each step involved in the corresponding method shown in Figure 4 The data transmission device shown in the embodiment of the present invention is used to implement the data transmission method of the embodiment of the present invention. The computer program can be recorded on a computer storage medium, for example, and loaded into the above-mentioned electronic device through the computer storage medium and run therein.

[0105] The embodiment of the present invention can obtain target data from an upstream database and generate at least one message to be transmitted based on the target data. Then, the transmission indication information of each message to be transmitted in the at least one message to be transmitted can be determined, and based on the transmission indication information of each message to be transmitted, M priority messages can be determined from the at least one message to be transmitted, where M is a non-negative integer. Further, the M priority messages can be sent to the output component in order of transmission priority from high to low, and the received priority messages can be written into the target downstream database through the output component. It can be seen that the embodiment of the present invention can send the determined priority messages in order of transmission priority from high to low, thereby avoiding low priority data affecting the transmission efficiency of high priority data, effectively improving data transmission efficiency, and further effectively improving the transmission efficiency of critical data (i.e., priority messages with higher transmission priority).

[0106] Based on the description of the above method embodiment and device embodiment, the exemplary embodiment of the present invention further provides an electronic device, including: at least one processor; and a memory connected to the at least one processor in communication. The memory stores a computer program that can be executed by the at least one processor, and the computer program is used to enable the electronic device to perform the method according to the embodiment of the present invention when executed by the at least one processor.

[0107] Exemplary embodiments of the present invention also provide a non-transitory computer-readable storage medium storing a computer program, wherein the computer program, when executed by a processor of a computer, is used to cause the computer to perform a method according to an embodiment of the present invention.

[0108] An exemplary embodiment of the present invention further provides a computer program product, comprising a computer program, wherein when the computer program is executed by a processor of a computer, the computer is used to enable the computer to perform a method according to an embodiment of the present invention.

[0109] An exemplary embodiment of the present invention further provides a data transmission system, which includes at least one upstream database, at least one downstream database, and a DTS service device; wherein the DTS service device can be used to execute the method according to the embodiment of the present invention.

[0110] refer to Figure 5, a block diagram of an electronic device 500 that can be used as a server or client of the present invention will now be described, which is an example of a hardware device that can be applied to various aspects of the present invention. The electronic device is intended to represent various forms of digital electronic computer devices, such as laptop computers, desktop computers, workbenches, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital processing, cellular phones, smart phones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present invention described and / or required herein.

[0111] like Figure 5 As shown, the electronic device 500 includes a computing unit 501, which can perform various appropriate actions and processes according to a computer program stored in a read-only memory (ROM) 502 or a computer program loaded from a storage unit 508 into a random access memory (RAM) 503. In the RAM 503, various programs and data required for the operation of the electronic device 500 can also be stored. The computing unit 501, the ROM 502, and the RAM 503 are connected to each other via a bus 504. An input / output (I / O) interface 505 is also connected to the bus 504.

[0112] A plurality of components in the electronic device 500 are connected to the I / O interface 505, including: an input unit 506, an output unit 507, a storage unit 508, and a communication unit 509. The input unit 506 may be any type of device capable of inputting information to the electronic device 500, and the input unit 506 may receive input digital or character information, and generate key signal inputs related to user settings and / or function control of the electronic device. The output unit 507 may be any type of device capable of presenting information, and may include, but is not limited to, a display, a speaker, a video / audio output terminal, a vibrator, and / or a printer. The storage unit 508 may include, but is not limited to, a disk, an optical disk. The communication unit 509 allows the electronic device 500 to exchange information / data with other devices through a computer network such as the Internet and / or various telecommunication networks, and may include, but is not limited to, a modem, a network card, an infrared communication device, a wireless communication transceiver, and / or a chipset, such as a Bluetooth™ device, a WiFi device, a WiMax device, a cellular communication device, and / or the like.

[0113] The computing unit 501 may be a variety of general and / or special processing components with processing and computing capabilities. Some examples of the computing unit 501 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various dedicated artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, digital signal processors (DSPs), and any appropriate processors, controllers, microcontrollers, etc. The computing unit 501 performs the various methods and processes described above. For example, in some embodiments, the data transmission method may be implemented as a computer software program, which is tangibly included in a machine-readable medium, such as a storage unit 508. In some embodiments, part or all of the computer program may be loaded and / or installed on the electronic device 500 via the ROM 502 and / or the communication unit 509. In some embodiments, the computing unit 501 may be configured to perform the data transmission method in any other appropriate manner (e.g., by means of firmware).

[0114] The program code for implementing the method of the present invention can be written in any combination of one or more programming languages. These program codes can be provided to a processor or controller of a general-purpose computer, a special-purpose computer or other programmable data processing device, so that the program code, when executed by the processor or controller, enables the functions / operations specified in the flow chart and / or block diagram to be implemented. The program code can be executed entirely on the machine, partially on the machine, partially on the machine as a stand-alone software package and partially on a remote machine, or entirely on a remote machine or server.

[0115] In the context of the present invention, a machine-readable medium may be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, device, or equipment. A machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium may include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, devices, or equipment, or any suitable combination of the foregoing. A more specific example of a machine-readable storage medium may include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0116] As used herein, the terms "machine-readable medium" and "computer-readable medium" refer to any computer program product, apparatus, and / or device (e.g., disk, optical disk, memory, programmable logic device (PLD)) for providing machine instructions and / or data to a programmable processor, including a machine-readable medium that receives machine instructions as a machine-readable signal. The term "machine-readable signal" refers to any signal for providing machine instructions and / or data to a programmable processor.

[0117] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user can provide input to the computer. Other types of devices can also be used to provide interaction with the user; for example, the feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including acoustic input, voice input, or tactile input).

[0118] The systems and techniques described herein may be implemented in a computing system that includes back-end components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes front-end components (e.g., a user computer with a graphical user interface or a web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such back-end components, middleware components, or front-end components. The components of the system may be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: a local area network (LAN), a wide area network (WAN), and the Internet.

[0119] A computer system may include clients and servers. Clients and servers are generally remote from each other and usually interact through a communication network. The relationship of client and server is generated by computer programs running on respective computers and having a client-server relationship to each other.

[0120] Furthermore, it should be understood that what is disclosed above is only a preferred embodiment of the present invention, and certainly cannot be used to limit the scope of rights of the present invention. Therefore, equivalent changes made according to the claims of the present invention are still within the scope of the present invention.

Claims

1. A data transmission method, characterized in that: The method comprises: Acquire target data from an upstream database, and generate at least one message to be transmitted based on the target data; Determine transmission indication information of each message to be transmitted in the at least one message to be transmitted, and determine M priority messages from the at least one message to be transmitted based on the transmission indication information of each message to be transmitted, where M is a non-negative integer; The M priority messages are sent to the output component in order of transmission priority from high to low, and the received priority messages are written into the target downstream database through the output component.

2. The method according to claim 1, characterized in that The determining of transmission indication information of each message to be transmitted in the at least one message to be transmitted includes: For any of the at least one message to be transmitted, determine whether the waiting confirmation time of any of the message to be transmitted is greater than a preset waiting confirmation time threshold, and / or determine whether the number of messages in the queue in which any of the message to be transmitted is located exceeds a queue cache threshold and whether any of the message to be transmitted is located within a specified position range in the corresponding queue; If the pending confirmation time of any message to be transmitted is greater than the preset pending confirmation time threshold, and / or the number of messages in the queue where any message to be transmitted is located exceeds the queue cache threshold and any message to be transmitted is located within the specified position range in the corresponding queue, then the priority indication information corresponding to any message to be transmitted is determined, and the priority indication information corresponding to any message to be transmitted is used as the transmission indication information of any message to be transmitted. A message with one transmission indication information as one priority indication information is treated as a priority message.

3. The method according to claim 1 or 2, characterized in that: The method further comprises: Determine the transmission priority of each priority message based on the transmission indication information of each priority message in the M priority messages; The M priority messages are stored in a message priority queue in descending order of transmission priority; The sending the M priority messages to the output component in order of transmission priority from high to low includes: The priority messages in the message priority queue are sent to the output component in sequence, so as to send the M priority messages to the output component in the order of transmission priority from high to low.

4. The method according to claim 3, characterized in that The determining, based on the transmission indication information of each priority message in the M priority messages, the transmission priority of each priority message comprises: Determining the sequence number of each priority message in the M priority messages; Based on the sequence number and transmission indication information of each priority message, the transmission priority of each priority message is determined; wherein, if the transmission level indicated by the transmission indication information of the first priority message among the M priority messages is equal to the transmission level indicated by the transmission indication information of the second priority message and the sequence number of the first priority message is less than the sequence number of the second priority message, or the transmission level indicated by the transmission indication information of the first priority message is higher than the transmission level indicated by the transmission indication information of the second priority message, then the transmission priority of the first priority message is higher than the transmission priority of the second priority message.

5. The method according to claim 3, characterized in that: The message priority queue is located in the priority confirmation component, and the method further includes: Receiving the M priority messages through the priority confirmation component; The determining, based on the transmission indication information of each priority message in the M priority messages, the transmission priority of each priority message comprises: The priority confirmation component determines the transmission priority of each priority message based on the transmission indication information of each priority message in the M priority messages.

6. The method according to claim 1 or 2, characterized in that: The M priority messages are sent to the output component via a priority confirmation component, and the method further includes: For any priority message among the M priority messages, after the any priority message is written into the target downstream database, the message confirmation information of the any priority message is returned to the priority confirmation component through the output component; Through the priority confirmation component, based on the message confirmation information of any priority message, it is confirmed that any priority message has been written into the target downstream database, and the next priority message of any priority message is sent to the output component in order of transmission priority from high to low.

7. The method according to claim 1 or 2, characterized in that: The method further comprises: For any priority message among the M priority messages, based on the transmission indication information of the any priority message, determine a retry strategy and a retry duration threshold corresponding to the any priority message; If the waiting retry time of any priority message exceeds the retry time threshold corresponding to any priority message, any priority message will be retried according to the retry strategy corresponding to any priority message until any priority message reaches the stop retry condition.

8. A data transmission device, characterized in that: The device comprises: An acquisition unit, used for acquiring target data from an upstream database; a processing unit, configured to generate at least one message to be transmitted based on the target data; The processing unit is further configured to determine transmission indication information of each message to be transmitted in the at least one message to be transmitted, and determine M priority messages from the at least one message to be transmitted based on the transmission indication information of each message to be transmitted, where M is a non-negative integer; The processing unit is also used to send the M priority messages to the output component in order of transmission priority from high to low, and write the received priority messages into the target downstream database through the output component.

9. An electronic device, characterized in that: include: processor; as well as Memory for storing programs, The program includes instructions, which, when executed by the processor, cause the processor to perform the method according to any one of claims 1 to 7.

10. A non-transitory computer-readable storage medium storing computer instructions, wherein: The computer instructions are used to make a computer execute the method according to any one of claims 1-7.

Images