Method and device for monitoring the flow of interaction data, storage medium and electronic device
By using the asynchronous threading mechanism in the message middleware, the unsuccessful data is stored in the target database and supplemented, the problem of data loss in the message middleware is solved, and the data integrity and reliability are achieved.
Patent Information
- Application Number
- CN202210283493.9
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2022-03-22
- Publication Date
- 2025-06-27
- Estimated Expiration
- 2042-03-22
AI Technical Summary
During the process of sending data to the data consumption device by sending data in the message middleware, the unsuccessful data will not be re-entered, resulting in data loss.
By using the first thread in the message middleware to send the first data set to the data consumption device, and when the data is not successfully sent, the second data set that is not successfully sent is stored in the target database, and the data in the target database is asynchronously sent through the second thread.
When there is data that has not been successfully sent to the data consumption device, the data that has not been successfully sent is supplemented for data re-entering, avoiding data loss and solving the problem of data loss.
Smart Images

Figure CN114911636B_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of communications, and in particular, to a method and device for monitoring the flow of interaction data, a storage medium, and an electronic device. Background Art
[0002] With the rapid development of the Internet of Things, more and more households are equipped with smart home appliances. When users use smart home appliances, for example, in a voice interaction scenario, when the voice background program processes a user's request, it will send the request parameters and responses to the message queue KAFKA asynchronously, and then transfer them to different types of databases through KAFKA. During the processing, due to unexpected situations such as data format problems, excessive network pressure, or high IO load, the data may not be successfully written into the database, resulting in data loss.
[0003] In view of the related art, during the process of the message middleware sending data to the data consumption device, there is no re-recording of the data that fails to be sent, resulting in the problem of data loss, and no effective solution has been proposed yet.
[0004] Therefore, it is necessary to improve the related art to overcome the above-mentioned defects in the related art. Summary of the Invention
[0005] Embodiments of the present invention provide a method and device for monitoring the flow of interaction data, a storage medium, and an electronic device, so as to at least solve the problem that during the process of the message middleware sending data to the data consumption device, there is no re-recording of the data that fails to be sent, resulting in data loss.
[0006] According to one aspect of an embodiment of the present invention, there is provided a method for monitoring the flow of interaction data, including: when a message middleware sends a first data set to a data consumption device through a first thread, determining whether the data in the first data set is successfully sent by the message middleware to the data consumption device through the first thread, where the first data set includes interaction data during the use of a target interaction device, and the interaction data in the first data set is the interaction data in a set of interaction scenarios of the target interaction device subscribed by the data consumption device; when there is a second data set in the first data set that is not successfully sent by the message middleware to the data consumption device, storing the second data set in the first data set that is not successfully sent to the data consumption device in a target database, where the second data set includes interaction data in one or more interaction scenarios in the set of interaction scenarios; sending, through a second thread, the interaction data in some or all of the interaction scenarios included in the second data set stored in the target database to the data consumption device, where the first thread and the second thread are asynchronous threads.
[0007] Further, the storing the second data set in the first data set that is not successfully sent to the data consumption device in the target database includes: when the first data set is a data set obtained from the log information of the target interaction device, storing the interaction data in the same interaction scenario included in the second data set at the same storage location in the target database, where at least some of the interaction data in the same interaction scenario in the set of interaction scenarios are not adjacent in the record position in the log information.
[0008] Further, before determining whether the data in the first data set is successfully sent by the message middleware to the data consumption device through the first thread, the method further includes: storing, through the message middleware, the interaction data in each interaction scenario in the set of interaction scenarios included in the first data set into corresponding multiple cache spaces of the message middleware, where each cache space in the multiple cache spaces is used to cache the interaction data in one interaction scenario in the corresponding set of interaction scenarios.
[0009] Further, before determining whether the data in the first data set is successfully sent by the message middleware to the data consumption device through the first thread, the method further includes: sequentially reading, through the message middleware, the interaction data in each interaction scenario in the set of interaction scenarios from the multiple cache spaces, and sending the interaction data read from the multiple cache spaces to the data consumption device through the first thread.
[0010] Further, sending, by the second thread, the interaction data in some or all of the interaction scenarios included in the second data set stored in the target database to the data consumption device includes: taking the interaction data in one interaction scenario as a unit, and sending, by the second thread, the interaction data in each of the some or all of the interaction scenarios stored in the target database to the data consumption device.
[0011] Further, sending, by the second thread, the interaction data in some or all of the interaction scenarios included in the second data set stored in the target database to the data consumption device includes: when it is detected that the data consumption device needs to process the interaction data in a target interaction scenario, determining whether there is interaction data in the target interaction scenario that has not been successfully sent stored in the target database; when it is determined that there is interaction data in the target interaction scenario that has not been successfully sent stored in the target database, sending, by the second thread, the interaction data in the target interaction scenario stored in the target database to the data consumption device.
[0012] Further, when it is detected that the data consumption device needs to process the interaction data in a target interaction scenario, determining whether there is interaction data in the target interaction scenario that has not been successfully sent stored in the target database includes: when it is detected that the data consumption device needs to process the interaction data in a target interaction scenario, comparing the interaction data in the target interaction scenario on the data consumption device with the interaction data in the target interaction scenario on a group of consumption devices, where the message middleware is used to send the first data set to the data consumption device and the group of consumption devices respectively; when the interaction data in the target interaction scenario on the data consumption device is different from the interaction data in the target interaction scenario on at least N consumption devices in the group of consumption devices, determining that the interaction data in the target interaction scenario on the data consumption device is incomplete, and determining whether there is interaction data in the target interaction scenario that has not been successfully sent stored in the target database, where N is a positive integer greater than or equal to 1.
[0013] According to another aspect of the embodiments of the present invention, there is also provided a device for monitoring the flow of interaction data, including: a determination module, configured to determine whether the data in the first data set is successfully sent by the message middleware to the data consumption device through the first thread when the message middleware sends the first data set to the data consumption device through the first thread, where the first data set includes interaction data during the use of the target interaction device, and the interaction data in the first data set is the interaction data in a set of interaction scenarios of the target interaction device subscribed by the data consumption device; a storage module, configured to store the second data set that is not successfully sent by the message middleware to the data consumption device in the first data set in the target database when there is a second data set in the first data set that is not successfully sent by the message middleware to the data consumption device, where the second data set includes the interaction data in one or more interaction scenarios in the set of interaction scenarios; a sending module, configured to send, through a second thread, the interaction data in some or all of the interaction scenarios included in the second data set stored in the target database to the data consumption device, where the first thread and the second thread are asynchronous threads.
[0014] According to still another aspect of the embodiments of the present invention, there is also provided a computer-readable storage medium storing a computer program, where the computer program is configured to execute the above-mentioned method for monitoring the flow of interaction data when running.
[0015] According to still another aspect of the embodiments of the present invention, there is also provided an electronic device including a memory, a processor, and a computer program stored on the memory and executable on the processor, where the above-mentioned processor executes the above-mentioned method for monitoring the flow of interaction data through the computer program.
[0016] Through the present invention, when the message middleware sends the first data set to the data consumption device through the first thread, it is determined whether the data in the first data set is successfully sent by the message middleware to the data consumption device through the first thread, and when there is a second data set in the first data set that is not successfully sent by the message middleware to the data consumption device, the second data set is stored in the target database, and the interaction data in some or all of the interaction scenarios included in the second data set stored in the target database is sent to the data consumption device through the second thread. By adopting the above technical solution, it is realized that when there is data in the message middleware that is not successfully sent to the data consumption device, the unsuccessfully sent data is supplemented, avoiding data loss. It solves the problem that during the process of the message middleware sending data to the data consumption device, the unsuccessfully sent data is not supplemented, resulting in data loss. Description of the Drawings
[0017] The accompanying drawings described herein are used to provide a further understanding of the present invention and form a part of this application. The exemplary embodiments of the present invention and their descriptions are used to explain the present invention and do not constitute an improper limitation of the present invention. In the drawings:
[0018] Figure 1 is a hardware structure block diagram of a computer terminal for the method of monitoring the flow of interaction data in an embodiment of the present invention;
[0019] Figure 2 is a flowchart of the method of monitoring the flow of interaction data according to an embodiment of the present invention;
[0020] Figure 3 is a framework diagram of the method of monitoring the flow of interaction data according to an embodiment of the present invention;
[0021] Figure 4 is a block diagram (I) of the structure of the device for monitoring the flow of interaction data according to an embodiment of the present invention.
[0022] Figure 5 is a block diagram (II) of the structure of the device for monitoring the flow of interaction data according to an embodiment of the present invention. Detailed implementation manners
[0023] In order to enable those skilled in the art of the present technology to better understand the solution of the present invention, the technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the protection scope of the present invention.
[0024] It should be noted that the terms "first", "second", etc. in the specification and claims of the present invention and the above-mentioned accompanying drawings are used to distinguish similar objects and do not necessarily need to describe a specific order or sequence. It should be understood that such used data can be interchanged under appropriate circumstances so that the embodiments of the present invention described herein can be implemented in an order different from those illustrated or described herein. In addition, the terms "comprising" and "having" and any variations thereof are intended to cover non-exclusive inclusion. For example, a process, method, system, product or device comprising a series of steps or units does not necessarily have to be limited to those steps or units clearly listed, but may include other steps or units not clearly listed or inherent to these processes, methods, products or devices.
[0025] The method embodiments provided in the embodiments of the present application can be executed on a computer terminal or a similar computing device. Taking the operation on a computer terminal as an example, Figure 1It is a hardware block diagram of a computer terminal for the interaction data flow monitoring method according to an embodiment of the present invention. As Figure 1 shown, the computer terminal may include one or more ( Figure 1 only one is shown in the figure) processors 102 (the processor 102 may include, but is not limited to, a microprocessor (abbreviated as MPU) or a programmable logic device (abbreviated as PLD)) and a memory 104 for storing data. In an exemplary embodiment, the above computer terminal may further include a transmission device 106 for communication functions and an input / output device 108. Those of ordinary skill in the art can understand that Figure 1 the structure shown is only illustrative and does not limit the structure of the above computer terminal. For example, the computer terminal may further include more or fewer components than Figure 1 shown in the figure, or have an equivalent function to Figure 1 shown in the figure or different configurations with more functions than Figure 1 shown in the figure.
[0026] The memory 104 can be used to store computer programs. For example, software programs and modules of application software, such as the computer program corresponding to the interaction data flow monitoring method in the embodiment of the present invention. The processor 102 executes various functional applications and data processing by running the computer program stored in the memory 104, that is, implements the above method. The memory 104 may include a high-speed random access memory, and may also include a non-volatile memory, such as one or more magnetic storage devices, flash memories, or other non-volatile solid-state memories. In some instances, the memory 104 may further include a memory remotely provided relative to the processor 102, and these remote memories can be connected to the computer terminal through a network. Examples of the above network include, but are not limited to, the Internet, enterprise intranets, local area networks, mobile communication networks, and combinations thereof.
[0027] The transmission device 106 is used to receive or send data via a network. Specific examples of the above network may include a wireless network provided by a communication provider of the computer terminal. In one instance, the transmission device 106 includes a network adapter (abbreviated as NIC), which can be connected to other network devices through a base station and thus communicate with the Internet. In one instance, the transmission device 106 may be a radio frequency (abbreviated as RF) module, which is used to communicate with the Internet wirelessly.
[0028] To solve the above problems, in this embodiment, a method for monitoring the flow of interaction data is provided. Figure 2It is a flowchart of a method for monitoring the flow of interaction data according to an embodiment of the present invention. The process includes the following steps:
[0029] Step S202, when the message middleware sends a first data set to the data consumption device through a first thread, determine whether the data in the first data set is successfully sent by the message middleware to the data consumption device through the first thread. Among them, the first data set includes interaction data generated when the target interaction device is in use, and the interaction data in the first data set is the interaction data in a set of interaction scenarios of the target interaction device subscribed by the data consumption device;
[0030] In an exemplary embodiment, the above message middleware includes but is not limited to KAFKA. Among them, KAFKA is an open-source stream processing platform developed by the Apache Software Foundation, written in Scala and Java. It is a message middleware with low latency, high reliability, scalability, and ease of use.
[0031] It should be noted that, in an exemplary embodiment, the above interaction data may be interaction data generated by a user's voice interaction with the target interaction device. The target interaction device includes but is not limited to smart speakers, smart refrigerators, smart air conditioners, etc. The above set of interaction scenarios includes but is not limited to: voice interaction scenarios, control interaction scenarios, etc.
[0032] Step S204, when there is a second data set in the first data set that is not successfully sent by the message middleware to the data consumption device, store the second data set in the first data set that is not successfully sent to the data consumption device in the target database. Among them, the second data set includes interaction data in one or more interaction scenarios in the set of interaction scenarios;
[0033] It should be noted that, in an exemplary embodiment, the above target database is REDIS. Among them, REDIS is the Remote Dictionary Server, an open-source log-type, Key-Value database written in ANSI C language, supporting networking, and can be based on memory or persistent, and provides APIs in multiple languages.
[0034] Step S206, send the interaction data in some or all of the interaction scenarios included in the second data set stored in the target database to the data consumption device through a second thread. Among them, the first thread and the second thread are asynchronous threads.
[0035] Through the above steps, when the message middleware sends the first data set to the data consumption device through the first thread, it is determined whether the data in the first data set is successfully sent to the data consumption device by the message middleware through the first thread. When there is a second data set in the first data set that is not successfully sent to the data consumption device by the message middleware, the second data set is stored in the target database, and the interaction data in some or all of the interaction scenarios included in the second data set stored in the target database is sent to the data consumption device through the second thread. By adopting the above technical solution, when there is data that has not been successfully sent to the data consumption device in the message middleware, the data that has not been successfully sent is supplemented, avoiding data loss. The problem that data loss occurs because the message middleware does not supplement the data that has not been successfully sent during the process of sending data to the data consumption device is solved.
[0036] In an exemplary embodiment, before performing the above step S202, the interaction data in each interaction scenario included in the first data set can be separately stored in multiple cache spaces corresponding to the message middleware by the message middleware, where each cache space in the multiple cache spaces is used to cache the interaction data in one interaction scenario included in the corresponding group of interaction scenarios.
[0037] It should be noted that if the message middleware is KAFKA, the above cache space is equivalent to a topic in KAFKA, where KAFKA abstracts and summarizes a group of data into a topic. That is to say, a topic corresponds to a classification of data. The producer sends the data to a specific topic, and the consumer subscribes to the topic for consumption processing after subscribing to it.
[0038] In an exemplary embodiment, before determining whether the data in the first data set is successfully sent to the data consumption device by the message middleware through the first thread, it is also necessary to read the interaction data in each interaction scenario included in the group of interaction scenarios from the multiple cache spaces in sequence by the message middleware, and send the interaction data read from the multiple cache spaces to the data consumption device through the first thread.
[0039] Specifically, the message middleware sequentially reads the interaction data of one interaction scenario in a group of interaction scenarios from each of the multiple cache spaces, and sequentially sends the interaction data of one interaction scenario to the data consumption device through the first thread.
[0040] In an exemplary embodiment, storing the second data set that has not been successfully sent to the data consumption device in the first data set into the target database can be achieved in the following manner: when the first data set is a data set obtained from the log information of the target interaction device, store the interaction data in the same interaction scenario included in the second data set at the same storage location in the target database. It should be noted that the recording positions of at least some of the interaction data in the same interaction scenario in the group of interaction scenarios are not adjacent in the log information.
[0041] That is to say, when storing the second data set in the target database, since there is data from different interaction scenarios in the second data set, in order to facilitate subsequent supplementary recording at the level of interaction scenarios, the interaction data with the same interaction scenario in the second data set is stored at the same storage location in the target database.
[0042] There are multiple implementation manners for the above step S206. In an exemplary embodiment, it can be achieved in the following manner: taking the interaction data in one interaction scenario as a unit, send the interaction data in each interaction scenario of the partial or all interaction scenarios stored in the target database to the data consumption device through the second thread.
[0043] In an exemplary embodiment, it can also be achieved in the following manner: when it is detected that the data consumption device needs to process the interaction data in the target interaction scenario, determine whether there is interaction data in the target interaction scenario that has not been successfully sent stored in the target database; when it is determined that there is interaction data in the target interaction scenario that has not been successfully sent stored in the target database, send the interaction data in the target interaction scenario stored in the target database to the data consumption device through the second thread.
[0044] Specifically, in an exemplary embodiment, when it is detected that the data consumption device needs to process the interaction data in the target interaction scenario, it can be determined whether the target database stores the interaction data in the target interaction scenario that has not been successfully sent in the following manner: when it is detected that the data consumption device needs to process the interaction data in the target interaction scenario, compare the interaction data in the target interaction scenario on the data consumption device with the interaction data in the target interaction scenario on a group of consumption devices, where the message middleware is used to send the first data set to the data consumption device and the group of consumption devices respectively; when the interaction data in the target interaction scenario on the data consumption device is different from the interaction data in the target interaction scenario on at least N consumption devices in the group of consumption devices, it is determined that the interaction data in the target interaction scenario on the data consumption device is incomplete, and it is determined whether the target database stores the interaction data in the target interaction scenario that has not been successfully sent, where N is a positive integer greater than or equal to 1.
[0045] In an alternative embodiment, it is also possible to count the first quantity of data sent by the message middleware to the data consumption device and the second quantity of data received by the data consumption device, and then, when the first quantity and the second quantity are not equal, send the difference between the first quantity and the second quantity to the management personnel through the communication software to inform the management personnel of how much data has not been successfully sent.
[0046] Furthermore, it is also possible to determine the third quantity of data supplemented to the data consumption device by the target database, and then send the difference between the fourth quantity and the third quantity to the management personnel through the communication software to inform the management personnel that multiple data have not been successfully supplemented, where the fourth quantity is the difference between the first quantity and the second quantity.
[0047] Obviously, the above-described embodiments are only a part of the embodiments of the present invention, rather than all of the embodiments. To better understand the above method for monitoring the flow of interaction data, the following will describe the above process in conjunction with embodiments, but it is not used to limit the technical solutions of the embodiments of the present invention. Specifically:
[0048] In an alternative embodiment, Figure 3 is a framework diagram of the method for monitoring the flow of interaction data according to the embodiments of the present invention. Specifically, the embodiments of the present application include the following parts:
[0049] (1) The voice application program sends data to the topic specified by KAFKA asynchronously, which can reduce the blocking of the main thread;
[0050] (2) During the process of data processing by the data processing application, if an exception is detected, the data is persisted to the REDIS database by means of a lock to avoid data loss caused by excessive data transfer among applications.
[0051] (3) Every once in a while, count the quantity of data transferred by KAFKA to each database and notify the staff by email.
[0052] (4) During data transfer, if the transfer fails, the data will be written into the REDIS database. After a period of time, the data in the REDIS database will be backfilled to each database to complete the data backfilling, and the staff will be notified by email of the backfilled data and the quantity of failed backfilling.
[0053] It should be noted that the function of completing data backfilling is a separate thread and will not affect the original business. At the same time, after an exception occurs, the data backfilling thread can receive messages in near real-time for data processing, ensuring the real-time nature of data backfilling.
[0054] That is, in the embodiment of the present application, in the case of an exception in data transfer, it is possible to quickly count the transferred data and notify the staff by email without affecting the main process. And if it is found that the data transfer fails during the data transfer process, automatic backfilling can be performed, and the staff will be notified of the successfully backfilled data and the quantity of data that has not been backfilled. When the staff receives the quantity of data that has not been successfully backfilled, they can query the corresponding data in REDIS to check the data structure or perform batch processing and then perform data storage again.
[0055] Through the description of the above implementation manners, those skilled in the art can clearly understand that the method according to the above embodiments can be implemented by means of software plus a necessary general hardware platform. Of course, it can also be implemented by hardware, but in many cases, the former is a better implementation manner. Based on such an understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art can be embodied in the form of a software product. This computer software product is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disc) and includes several instructions for causing a terminal device (which can be a mobile phone, computer, server, or network device, etc.) to execute the methods of the various embodiments of the present invention.
[0056] In this embodiment, a device for monitoring the transfer of interactive data is also provided. This device is used to implement the above embodiments and preferred implementation manners, and those that have been described will not be repeated here. As used below, the term "module" can be a combination of software and / or hardware that can achieve a predetermined function. Although the devices described in the following embodiments are preferably implemented in software, implementation in hardware, or a combination of software and hardware is also possible and contemplated.
[0057] Figure 4 It is a structural block diagram (1) of an interaction data flow monitoring device according to an embodiment of the present invention. The device includes:
[0058] A determination module 42, configured to determine whether the data in the first data set is successfully sent by the message middleware to the data consumption device through the first thread when the message middleware sends the first data set to the data consumption device through the first thread. Wherein, the first data set includes interaction data during the use of the target interaction device, and the interaction data in the first data set is the interaction data in a set of interaction scenarios of the target interaction device subscribed by the data consumption device;
[0059] A storage module 44, configured to store the second data set that is not successfully sent by the message middleware to the data consumption device in the first data set into the target database when there is a second data set in the first data set that is not successfully sent by the message middleware to the data consumption device. Wherein, the second data set includes interaction data in one or more interaction scenarios in the set of interaction scenarios;
[0060] A sending module 46, configured to send the interaction data in some or all of the interaction scenarios included in the second data set stored in the target database to the data consumption device through a second thread. Wherein, the first thread and the second thread are asynchronous threads.
[0061] With the above device, when the message middleware sends the first data set to the data consumption device through the first thread, it determines whether the data in the first data set is successfully sent by the message middleware to the data consumption device through the first thread, and when there is a second data set in the first data set that is not successfully sent by the message middleware to the data consumption device, it stores the second data set in the target database, and sends the interaction data in some or all of the interaction scenarios included in the second data set stored in the target database to the data consumption device through the second thread. By adopting the above technical solution, it realizes data supplementation for the data that is not successfully sent when there is data in the message middleware that is not successfully sent to the data consumption device, avoiding data loss. It solves the problem that during the process of the message middleware sending data to the data consumption device, it does not supplement the data that is not successfully sent, resulting in data loss.
[0062] In an exemplary embodiment, the storage module 44 is further configured to, when the first data set is a data set obtained from the log information of the target interaction device, store the interaction data in the same interaction scenario included in the second data set at the same storage location in the target database, where at least some of the interaction data in the same interaction scenario in the group of interaction scenarios are not adjacent in the recording positions in the log information.
[0063] In an exemplary embodiment, the storage module 44 is further configured to, through the message middleware, store the interaction data in each interaction scenario in the group of interaction scenarios included in the first data set into multiple corresponding cache spaces of the message middleware, where each cache space in the multiple cache spaces is used to cache the interaction data in one interaction scenario in the corresponding group of interaction scenarios.
[0064] Figure 5 FIG. (2) is a structural block diagram of a device for monitoring the flow of interaction data according to an embodiment of the present invention. The device includes: a processing module 48.
[0065] In an exemplary embodiment, the processing module 48 is further configured to sequentially read, through the message middleware, the interaction data in each interaction scenario in the group of interaction scenarios from the multiple cache spaces, and send the interaction data read from the multiple cache spaces to the data consumption device through the first thread.
[0066] In an exemplary embodiment, the sending module 46 is further configured to, taking the interaction data in one interaction scenario as a unit, send the interaction data in each interaction scenario in the partial or all interaction scenarios stored in the target database to the data consumption device through the second thread.
[0067] In an exemplary embodiment, the sending module 46 is further configured to, when detecting that the data consumption device needs to process the interaction data in a target interaction scenario, determine whether there is any interaction data in the target interaction scenario that has not been successfully sent stored in the target database; and when it is determined that there is interaction data in the target interaction scenario that has not been successfully sent stored in the target database, send the interaction data in the target interaction scenario stored in the target database to the data consumption device through the second thread.
[0068] In an exemplary embodiment, the processing module 48 is further configured to, when it is detected that the data consumption device needs to process the interaction data in the target interaction scenario, compare the interaction data in the target interaction scenario on the data consumption device with the interaction data in the target interaction scenario on a group of consumption devices, where the message middleware is used to send the first data set to the data consumption device and the group of consumption devices respectively; when the interaction data in the target interaction scenario on the data consumption device is different from the interaction data in the target interaction scenario on at least N consumption devices in the group of consumption devices, it is determined that the interaction data in the target interaction scenario on the data consumption device is incomplete, and it is determined whether the interaction data in the target interaction scenario that has not been successfully sent is stored in the target database, where N is a positive integer greater than or equal to 1.
[0069] An embodiment of the present invention further provides a computer-readable storage medium, in which a computer program is stored, where the computer program is configured to execute the steps in any one of the above method embodiments when running.
[0070] Optionally, in this embodiment, the above storage medium may be configured to store a computer program for executing the following steps:
[0071] S1. When the message middleware sends the first data set to the data consumption device through the first thread, determine whether the data in the first data set is successfully sent to the data consumption device by the message middleware through the first thread, where the first data set includes the interaction data when the target interaction device is in use, and the interaction data in the first data set is the interaction data in a group of interaction scenarios of the target interaction device subscribed by the data consumption device;
[0072] S2. When there is a second data set in the first data set that has not been successfully sent to the data consumption device by the message middleware, store the second data set in the first data set that has not been successfully sent to the data consumption device in the target database, where the second data set includes the interaction data in one or more interaction scenarios in the group of interaction scenarios;
[0073] S3. Send the interaction data in some or all of the interaction scenarios included in the second data set stored in the target database to the data consumption device through the second thread, where the first thread and the second thread are asynchronous threads.
[0074] In an exemplary embodiment, the above computer-readable storage medium may include, but is not limited to: various media such as USB flash drives, read-only memory (ROM for short), random access memory (RAM for short), mobile hard disks, magnetic disks, or optical discs that can store computer programs.
[0075] Specific examples in this embodiment may refer to the examples described in the above embodiments and exemplary embodiments, and will not be elaborated here.
[0076] An embodiment of the present invention also provides an electronic device, including a memory and a processor. A computer program is stored in the memory, and the processor is configured to run the computer program to execute the steps in any one of the above method embodiments.
[0077] Optionally, in this embodiment, the above processor may be configured to execute the following steps through a computer program:
[0078] S1. When the message middleware sends a first data set to the data consumption device through a first thread, determine whether the data in the first data set is successfully sent to the data consumption device by the message middleware through the first thread. Wherein, the first data set includes interaction data during the use of the target interaction device, and the interaction data in the first data set is the interaction data in a group of interaction scenarios of the target interaction device subscribed by the data consumption device;
[0079] S2. When there is a second data set in the first data set that is not successfully sent to the data consumption device by the message middleware, store the second data set in the first data set that is not successfully sent to the data consumption device in the target database. Wherein, the second data set includes interaction data in one or more interaction scenarios in the group of interaction scenarios;
[0080] S3. Send the interaction data in some or all of the interaction scenarios included in the second data set stored in the target database to the data consumption device through a second thread, where the first thread and the second thread are asynchronous threads.
[0081] In an exemplary embodiment, the above electronic device may further include a transmission device and an input / output device. Wherein, the transmission device is connected to the above processor, and the input / output device is connected to the above processor.
[0082] Specific examples in this embodiment may refer to the examples described in the above embodiments and exemplary embodiments, and will not be elaborated here.
[0083] Obviously, those skilled in the art should understand that the above-mentioned modules or steps of the present invention can be implemented by a general-purpose computing device. They can be concentrated on a single computing device or distributed on a network composed of multiple computing devices. They can be implemented by program codes executable by the computing device. Thus, they can be stored in a storage device and executed by the computing device. And in some cases, the steps shown or described can be executed in a sequence different from that here, or they can be separately fabricated into individual integrated circuit modules, or multiple modules or steps among them can be fabricated into a single integrated circuit module for implementation. In this way, the present invention is not limited to any specific combination of hardware and software.
[0084] The above are only the preferred embodiments of the present invention and are not intended to limit the present invention. For those skilled in the art, the present invention can have various changes and modifications. Any modification, equivalent replacement, improvement, etc. made within the principle of the present invention shall be included within the protection scope of the present invention.
Claims
1. A method for monitoring the flow of interactive data, characterized in that, Including: When the message middleware sends a first data set to a data consumption device through a first thread, determining whether the data in the first data set is successfully sent by the message middleware to the data consumption device through the first thread, where the first data set includes interaction data during the use of a target interaction device, and the interaction data in the first data set is the interaction data in a set of interaction scenarios of the target interaction device subscribed to by the data consumption device; When there is a second data set in the first data set that is not successfully sent by the message middleware to the data consumption device, storing the second data set in the first data set that is not successfully sent to the data consumption device in a target database, where the second data set includes interaction data in one or more interaction scenarios in the set of interaction scenarios; Sending, through a second thread, the interaction data in some or all of the interaction scenarios included in the second data set stored in the target database to the data consumption device, where the first thread and the second thread are asynchronous threads.
2. The method according to claim 1, characterized in that, The storing the second data set in the first data set that is not successfully sent to the data consumption device in the target database includes: When the first data set is a data set obtained from the log information of the target interaction device, storing the interaction data in the same interaction scenario included in the second data set at the same storage location in the target database, where at least some of the interaction data in the same interaction scenario in the set of interaction scenarios are not adjacent in the recording positions in the log information.
3. The method according to claim 1, wherein Before determining whether the data in the first data set is successfully sent by the message middleware to the data consumption device through the first thread, the method further includes: Through the message middleware, storing the interaction data in each interaction scenario in the set of interaction scenarios included in the first data set into corresponding multiple cache spaces of the message middleware, where each cache space in the multiple cache spaces is used to cache the interaction data in one interaction scenario in the corresponding set of interaction scenarios.
4. The method according to claim 3, characterized in that Before determining whether the data in the first data set is successfully sent by the message middleware to the data consumption device through the first thread, the method further includes: Through the message middleware, sequentially reading the interaction data in each interaction scenario in the set of interaction scenarios from the multiple cache spaces, and sending the interaction data read from the multiple cache spaces to the data consumption device through the first thread.
5. The method according to any one of claims 1 to 4, characterized in that, The sending, through a second thread, the interaction data in some or all of the interaction scenarios included in the second data set stored in the target database to the data consumption device includes: Taking the interaction data in one interaction scenario as a unit, and sending, through the second thread, the interaction data in each of the some or all of the interaction scenarios stored in the target database to the data consumption device.
6. The method according to any one of claims 1-4, characterized in that, Sending, by the second thread, some or all of the interaction data in the interaction scenarios included in the second data set stored in the target database to the data consumption device includes: When it is detected that the data consumption device needs to process the interaction data in a target interaction scenario, determining whether there is interaction data in the target interaction scenario that has not been successfully sent stored in the target database; When it is determined that there is interaction data in the target interaction scenario that has not been successfully sent stored in the target database, sending, by the second thread, the interaction data in the target interaction scenario stored in the target database to the data consumption device.
7. The method according to claim 6, characterized in that, The step of, when it is detected that the data consumption device needs to process the interaction data in a target interaction scenario, determining whether there is interaction data in the target interaction scenario that has not been successfully sent stored in the target database includes: When it is detected that the data consumption device needs to process the interaction data in a target interaction scenario, comparing the interaction data in the target interaction scenario on the data consumption device with the interaction data in the target interaction scenario on a group of consumption devices, where the message middleware is used to send the first data set to the data consumption device and the group of consumption devices respectively; When the interaction data in the target interaction scenario on the data consumption device is different from the interaction data in the target interaction scenario on at least N consumption devices in the group of consumption devices, determining that the interaction data in the target interaction scenario on the data consumption device is incomplete, and determining whether there is interaction data in the target interaction scenario that has not been successfully sent stored in the target database, where N is a positive integer greater than or equal to 1.
8. An interaction data transfer monitoring device, characterized in that, It includes: A determination module, configured to determine whether the data in the first data set has been successfully sent to the data consumption device by the message middleware through the first thread when the message middleware sends the first data set to the data consumption device through the first thread, where the first data set includes interaction data during the use of a target interaction device, and the interaction data in the first data set is the interaction data in a group of interaction scenarios of the target interaction device subscribed by the data consumption device; A storage module, configured to store, in the target database, the second data set in the first data set that has not been successfully sent to the data consumption device when there is a second data set in the first data set that has not been successfully sent to the data consumption device by the message middleware, where the second data set includes interaction data in one or more interaction scenarios in the group of interaction scenarios; A sending module, configured to send, by the second thread, some or all of the interaction data in the interaction scenarios included in the second data set stored in the target database to the data consumption device, where the first thread and the second thread are asynchronous threads.
9. A computer-readable storage medium, characterized in that, The computer-readable storage medium includes a stored program, wherein the program, when running, executes the method described in any one of claims 1 to 7 above.
10. An electronic device, comprising a memory and a processor, characterized in that, A computer program is stored in the memory, and the processor is configured to execute the method described in any one of claims 1 to 7 through the computer program.
Citation Information
Patent Citations
Data processing method and device, equipment and storage medium
CN110505315A
Interaction statement processing method, electronic equipment and computer readable storage medium
CN113064913A