IVVR telephone traffic log collection and storage method and electronic equipment

Abstract

The invention discloses an IVVR telephone traffic log collection and storage method and electronic equipment, and belongs to the technical field of data processing. Collection services are introduced into business services, the business services and the collection services are combined together to serve as a collection part, an application program is generated, and the application program is started to complete initialization of the collection part; the collection part collects log data in real time and sends the collected log data to the JMQ; a storage part obtains the log data from the JMQ and stores the log data into a storage layer according to a log storage algorithm, and the storage part comprises a storage service and a data service rate. According to the invention, the complexity is reduced, and the resource consumption is reduced; the dependency relationship between systems is reduced, the storage write-in pressure is reduced, and the storage efficiency is improved.

Description

Technical Field

[0001] The present application belongs to the field of data processing technology, and in particular relates to an IVVR traffic log collection and storage method and electronic device. Background Art

[0002] In the process of IVVR (interactive voice and video response) call processing provided by the existing intelligent audio and video platform, the product business needs to obtain the interaction records between the user terminal and the intelligent audio and video platform in real time to generate call details. The technical support side needs to centrally store this part of the interaction records to facilitate problem tracking based on logs; the interaction records are generated by the call service module of the intelligent audio and video platform and printed into the log file. The call service is distributed and deployed on different machines. Now it is necessary to formulate a plan to collect and store the call logs printed on different servers for subsequent analysis and retrieval.

[0003] Although the existing solutions for IVVR call log management are widely used, they have problems such as high threshold for use, high resource consumption, high hardware cost and performance bottleneck.

[0004] In view of the above problems, it is necessary to propose an IVVR traffic log collection and storage method and electronic device of the present application. Summary of the invention

[0005] In order to solve the deficiencies of the prior art, the present application provides an IVVR traffic log collection and storage method and electronic device, which can solve the problems of high usage threshold, high resource consumption, high hardware cost and performance bottleneck in the prior art of IVVR traffic log management.

[0006] The technical effects to be achieved by this application are achieved through the following solutions:

[0007] In a first aspect, the present application provides a method for collecting and storing IVVR traffic logs, the method comprising:

[0008] Introducing a collection service into a business service, merging the business service and the collection service together as a collection part and generating an application, and starting the application to complete the initialization of the collection part;

[0009] The collection part collects log data in real time and sends the collected log data to JMQ;

[0010] The storage part obtains the log data from the JMQ and stores the log data in a storage layer according to a log storage algorithm, wherein the storage part includes a storage service and a data service.

[0011] In some embodiments, the collecting part collects log data in real time, including:

[0012] The business service initiates a log collection request, and the SDK of the corresponding collection service is started;

[0013] The task manager of the collection service receives the log collection request and generates task information according to the current thread and system time;

[0014] The task manager stores the task information in a task queue maintained by the task manager.

[0015] In some embodiments, the method further comprises:

[0016] The task manager polls the task queue and obtains the first thread from the task queue;

[0017] Determine whether the first thread ends. If the first thread does not end, continue to perform the polling operation. If the first thread ends, delete the first thread from the task queue and transfer the first thread to the task controller.

[0018] In some embodiments, the method further comprises:

[0019] The task controller maintains a target thread pool, and when the target thread pool is not full, puts the first thread into the target thread pool for execution.

[0020] In some embodiments, the method further comprises:

[0021] The task controller polls the task list and executes the log collection task;

[0022] Determine whether the log file corresponding to the current instance process is clear. If the log file corresponding to the current instance process is not clear, you need to loop the file list. If the log file corresponding to the current instance process is clear, read the log file.

[0023] In some embodiments, reading the log file includes:

[0024] Determine whether the time of the log file meets the time required for collection;

[0025] When the time required for collection is met, log data is read from the log file, and the offset after each reading is recorded and updated, and the position of the number of lines and length is recorded;

[0026] The read log data is stored.

[0027] In some embodiments, the log data is read from the log file using the RandomAccessFile tool under the java.io package.

[0028] In some embodiments, storing the log data in a storage layer according to a log storage algorithm includes:

[0029] After the storage part is started, the storage service of the storage part reads the log message in the JMQ and calculates the size of the log body;

[0030] If the length exceeds the first threshold, the file is stored using MongoDB's own GridFS. If the length does not exceed the first threshold, it is directly stored in the Mongo database table.

[0031] In a second aspect, the present application provides an electronic device, comprising: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements any one of the methods described above when executing the computer program.

[0032] In a third aspect, the present application provides a computer-readable storage medium, wherein the computer-readable storage medium stores one or more programs, and the one or more programs can be executed by one or more processors to implement any of the methods described above.

[0033] The IVVR traffic log collection and storage method and electronic device provided by the present application include a log collection part and a log storage part. The log collection part is embedded in the business service, and log collection can be achieved without the need to independently deploy a collection process, thereby reducing complexity and resource consumption. In addition, by introducing a message middleware, the log collection part is decoupled from the log storage part, reducing the dependency between systems, reducing storage write pressure, and improving efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0034] In order to more clearly illustrate the embodiments of the present application or the existing technical solutions, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments recorded in the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0035] Figure 1 This is a flow chart of a method for collecting and storing IVVR traffic logs in an embodiment of the present application;

[0036] Figure 2 This is a structural diagram of a method for collecting and storing IVVR traffic logs in an embodiment of the present application;

[0037] Figure 3 This is an overall business diagram of the IVVR traffic log collection and storage method in one embodiment of the present application;

[0038] Figure 4 This is a real-time collection service flow chart of the IVVR traffic log collection and storage method in one embodiment of the present application;

[0039] Figure 5 This is a historical collection service flow chart of the IVVR traffic log collection and storage method in one embodiment of the present application;

[0040] Figure 6 It is a schematic block diagram of an electronic device in an embodiment of the present application. DETAILED DESCRIPTION

[0041] In order to make the purpose, technical solution and advantages of the present application clearer, the technical solution of the present application will be clearly and completely described below in conjunction with specific embodiments and corresponding drawings. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in the field without creative work are within the scope of protection of the present application.

[0042] It should be noted that, unless otherwise defined, the technical terms or scientific terms used in one or more embodiments of the present application should be understood by people with ordinary skills in the field to which the present application belongs. The "first", "second" and similar words used in one or more embodiments of the present application do not indicate any order, quantity or importance, but are only used to distinguish different components. "Include" or "comprise" and similar words mean that the elements or objects appearing before the word cover the elements or objects listed after the word and their equivalents, without excluding other elements or objects. "Connect" or "connected" and similar words are not limited to physical or mechanical connections, but may include electrical connections, whether direct or indirect. "Up", "down", "left", "right" and the like are only used to indicate relative positional relationships. When the absolute position of the described object changes, the relative positional relationship may also change accordingly.

[0043] Among the related log management solutions, such as the ELK solution, although the solution is widely used, some shortcomings are also found in the process of selecting technical solutions based on current business. The following are some major shortcomings summarized in combination with the current intelligent audio and video business scenarios.

[0044] 1) High threshold for use: ELK is a complex system. For beginners, it requires a certain amount of learning cost, including knowledge of Elasticsearch data modeling, Logstash data collection and processing, and Kibana data visualization.

[0045] 2) High resource consumption: Logstash, as a log collection component, must be deployed and installed on each server that needs to collect logs. It occupies a high CPU and memory usage during operation, which consumes a lot of resources.

[0046] 3) High hardware cost: Since ELK needs to process a large amount of data, it requires corresponding hardware support, including high-speed disks, large memory, high-performance processors, etc., which will increase costs and complexity.

[0047] 4) Performance bottleneck: If the message queue cache is not used, Elasticsearch will have a performance bottleneck for the entire ELK due to its own insufficient write performance. Therefore, it is necessary to adopt the XX technical effect provided by this application.

[0048] Therefore, it is necessary to adopt the IVVR traffic log collection and storage method of the present application to solve the various problems of the above method.

[0049] Various non-limiting implementations of the present application are described in detail below in conjunction with the accompanying drawings.

[0050] First, refer to Figure 1 , the IVVR traffic log collection and storage method of the present application is described in detail. The present application provides an IVVR traffic log collection and storage method, the method comprising:

[0051] S1: Introduce a collection service into a business service, merge the business service and the collection service together as a collection part and generate an application, start the application to complete the initialization of the collection part;

[0052] S2: The collection part collects log data in real time and sends the collected log data to JMQ;

[0053] S3: The storage part obtains the log data from the JMQ and stores the log data in a storage layer according to a log storage algorithm, wherein the storage part includes a storage service and a data service.

[0054] Specifically, Figure 2As shown, the collection part consists of business services and collection services. The business service is a business support service unit, and the collection service is a log collection unit designed for this application. The collection service will be encapsulated as an SDK and embedded in the business service. It is responsible for receiving the log data generated by each logical unit in the business service, and the collected log data is written into the message middleware.

[0055] Specifically, the collection service includes a task manager, a task controller, a task executor, and a history collector.

[0056] The task manager performs task queue maintenance, thread status monitoring, and sending mobile phone tasks; the task controller performs collection queue maintenance, thread pool maintenance, and executes collection tasks; the task executor reads log files, caches data, and sends log messages; and the history collector performs Spring monitoring, task caching, and distributed locking.

[0057] like Figure 3 As shown, the history collector executes the history collection service, implements distributed locks and single-machine multi-instance reading, reads historical log file data, organizes the log data and sends it to JMQ.

[0058] Specifically, the task manager is mainly responsible for receiving the collection tasks issued by the business program. When the manager finds that the task can extract logs, it sends the task to the task controller, and the task controller is responsible for creating and executing the collection task thread.

[0059] For example, Figure 5 As shown in the figure, after the application is started, a distributed lock is added to the collection service. The purpose of adding the lock is to prevent multiple application instances in the same server from reading history at the same time. The distributed lock is marked as {IP}_history_lock.

[0060] After locking, loop through the unexecuted task queues cached in Redis and use Redis's pop method to get one until there is no data in the queue. The purpose of using the pop method is to get only one at a time. Although multiple requests to Redis are required, it can prevent the server from being shut down again. The cache in Redis is still the unexecuted task queue, rather than the original full queue data.

[0061] The history collector maintains two maps, FirstValidMap and LogCacheMap. FirstValidMap mainly caches the last valid log of each file to avoid reading it every time a task is executed. LogCacheMap caches log data because it is necessary to read the history file when reading the history data. There is no log written to the file, so it is cached to avoid repeated reading for each task.

[0062] After getting the unexecuted tasks from Redis, all file collections will be obtained. This file collection contains historical log files, such as catalina.out.2024-09-15xxx. The standard for obtaining the file collection needs to be based on the file list listed in the log.files configuration, obtain all the files in the directory where these files are located, and compare which files have been recently modified based on time, so as to obtain the historical log file collection.

[0063] Because multiple historical files are obtained, the file collection needs to be looped. For example, you can check whether the last valid log in FirstValidMap exists in the cache based on the file name. If not, you need to read it. The reading standard here is consistent with the real-time reading method of the last valid log, and add the last valid log to FirstValidMap.

[0064] If the valid log does not exceed the time, the log data will be matched when the last valid log meets the time search range. When matching log data, it is also necessary to first determine whether there is cached data in LogCacheMap and match whether these data meet the current task. If these data do not reach the maximum value and the log time has not ended, you can continue to read until the unreadable condition is met (time exceeds or exceeds the maximum value).

[0065] During the reading process, cached data must be appended for use in subsequent tasks. The difference between historical reading and real-time reading is that historical reading can cache data because new data will not be written to the historical file. The rest is similar to the real-time reading in the following text, so I will not go into details here.

[0066] When there is no data in the unfinished task queue in Redis, FirstValidMap and LogCacheMap are set to null and wait for GC. Finally, the distributed lock is released in finally to complete the historical reading.

[0067] Exemplarily, the storage part is composed of a storage service and a data service. The storage service receives the log information from the collection service, organizes and packages it according to relevant algorithms, and then persists the log data to the data service, which provides unified and centralized storage.

[0068] This application is based on Spring, uses Redis as cache, relies on the message middleware JMQ, and uses MongoDB and its own GridFS file storage service for log data storage.

[0069] Unlike related log collection solutions (for example, ELK solutions), the log collection part in the related solutions requires a separate deployment of the collection process. This application integrates the log collection function into an SDK format and embeds it into business services. Log collection can be achieved without the need to independently deploy a collection process, thereby reducing the complexity of the deployment architecture and the difficulty of system maintenance, and reducing resource consumption.

[0070] In addition, in the related solutions, the log collection part writes the collected logs to ES, and there will be performance bottlenecks in the write operation of ES. This application introduces the message middleware to decouple the log collection part from the storage part and reduce the dependency between systems. When facing sudden large traffic, the message queue can effectively cut the peak, reduce the storage write pressure, and avoid backlogs in the collection part.

[0071] For example, in this application, a collection service is introduced into a business service, and the two parts are packaged together to generate an application program, such as Figure 3 As shown, during the application startup process, the log collection service packaged in the application will also perform a series of initialization operations, including creating a collection thread pool, task management queue, etc. At the same time, it will detect the tasks that were not executed when the application was stopped last time from the cache, read information from the cache, and collect historical tasks in a single thread.

[0072] After initializing the collection service, you can receive tasks sent by the application's business service. Through the execution of the scheduled tasks in the collection service, the thread log described by the task will be read. After the relevant reading and matching operations, the collection service will send the collected logs to JMQ.

[0073] Exemplarily, the storage service consumes and parses the log data written to JMQ by the log collection service, and persists the data to the storage layer according to the log storage algorithm.

[0074] Specifically, when the application is closed, unexecuted tasks in the log collection service are recorded and put into the cache queue for historical collection the next time the application is started.

[0075] In some embodiments, the collecting part collects log data in real time, including:

[0076] The business service initiates a log collection request, and the SDK of the corresponding collection service is started;

[0077] The task manager of the collection service receives the log collection request and generates task information according to the current thread and system time;

[0078] The task manager stores the task information in a task queue maintained by the task manager.

[0079] like Figure 4 As shown, the business service initiates a log collection request, and the task manager polls the thread task queue. If the thread ends, it is transferred to the task controller. If the thread does not end, polling continues; the task controller polls the task list and starts to execute the log collection task. If the log file is not clear, the file list needs to be looped; if the log file is clear, N lines of logs in the log file are read, and the position is recorded, and the matching thread log is recorded. It is determined that the time exceeds or exceeds the maximum value. If it does not exceed or does not exceed the maximum value, the log continues to be read. If the reading time exceeds or exceeds the maximum value setting, this reading ends; continue to determine whether the log is collected. If the log is collected, the collection success MQ is sent and the clear log file is updated; if the log is not collected, it is determined again whether there is a clear log file. If so, the collection failure MQ is sent and the process ends.

[0080] For example, after the application is started, the business service will initiate a log collection request and then enter the log collection service SDK. After the task manager of the log collection service receives the log collection request from the business service, the task manager will generate task information based on parameters such as the current thread and system time, and place it in the task queue maintained by the task manager. The task queue is of CopyOnWriteArrayList type, which can effectively avoid read and write confusion problems.

[0081] In some embodiments, the method further comprises:

[0082] The task manager polls the task queue and obtains the first thread from the task queue;

[0083] Determine whether the first thread ends. If the first thread does not end, continue to perform the polling operation. If the first thread ends, delete the first thread from the task queue and transfer the first thread to the task controller.

[0084] Exemplarily, the task manager continuously polls the task queue, obtains thread information from the task queue, and determines whether the thread status has been terminated. If a task thread that needs to collect logs has been terminated, the thread is deleted from the queue and handed over to the task controller.

[0085] In some embodiments, the method further comprises:

[0086] The task controller maintains a target thread pool, and when the target thread pool is not full, puts the first thread into the target thread pool for execution.

[0087] In some embodiments, the method further comprises:

[0088] The task controller polls the task list and executes the log collection task;

[0089] Determine whether the log file corresponding to the current instance process is clear. If the log file corresponding to the current instance process is not clear, you need to loop the file list. If the log file corresponding to the current instance process is clear, read the log file.

[0090] Exemplarily, the task controller internally maintains a collection task queue of the CopyOnWriteArrayList type similar to the task manager, and a target thread pool with a fixed number of active threads. Its working mechanism is to poll the collection task queue to be executed, and put it into the target thread pool for execution if the thread pool is not full.

[0091] For example, the main purpose of adding a queue of tasks to be executed in the task controller instead of directly putting the thread into the target thread pool for execution is to avoid queue blocking when the target thread pool is full. Adding this queue of tasks to be executed can retain unexecuted tasks within a controllable range. When the application stops, the tasks to be executed can be obtained from the task queue and cached in Redis, with the KEY being {IP}_history_logThread_list, so that the historical log can be read next time.

[0092] For example, when the thread of the collection task starts to execute, it first checks whether the log file path of this instance has been configured. Multiple log files can be configured, and a server may deploy multiple instances. The content of this instance will not appear in the log files of other instances. Therefore, after hitting the log file of the current instance process, it is considered that the log file is the log file of the current instance. It can be directly read from the confirmed file later. Otherwise, it is necessary to loop through the configured file list.

[0093] Before the instance file is clear, you need to loop through the file list to get the last valid log at the current time (valid logs are non-abnormal stack logs, non-server container logs, and the format meets the log.format format).

[0094] After obtaining the valid log, determine whether the time of the log meets the required collection time. The purpose of this step is to avoid wasting resources. When the required collection time is met, read the data from the file backwards. The number of lines read each time is the realtime.next value in the configuration, and record and update the offset after each reading, record the number of lines and length, so that the next time you start reading backwards from this position. To read files backwards from a specified position, you can use the RandomAccessFile tool in the java.io package.

[0095] Specifically, each time the log data is read, it is necessary to match the log to see if it is the log that needs to be collected by the task. The matching standard is to format the log through log.format and determine whether the thread name or thread ID is equal. If it is found during the matching process that the time has exceeded the start time of the thread task, no more matching will be performed, and the collection will be exited without reading. Another point to end the collection task is to determine whether the latest offset has exceeded the configured maximum range after each reading. If it exceeds, it will not continue to read.

[0096] When the file of this instance is not clear, if the log of the task has been matched, you need to update the local log file variable so that you can read directly from the file next time, saving the time of determining the last valid log. The reading method is the same as the reading method above.

[0097] In some embodiments, reading the log file includes:

[0098] Determine whether the time of the log file meets the time required for collection;

[0099] When the time required for collection is met, log data is read from the log file, and the offset after each reading is recorded and updated, and the position of the number of lines and length is recorded;

[0100] The read log data is stored.

[0101] Exemplarily, after the collection task is completed, the collected logs are sorted and sent to the JMQ message middleware.

[0102] In some embodiments, the log data is read from the log file using the RandomAccessFile tool under the java.io package.

[0103] In some embodiments, storing the log data in a storage layer according to a log storage algorithm includes:

[0104] After the storage part is started, the storage service of the storage part reads the log message in the JMQ and calculates the size of the log body;

[0105] If the length exceeds the first threshold, the file is stored using MongoDB's own GridFS. If the length does not exceed the first threshold, it is directly stored in the Mongo database table.

[0106] For example, Figure 2As shown in the figure, the storage part includes storage service and data service. The storage service consists of log message parsing, log storage algorithm, read and write database, log retrieval and other functions. Through coding, the storage service will run as an independent application. After the storage service is started, it will connect to the JMQ message middleware. When the collection service writes the collected log data to the message middleware JMQ, the storage service will consume and organize the data in real time, and finally write the processed data to the MongoDB data service for storage.

[0107] Specifically, the main process of the storage service saving logs is as follows:

[0108] When the storage service is started, it starts to consume log messages in JMQ. First, the size of the log body will be calculated. If the length exceeds the first threshold (for example, N, where N is a positive integer, which can be set by technicians in this field according to the actual situation, and is not limited here), the file will be stored in MongoDB's own GridFS. Otherwise, it will be directly stored in the Mongo database table. The specific first threshold can be adjusted dynamically according to the actual situation and take effect in real time. Later, the business system can also specify which storage to use when sending logs.

[0109] After the above judgment results are obtained, the batch logs are saved according to the storage format. If the file is stored, it is stored in the file server GridFS and the records are saved (fs.chunks table). GridFS can also support online preview, which is very convenient for log viewing. If it is conventionally stored, the batch log details are added to the log details table (yb_log_infos table).

[0110] An ID can be generated through file storage. Whether it is file storage or regular storage, there will be a unique identifier, which is the ID of the log detail. At this time, the log detail ID and the log detail ID are stored together in the basic information table describing the batch log.

[0111] After collecting and storing the start and end of IVVR calls through the method of the present application, the overall IVVR call situation can be queried based on the calling number, called number, time range and other conditions, and the interaction record details can be queried based on the service number (callID), and the specific interaction situation under the same service number can be seen.

[0112] The IVVR traffic log collection and storage solution of this application achieves the following purposes:

[0113] 1) Lower the usage threshold: Customized development is carried out in combination with business scenarios, the complex configuration of ELK is abandoned, some core parameter configurations are retained, and the difficulty of use is reduced.

[0114] 2) Reduce resource consumption: Use plug-in design to encapsulate the log collection logic into an SDK and embed it into the business logic service of the application to avoid additional deployment and reduce hardware resource consumption; start and stop with the application to reduce maintenance costs.

[0115] 3) Solve performance bottlenecks: reuse the intelligent audio and video platform cache service Redis to save historical status, reuse the message queue JMQ to implement log collection buffering, reuse the data storage MongoDB to make full use of memory to read data, make full use of Bson format data, and easily expand the log structure. The built-in distributed file system GridFS can perfectly save logs, support the storage of massive documents and files, support disaster recovery, and facilitate backup. Overall, it ensures that there will be no performance bottlenecks in each link and there is no risk of data loss.

[0116] It should be noted that the method of one or more embodiments of the present application can be performed by a single device, such as a computer or server. The method of this embodiment can also be applied to a distributed scenario and completed by multiple devices cooperating with each other. In the case of such a distributed scenario, one of the multiple devices can only perform one or more steps in the method of one or more embodiments of the present application, and the multiple devices will interact with each other to complete the described method.

[0117] It should be noted that the above describes specific embodiments of the present application. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recorded in the claims can be performed in an order different from that in the embodiments and still achieve the desired results. In addition, the processes depicted in the accompanying drawings do not necessarily require the specific order or continuous order shown to achieve the desired results. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0118] Based on the same inventive concept, corresponding to any of the above-mentioned embodiments and methods, the present application also discloses an electronic device;

[0119] Specifically, Figure 6 The hardware structure diagram of an electronic device for collecting and storing an IVVR traffic log provided in this embodiment is shown, and the device may include: a processor 410, a memory 420, an input / output interface 430, a communication interface 440, and a bus 450. The processor 410, the memory 420, the input / output interface 430, and the communication interface 440 are connected to each other in communication within the device through the bus 450.

[0120] The processor 410 can be implemented by a general-purpose CPU (Central Processing Unit), a microprocessor, an application-specific integrated circuit (ASIC), or one or more integrated circuits, and is used to execute relevant programs to implement the technical solutions provided in the embodiments of the present application.

[0121] The memory 420 can be implemented in the form of ROM (Read Only Memory), RAM (Random Access Memory), static storage device, dynamic storage device, etc. The memory 420 can store an operating system and other application programs. When the technical solution provided in the embodiment of the present application is implemented by software or firmware, the relevant program code is stored in the memory 420 and called and executed by the processor 410.

[0122] The input / output interface 430 is used to connect the input / output module to realize information input and output. The input / output module can be configured in the device as a component (not shown in the figure), or it can be externally connected to the device to provide corresponding functions. The input device may include a keyboard, a mouse, a touch screen, a microphone, various sensors, etc., and the output device may include a display, a speaker, a vibrator, an indicator light, etc.

[0123] The communication interface 440 is used to connect a communication module (not shown) to realize communication interaction between the device and other devices. The communication module can realize communication through a wired mode (for example, USB, network cable, etc.) or a wireless mode (for example, mobile network, WIFI, Bluetooth, etc.).

[0124] The bus 450 includes a path that transmits information between the various components of the device (eg, the processor 410 , the memory 420 , the input / output interface 430 , and the communication interface 440 ).

[0125] It should be noted that, although the above device only shows the processor 410, the memory 420, the input / output interface 430, the communication interface 440 and the bus 450, in the specific implementation process, the device may also include other components necessary for normal operation. In addition, it can be understood by those skilled in the art that the above device may also only include the components necessary for implementing the embodiment of the present application, and does not necessarily include all the components shown in the figure.

[0126] The electronic device of the above embodiment is used to implement the corresponding IVVR traffic log collection and storage method in any of the above embodiments, and has the beneficial effects of the corresponding method embodiment, which will not be repeated here.

[0127] Based on the same inventive concept, corresponding to any of the above-mentioned embodiments and methods, one or more embodiments of the present application further provide a non-transitory computer-readable storage medium, wherein the non-transitory computer-readable storage medium stores computer instructions, and the computer instructions are used to enable the computer to execute the IVVR traffic log collection and storage method as described in any of the above embodiments.

[0128] The computer-readable medium of this embodiment includes permanent and non-permanent, removable and non-removable media, and information storage can be implemented by any method or technology. Information can be computer-readable instructions, data structures, modules of programs, or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, read-only compact disk read-only memory (CD-ROM), digital versatile disk (DVD) or other optical storage, magnetic cassettes, tape disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device.

[0129] The computer instructions stored in the storage medium of the above embodiment are used to enable the computer to execute the IVVR traffic log collection and storage method as described in any of the above embodiments, and have the beneficial effects of the corresponding method embodiments, which will not be repeated here.

[0130] Those skilled in the art should understand that the discussion of any of the above embodiments is merely illustrative and is not intended to imply that the scope of the present application (including the claims) is limited to these examples. In line with the concept of the present application, the technical features in the above embodiments or different embodiments may be combined, the steps may be implemented in any order, and there are many other variations of different aspects of one or more embodiments of the present application as described above, which are not provided in detail for the sake of simplicity.

[0131] In addition, to simplify the description and discussion, and in order not to make one or more embodiments of the present application difficult to understand, the known power / ground connections to the integrated circuit (IC) chip and other components may or may not be shown in the provided drawings. In addition, the device can be shown in the form of a block diagram to avoid making one or more embodiments of the present application difficult to understand, and this also takes into account the fact that the details of the implementation of these block diagram devices are highly dependent on the platform on which one or more embodiments of the present application will be implemented (that is, these details should be fully within the scope of understanding of those skilled in the art). In the case of elaborating specific details (e.g., circuits) to describe exemplary embodiments of the present application, it is obvious to those skilled in the art that one or more embodiments of the present application can be implemented without these specific details or when these specific details are changed. Therefore, these descriptions should be considered illustrative rather than restrictive.

[0132] Although the present application has been described in conjunction with specific embodiments of the present application, many replacements, modifications and variations of these embodiments will be apparent to those skilled in the art from the foregoing description. For example, other memory architectures (e.g., dynamic RAM (DRAM)) may use the embodiments discussed.

[0133] One or more embodiments of the present application are intended to cover all such substitutions, modifications and variations that fall within the broad scope of the appended claims. Therefore, any omissions, modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of one or more embodiments of the present application should be included in the scope of protection of the present application.

Claims

1. A method for collecting and storing IVVR traffic logs, characterized in that: The method comprises: Introducing a collection service into a business service, merging the business service and the collection service together as a collection part and generating an application, and starting the application to complete the initialization of the collection part; The collection part collects log data in real time and sends the collected log data to JMQ; The storage part obtains the log data from the JMQ and stores the log data in a storage layer according to a log storage algorithm, wherein the storage part includes a storage service and a data service.

2. The IVVR traffic log collection and storage method according to claim 1, characterized in that: The collection part collects log data in real time, including: The business service initiates a log collection request, and the SDK of the corresponding collection service is started; The task manager of the collection service receives the log collection request and generates task information according to the current thread and system time; The task manager stores the task information in a task queue maintained by the task manager.

3. The IVVR traffic log collection and storage method according to claim 2, characterized in that: The method further comprises: The task manager polls the task queue and obtains the first thread from the task queue; Determine whether the first thread ends. If the first thread does not end, continue to perform the polling operation. If the first thread ends, delete the first thread from the task queue and transfer the first thread to the task controller.

4. The IVVR traffic log collection and storage method according to claim 3, characterized in that: The method further comprises: The task controller maintains a target thread pool, and when the target thread pool is not full, puts the first thread into the target thread pool for execution.

5. The IVVR traffic log collection and storage method according to claim 4, characterized in that: The method further comprises: The task controller polls the task list and executes the log collection task; Determine whether the log file corresponding to the current instance process is clear. If the log file corresponding to the current instance process is not clear, you need to loop the file list. If the log file corresponding to the current instance process is clear, read the log file.

6. The IVVR traffic log collection and storage method according to claim 5, characterized in that: The reading of the log file comprises: Determine whether the time of the log file meets the time required for collection; When the time required for collection is met, log data is read from the log file, and the offset after each reading is recorded and updated, and the position of the number of lines and length is recorded; The read log data is stored.

7. The IVVR traffic log collection and storage method according to claim 6, characterized in that: Use the RandomAccessFile tool under the java.io package to read log data from the log file.

8. The IVVR traffic log collection and storage method according to claim 1, characterized in that: According to the log storage algorithm, the log data is stored in the storage layer, including: After the storage part is started, the storage service of the storage part reads the log message in the JMQ and calculates the size of the log body; If the length exceeds the first threshold, the file is stored using MongoDB's own GridFS. If the length does not exceed the first threshold, it is directly stored in the Mongo database table.

9. An electronic device, comprising: A memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the method according to any one of claims 1 to 8 when executing the computer program.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores one or more programs, and the one or more programs can be executed by one or more processors to implement the method according to any one of claims 1 to 8.

Images