Collection method of broadcast and television CM terminal log file

Abstract

According to the broadcast and television CM terminal log file collection method, a main control unit determines the collection length and the next collection time according to the maximum performance ratio P, after the to-be-collected rate is uploaded through the CM terminal, the priority level of the CM terminal is increased, the position of a CM information unit of the CM terminal in a collection queue is updated according to the collected priority level, and the collection time of the CM terminal is shortened. And waiting for the next collection, and collecting the log files to be collected of the corresponding CM terminals according to the determined collection length and the sequence of the CM information units in the collection queue, thereby dynamically lengthening or shortening the collection length required by each collection and the next collection time, and enabling the number of CM terminals required to be collected each time in the collection queue list to be adaptively adjusted. And the time length of each collection is adaptively adjusted, so that the performance of the broadcast and television operation system is more reasonably and fully utilized, the collection of the CM terminal log file is smoother and more comprehensive, the blockage and the leaving are avoided, and the service quality of the broadcast and television is improved.

Description

Technical Field

[0001] The present invention relates to the field of broadcasting and television CM terminal data collection, in particular to a method for collecting log files of broadcasting and television CM terminals. Background Art

[0002] With the development of the application scale of two-way radio and television services, user behavior analysis has become a new research hotspot. CM (Cable Modem) is an important device for radio and television users to connect to access services. Each CM terminal connected to the radio and television operation network and operation system records the corresponding user's real-time behavior records and the operation status information of the device and generates a log file. In the prior art, when the log file entries of the CM terminal connected to the radio and television operation network and operation system meet the set number, each CM terminal uploads it to the log collection front-end server by itself. Since each CM terminal uploads it to the log collection front-end server through the radio and television operation system according to its own strategy, the upload process belongs to a disorderly management state. The usage behavior habits of radio and television users are in a tidal state. Usually 6:30-9:00, 11:30-13:00, and 15:00-22:30 are a tidal peak. When the tide comes, the CM terminal will generate a large number of logs. If each CM terminal uploads log files according to its own strategy during the tidal peak, there will be problems such as insufficient performance of the radio and television operation system due to uploading too many log files, affecting the quality of radio and television services. Summary of the invention

[0003] The object of the present invention is to provide a method for dynamically adjusting the number of log files collected by a radio and television CM terminal according to the resources of a radio and television operation system, so as to reduce the performance pressure of the radio and television operation system and improve the service quality.

[0004] The method for collecting log files of a radio and television CM terminal of the present invention comprises the following steps:

[0005] The main control unit sets a collection queue, and collects the log files to be collected of the CM terminal in the order of the collection queue arrangement; the CM information units are arranged in the collection queue, and the CM information units include the MAC (the unique physical address of the CM network card) of the corresponding CM, the access time and the collection priority level; the arrangement order is arranged in the order of the collection priority level, with the higher one at the front and the lower one at the back. In the case of the same collection priority level, the access time is arranged in the order of the access time, with the earlier one at the front and the later one at the back;

[0006] The CM terminal generates log records and writes the generated log records into a log file. When the number of log records reaches a set threshold, it is saved as a log file to be collected, and a new log file is generated for writing new log records.

[0007] The main control unit presets a performance adjustment coefficient M that reflects the corresponding relationship between the performance ratio P of the radio and television operation system equipment and the collection length L and the collection time T; the main control unit obtains the maximum performance ratio P of the equipment of the radio and television operation system, and dynamically adjusts the CM terminal collection length L in the collection queue and the next CM terminal collection time T according to the preset performance adjustment coefficient M based on the maximum performance ratio P;

[0008] The main control unit collects the log files to be collected of the corresponding CM terminal according to the determined collection length and the order of the CM information units in the collection queue, and marks the collected log files to be collected on the CM terminal as collected;

[0009] The CM terminal calculates the waiting rate of the log files to be collected according to the collection status of the log files to be collected, wherein the waiting rate = the number of log files to be collected / the maximum number of log files to be stored;

[0010] The CM terminal sets a threshold value for the waiting rate. When the waiting rate exceeds the threshold, the CM terminal uploads the waiting rate to the main control unit.

[0011] After receiving the waiting rate uploaded by the CM terminal, the main control unit raises the priority level of the CM terminal, and updates the position of its CM information unit in the collection queue according to the collection priority level, waiting for the next collection.

[0012] In the method for collecting log files of radio and television CM terminals of the present invention, the main control unit determines the collection length and the next collection time according to the maximum performance ratio P, and after the waiting collection rate is uploaded by the CM terminal, the priority level of the CM terminal can be increased, and the position of its CM information unit in the collection queue can be updated according to the collection priority level, waiting for the next collection, and according to the determined collection length, the waiting collection log files of the corresponding CM terminal are collected in the order of the CM information units in the collection queue, which can simultaneously achieve the extension or shortening of the collection length and the next collection time required for each collection, so that the number of CM terminals required to be collected each time in the collection queue list is The amount of data collected is adjusted adaptively, and the length of each collection time is also adjusted adaptively, so that the performance of the radio and television operation system can be more reasonably and fully utilized, and the collection of CM terminal log files can be smoother and more comprehensive, avoiding blockage and legacy, thereby improving the service quality of radio and television; in addition, because the reporting of the waiting rate is determined by whether it exceeds the waiting rate threshold, it is reported only when it exceeds, thus avoiding the CM terminal from frequently reporting the waiting rate to the main control unit, avoiding the main control unit adjusting the priority order of the collection queue due to the reporting of those waiting rates that are not urgently processed, reducing the pressure on the main control unit's operating performance, and making the main control unit's operating performance more reasonably and fully utilized.

[0013] As a preferred solution of the present invention, the main control unit presets a correspondence between the collection priority level and the waiting rate of the CM terminal, and different priority levels correspond to different waiting rate values, and the higher the waiting rate value, the higher the priority level; after receiving the waiting rate uploaded by the CM terminal, the collection priority level of the CM terminal is determined according to the corresponding value of the waiting rate, and the position of its CM information unit in the collection queue is updated according to the collection priority level, waiting for the next collection.

[0014] As a preferred solution of the present invention, the method for obtaining the performance adjustment coefficient M is as follows:

[0015] Obtain historical data on the performance ratio P of CPU, memory, network traffic and disk IO of each role node in the radio and television operation network operation system;

[0016] Obtain historical data of the CM terminal collection length L and the next CM terminal collection time T of the main control unit;

[0017] Through neural network The historical data of the performance ratio P, the CM terminal acquisition length L and the next CM terminal acquisition time T are analyzed to obtain the performance adjustment coefficient M of the relationship between the optimal output CM terminal acquisition length L of the performance ratio P and the next CM terminal acquisition time T at a certain time t; wherein f represents the neural network function, t represents a certain time, n represents n numbers, n is an integer, i represents the i-th number among the n numbers, i starts from 1 and is an integer, and θ is the adjustment parameter of the neural network function. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] Figure 1 A schematic diagram of a flow chart of a method for collecting log files of a radio and television CM terminal according to the present invention;

[0019] Figure 2 Schematic diagram of the method for collecting log files of a radio and television CM terminal of the present invention

[0020] Figure 3 This is a schematic diagram of the collection team list;

[0021] Figure 4 A schematic diagram of a collection queue table with priority adjustment;

[0022] Figure 5 This is a schematic diagram of the log file waiting rate of the CM terminal at each moment;

[0023] Figure 6 Schematic diagram of dynamically adjusting the collection queue priority based on the CM terminal waiting rate. DETAILED DESCRIPTION

[0024] The technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, rather than all the embodiments.

[0025] A method for collecting log files of radio and television CM terminals, such as Figure 1 and Figure 2 As shown, the following steps are included:

[0026] S1. The main control unit sets a collection queue, and collects the log files to be collected from the CM terminal in the order of the collection queue arrangement; arranges CM information units in the collection queue, and the CM information units include the corresponding CM's MAC, access time and collection priority level; the arrangement order is arranged in the order of the collection priority level, with the higher one at the front and the lower one at the back. In the case of the same collection priority level, the order is arranged in the order of the access time, with the earlier one at the front and the later one at the back. The collection queue table is as follows Figure 3 shown.

[0027] S2. The CM terminal generates log records and writes the generated log records into a log file. When the number of log records reaches a set threshold, the log files are saved as log files to be collected, and a new log file is generated for writing new log records.

[0028] For example, if the threshold of the number of log records is set to 50, each log file will only keep 50 log records. Log files are named by the current year, current month, current day, current hour, file sequence number, and starting number. For example, the log file is named yymmddhhxxxx.log, where yy represents the current year, mm represents the current month, dd represents the current day, hh represents the current hour, xxxx represents the file sequence number, and the starting number is 0000. Each log file is sorted by the current year, current month, current day, current hour, and file sequence number. By splitting into several log files, it can be ensured that when the CM terminal system crashes unexpectedly, only the current log file is damaged, and the previously segmented log files are not affected, thereby improving the security of log file storage.

[0029] S3. The main control unit presets a performance adjustment coefficient M that reflects the corresponding relationship between the performance ratio P of the radio and television operation system equipment and the collection length L and the collection time T; the main control unit obtains the maximum performance ratio P of the equipment of the radio and television operation system, and dynamically adjusts the CM terminal collection length L in the collection queue and the next CM terminal collection time T according to the preset performance adjustment coefficient M based on the maximum performance ratio P.

[0030] The method for obtaining the performance adjustment coefficient M is as follows: obtaining the historical data of the performance ratio P of the CPU, memory, network traffic and disk IO of each role node in the radio and television operation network operation system; obtaining the historical data of the CM terminal acquisition length L and the next CM terminal acquisition time T of the main control unit; and then using the neural network to calculate the performance adjustment coefficient M. The historical data of the performance ratio P, the CM terminal acquisition length L and the next CM terminal acquisition time T are analyzed to obtain the performance adjustment coefficient M of the relationship between the optimal output CM terminal acquisition length L of the performance ratio P and the next CM terminal acquisition time T at a certain time t; wherein f represents the neural network function, t represents a certain time, n represents n numbers, n is an integer, i represents the i-th number among the n numbers, i starts from 1 and is an integer, and θ is the adjustment parameter of the neural network function.

[0031] The method for obtaining the performance adjustment coefficient M can also be based on the historical data of the performance ratio P of the CPU, memory, network traffic and disk IO of each role node in the radio and television operation network operation system and the historical data of the CM terminal collection length L of the main control unit and the next CM terminal collection time T, and summarize the performance adjustment coefficient M of the relationship between the optimal output CM terminal collection length L of the performance ratio P at a certain time t and the next CM terminal collection time T through experience.

[0032] Through the historical data of the performance ratio P of the CPU, memory, network traffic and disk IO of each role node in the radio and television operation network operation system and the historical data of the CM terminal collection length L and the next CM terminal collection time T of the main control unit, the neural network function determines the performance adjustment coefficient M of the relationship between the optimal output CM terminal collection length L of the performance ratio P at a certain time t and the next CM terminal collection time T, and then the performance adjustment coefficient M and the maximum performance ratio P are used to dynamically adjust the CM terminal collection length L and the next CM terminal collection time T in the collection queue, and then the performance adjustment coefficient M is dynamically adjusted by the CM terminal collection length L and the next CM terminal collection time T, so as to form a closed-loop dynamic adjustment loop, so that the performance of the radio and television operation system can be more reasonably and fully utilized, and the collection of CM terminal log files is smoother and more comprehensive, avoiding congestion and legacy, thereby improving the service quality of radio and television.

[0033] S4, the main control unit collects the log files to be collected of the corresponding CM terminal according to the determined collection length and the order of the CM information units in the collection queue, and marks the collected log files to be collected on the CM terminal as collected;

[0034] S5. The CM terminal calculates the waiting rate of the log files to be collected according to the collection status of the log files to be collected, wherein the waiting rate = the number of log files to be collected / the maximum number of log files to be stored;

[0035] S6. The CM terminal sets a threshold value of the waiting rate. When the waiting rate exceeds the threshold, the CM terminal uploads the waiting rate to the main control unit.

[0036] S7. After receiving the waiting rate uploaded by the CM terminal, the main control unit raises the priority level of the CM terminal and updates the position of its CM information unit in the collection queue according to the collection priority level, waiting for the next collection.

[0037] like Figure 4 In the first row of the collection queue list, CM terminal device-1, time queue 202407020505001 shows that it was added to the collection queue list at 05:00 on July 2, 2024. The time sequence 2024070305002 of the second row of CM terminal shows that it was added to the collection queue list at 03:00 on July 2, 2024. The time of device-1 is later than that of device-2. Logically, device-1 should be ranked behind device-2, but because the collection priority of device-1 is adjusted to 10, which is higher than the collection priority of device-2, device-1 is ranked ahead of device-2. In addition, when collecting log files of CM terminals, when the priorities of CM terminals are equal, the log files corresponding to the CM terminals with earlier time are collected first.

[0038] The corresponding relationship between the collection priority level preset by the main control unit and the waiting rate of the CM terminal, different priority levels correspond to different waiting rate values, and the higher the waiting rate value, the higher the priority level; after receiving the waiting rate uploaded by the CM terminal, the collection priority level of the CM terminal is determined according to the corresponding value of the waiting rate, and the position of its CM information unit in the collection queue is updated according to the collection priority level, waiting for the next collection. Figure 5 As shown in the figure, the changes in the number of log files written to the CM terminal log record, time is a certain moment of the CM terminal, lr is the number of log files collected by the CM terminal, lw is the number of log files not collected by the CM terminal, tl is the maximum number of logs stored in the CM terminal, ap is the waiting rate, and the waiting rate is exist Figure 6 The figure describes the change in the number of log files of a CM terminal. When the waiting rate threshold is set to 60, the waiting rate exceeds 60, and the CM terminal reports to the main control unit to collect data. However, the system is busy at t5 and t6 and no data can be collected. At t7, the waiting rate has reached a very high value of 90, so Figure 6The collection priority of the CM terminal is adjusted to a higher level in t8, and it returns to its normal state after being collected. By determining the collection priority of the CM terminal corresponding to the waiting rate, the more urgent CM terminal log files can be collected first, further making more reasonable and full use of the performance of the radio and television operation system, and the collection of CM terminal log files is smoother and more comprehensive, avoiding blockage and legacy, thereby improving the service quality of radio and television.

[0039] The main control unit obtains the maximum performance ratio P of the equipment in the radio and television operation system, and dynamically adjusts the CM terminal acquisition length L in the acquisition queue and the next CM terminal acquisition time T according to the preset performance adjustment coefficient M based on the maximum performance ratio P. The calculation formula for the next acquisition time is T = M(t)P, where M(t) is the performance adjustment coefficient M output as the acquisition time T, and P is the maximum performance ratio of the radio and television operation system equipment. The calculation formula for the maximum performance ratio of the radio and television operation system equipment is P = max(p1,…p n ), p1 represents the ratio of the first device in the current radio and television operation system, p n is the ratio of the nth device in the current radio and television operation system. The above formula can be used to calculate the ratio P of the maximum performance of the radio and television operation system at the current moment. The calculation formula for the acquisition length is L = M(l)P, where L represents the acquisition length of the CM terminal in the acquisition queue table, M(l) is the performance adjustment coefficient M output as the acquisition length L, and P is the ratio of the maximum performance of the radio and television operation system equipment.

[0040] By dynamically adjusting the collection length L and the next collection time T of the CM terminal in the collection queue list according to the maximum performance ratio P of the radio and television operation system equipment, the performance of the radio and television operation system can be more reasonably and fully utilized, and the collection of CM terminal log files can be smoother and more comprehensive, avoiding congestion and legacy, thereby improving the service quality of radio and television.

[0041] The above embodiments are only used to illustrate the detailed scheme of the present invention, and the present invention is not limited to the above detailed scheme, that is, it does not mean that the present invention must rely on the above detailed scheme to be implemented. Those skilled in the art should understand that any improvement of the present invention, equivalent replacement of various raw materials of the product of the present invention, addition of auxiliary components, selection of specific methods, etc., all fall within the protection scope and disclosure scope of the present invention.

Claims

1. A method for collecting log files of a radio and television CM terminal, characterized in that: The following steps are involved: The main control unit sets a collection queue, and collects the log files to be collected of the CM terminal in the order of the collection queue arrangement; arranges CM information units in the collection queue, and the CM information units include the MAC of the corresponding CM, the access time and the collection priority level; the arrangement order is arranged in the order of the collection priority level, with the higher one at the front and the lower one at the back. In the case of the same collection priority level, the access time is arranged in the order of the access time, with the earlier one at the front and the later one at the back; The CM terminal generates log records and writes the generated log records into a log file. When the number of log records reaches a set threshold, it is saved as a log file to be collected, and a new log file is generated for writing new log records. The main control unit presets a performance adjustment coefficient M that reflects the corresponding relationship between the performance ratio P of the radio and television operation system equipment and the acquisition length L and the acquisition time T; The main control unit obtains the maximum performance ratio P of the equipment of the radio and television operation system, and dynamically adjusts the CM terminal collection length L in the collection queue and the next CM terminal collection time T according to the maximum performance ratio P and the preset performance adjustment coefficient M; The main control unit collects the log files to be collected of the corresponding CM terminal according to the determined collection length and the order of the CM information units in the collection queue, and marks the collected log files to be collected on the CM terminal as collected; The CM terminal calculates the waiting rate of the log files to be collected according to the collection status of the log files to be collected, wherein the waiting rate = the number of log files to be collected / the maximum number of log files to be stored; The CM terminal sets a threshold value for the waiting rate. When the waiting rate exceeds the threshold, the CM terminal uploads the waiting rate to the main control unit. After receiving the waiting rate uploaded by the CM terminal, the main control unit raises the priority level of the CM terminal, and updates the position of its CM information unit in the collection queue according to the collection priority level, waiting for the next collection.

2. The method for collecting log files of a radio and television CM terminal according to claim 1, characterized in that: The correspondence between the collection priority level preset by the main control unit and the waiting rate of the CM terminal, different priority levels correspond to different waiting rate values, and the higher the waiting rate value, the higher the priority level; after receiving the waiting rate uploaded by the CM terminal, the collection priority level of the CM terminal is determined according to the corresponding value of the waiting rate, and the position of its CM information unit in the collection queue is updated according to the collection priority level, waiting for the next collection.

3. The method for collecting log files of a radio and television CM terminal according to claim 1, characterized in that: The method for obtaining the performance adjustment coefficient M is as follows: Obtain historical data on the performance ratio P of CPU, memory, network traffic and disk IO of each role node in the radio and television operation network operation system; Obtain historical data of the CM terminal collection length L and the next CM terminal collection time T of the main control unit; Through neural network The historical data of the performance ratio P, the CM terminal acquisition length L and the next CM terminal acquisition time T are analyzed to obtain the performance adjustment coefficient M of the relationship between the optimal output CM terminal acquisition length L of the performance ratio P and the next CM terminal acquisition time T at a certain time t; wherein f represents the neural network function, t represents a certain time, n represents n numbers, n is an integer, i represents the i-th number among the n numbers, i starts from 1 and is an integer, and θ is the adjustment parameter of the neural network function.

Images