A method for determining abnormal data source, storage medium and electronic device
By determining the reliability parameters of the data source on the server and dynamically adjusting the threshold in combination with weights and time intervals, the problem of normal data sources being misjudged as abnormal is solved, improving the accuracy of abnormal data sources and user experience.
Patent Information
- Application Number
- CN202310439085.2
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2023-04-23
- Publication Date
- 2024-11-08
- Estimated Expiration
- 2043-04-23
AI Technical Summary
In existing technologies, due to factors such as network fluctuations, even normal data sources may experience high failure rates within a short period of time, resulting in lower accuracy of abnormal data sources.
By obtaining reliability parameters of data failure rate from the server over a longer period of time, and combining them with weights and time intervals, the threshold is dynamically adjusted to determine whether the data source is an abnormal data source.
It improves the accuracy of identifying abnormal data sources, reduces the possibility of normal data sources being misjudged as abnormal, and enhances user experience and the success rate of data acquisition.
Smart Images

Figure CN116610562B_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of data processing, and in particular to a method for determining an abnormal data source, a storage medium and an electronic device. Background Art
[0002] At present, when determining an abnormal data source, it is usually determined whether the data source is an abnormal data source based on the failure rate of obtaining data from the data source during the test period at the end of the test period or after the test period. If the failure rate of obtaining data from the data source during the test period is high, the data source is an abnormal data source. In order to ensure that the data source can be determined as an abnormal data source after the abnormal state lasts for a short period of time, a shorter test period is generally used.
[0003] However, due to the influence of network fluctuations and other conditions, normal data sources also have a high failure rate in obtaining data from them within a short period of time. Therefore, it is less accurate to determine a data source as an abnormal data source based on the high failure rate in obtaining data from the data source during the test period, that is, the accuracy of determining the abnormal data source is low. Summary of the invention
[0004] In view of the above technical problems, the technical solution adopted by the present invention is:
[0005] According to one aspect of the present invention, a method for determining an abnormal data source is provided, and the method is applied to a server; the server is connected to a target data source, and the server is connected to the target data source through a plurality of data interfaces of the server; the server is used to send a plurality of corresponding data acquisition requests to the target data source through each data interface; the target data source is configured to, in response to receiving each data acquisition request, return acquired data corresponding to the data acquisition request or a target failure identifier to the server through the corresponding data interface;
[0006] The determination method comprises the following steps:
[0007] S100, in response to reaching the first time mom1, obtaining the target time mom0=mom1, and entering step S200.
[0008] S200, obtain the target failure identification list BS corresponding to the target data source = (bs1, bs2, ..., bs i ,...,bs n ), bs i =(bs i1 ,bs i2 ,...,bs ij ,...,bs im(i)), i = 1, 2, ..., n, j = 1, 2, ..., m(i); where bs i is the target failure identification group corresponding to the i-th data interface, n is the number of data interfaces; bs ij is the j-th target failure identifier returned by the target data source to the server through the i-th data interface between mom2 and mom1, mom2 is the second time, mom2<mom1; m(i) is the number of target failure identifiers returned by the target data source to the server through the i-th data interface between mom2 and mom1.
[0009] S300, according to BS, obtain the first reliability parameter par of the target data source = ∑ i=1 n [w i *s(i) / v(i)];w i is the weight corresponding to the i-th data interface, s(i) is bs i The number of target failure identifiers that are the same as any target preset failure identifier in , v(i) is the number of data acquisition requests sent by the server to the target data source through the i-th data interface between mom2 and mom1.
[0010] S400, determine whether par is greater than or equal to pre1; if so, determine the target data source as an abnormal data source; otherwise, if mom1=mom0, proceed to step S500; pre1 is the first target threshold.
[0011] S500, determine whether par is greater than or equal to pre2; if so, proceed to step S600; pre2 is the second target threshold, pre2<pre1.
[0012] S600, in response to reaching the third time mom3, mom1 is updated to mom3, pre1 is updated to γ*pre1, and the process goes to step S200; mom3>mom0, γ is the reduction parameter, γ={[(mom0-mom2) / (mom3-mom2)]*(v 1 / v 2 )*H}^(1 / 3);v 1 is the number of data acquisition requests sent by the server to the target data source between mom2 and mom0, v 1 =∑ i=1 n v(i);v 2 is the number of data acquisition requests sent by the server to the target data source between mom2 and mom3; H is the influence coefficient. ave ≤dur0, then H=(dur0-dur ave ) / dur0, if durave >dur0, then H=0; dur0 is the preset response time; dur ave is the reference response time, dur ave =∑ L=1 v(i) dur L ,dur L It is the difference between the time when the server sends the Lth data acquisition request to the target data source between mom2 and mom3 and the time when the server receives the acquired data or the target failure identifier corresponding to the data acquisition request.
[0013] According to another aspect of the present invention, a non-transitory computer-readable storage medium is provided, in which at least one instruction or at least one program is stored, and the at least one instruction or at least one program is loaded and executed by a processor to implement the above-mentioned method for determining the abnormal data source.
[0014] According to another aspect of the present invention, there is also provided an electronic device, comprising a processor and the above-mentioned non-transitory computer-readable storage medium.
[0015] The present invention has at least the following beneficial effects:
[0016] In the present invention, the first reliability parameter par of the target data source is determined by the failure rate of the server in acquiring data from the target data source through the i-th data interface between mom2 and mom0, and the target data source is determined as an abnormal data source when par is greater than or equal to pre1. When par is less than pre1 and par is greater than or equal to pre2, the first reliability parameter par of the target data source is updated according to the failure rate of the server in acquiring data from the target data source through the i-th data interface between mom2 and mom3, and the target data source is determined as an abnormal data source when the updated par is greater than or equal to the updated pre1.
[0017] Compared with the related art in which the target data source is determined as an abnormal data source only based on the high failure rate of obtaining data from the target data source within a shorter test time period, in the present invention, when par is less than pre1 and par is greater than or equal to pre2, whether the target data source is an abnormal data source is determined based on the failure rate of obtaining data from the target data source between mom2 and mom3, that is, when par is larger, whether the target data source is an abnormal data source is determined based on a longer test time period, thereby reducing the possibility of determining a normally working target data source as an abnormal data source due to network fluctuations and the like, thereby achieving the purpose of improving the accuracy of determining abnormal data sources.
[0018] In addition, in the present invention, when determining whether the target data source is an abnormal data source based on mom2 and mom3, pre1 at this time is lower than pre1 based on mom2 and mom0 to determine whether the target data source is an abnormal data source. Since the target data source will be determined as an abnormal data source when par is greater than or equal to pre1, the present invention can reduce the influence of the low par caused by the large time interval between mom2 and mom3 on the determination of the abnormal data source when determining whether the target data source is an abnormal data source based on mom2 and mom3, thereby reducing the possibility of a normally operating target data source being determined as an abnormal data source, thereby achieving the purpose of further improving the accuracy of determining the abnormal data source.
[0019] Furthermore, the pre1 in the present invention when determining whether the target data source is an abnormal data source based on mom2 and mom3 not only takes into account the time interval between mom0 and mom2, the time interval between mom3 and mom2, the number of data acquisition requests sent by the server to the target data source between mom2 and mom0, and the number of requests v. 1 , the number of data acquisition requests sent by the server to the target data source between mom2 and mom3 v 2 , and also takes into account the difference between the time when the server sends the Lth data acquisition request to the target data source between mom2 and mom3 and the time when the server receives the acquired data or the target failure identifier corresponding to the data acquisition request. Therefore, the longer the response time of the target data source between mom2 and mom3, the smaller the pre1 is when determining whether the target data source is an abnormal data source based on mom2 and mom3. At this time, the target data source is more likely to be determined as an abnormal data source, that is, in the present invention, a target data source with too long a response time can be determined as an abnormal data source, which can improve the user experience when the user uses a target data source that works normally. BRIEF DESCRIPTION OF THE DRAWINGS
[0020] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.
[0021] Figure 1 A flowchart of a method for determining an abnormal data source provided by an embodiment of the present invention. DETAILED DESCRIPTION
[0022] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative work are within the scope of protection of the present invention.
[0023] An embodiment of the present invention provides a method for determining an abnormal data source, which is applied to a server, wherein the server is connected to a target data source, and the server is connected to the target data source through a plurality of data interfaces of the server; the server is used to send a corresponding plurality of data acquisition requests to the target data source through each data interface; the target data source is configured to, in response to receiving each data acquisition request, return the acquired data or a target failure identifier corresponding to the data acquisition request to the server through the corresponding data interface.
[0024] Specifically, the data interface is an API (Application Programming Interface) interface. The target failure identifier is an error code.
[0025] In a specific implementation, the server is an airline server, and any two data interfaces correspond to different types of services. The server is used to respond to the user's operation corresponding to each data interface and send a corresponding data acquisition request to the target data source through the data interface. For example, there are 4 data interfaces, and the 4 data interfaces are respectively the first data interface, the second data interface, the third data interface and the fourth data interface. The first data interface is a data interface corresponding to the ticketing service, the second data interface is a data interface corresponding to the query service, the third data interface is a data interface corresponding to the refund service, and the fourth data interface is a data interface corresponding to the ticket change service. Correspondingly, the server sends a corresponding data acquisition request to the target data source through the first data interface in response to the user's ticket purchase operation, the server sends a corresponding data acquisition request to the target data source through the second data interface in response to the user's query operation, the server sends a corresponding data acquisition request to the target data source through the third data interface in response to the user's ticket refund operation, and the server sends a corresponding data acquisition request to the target data source through the fourth data interface in response to the user's ticket change operation.
[0026] The following will refer to Figure 1 The flowchart of the method for determining an abnormal data source is shown, and the method for determining an abnormal data source is introduced.
[0027] The determination method comprises the following steps:
[0028] S100 , in response to reaching the first time mom1 , obtaining the target time mom0 = mom1 .
[0029] In a specific implementation manner, the end time of each preset sliding time window among a plurality of preset sliding time windows is taken as mom1, and the determination method of the embodiment of the present invention can be executed for any mom1.
[0030] S200, obtain the target failure identification list BS corresponding to the target data source = (bs1, bs2, ..., bs i ,...,bs n ),
[0031] bs i =(bs i1 ,bs i2 ,...,bs ij ,...,bs im(i) ), i=1,2,...,n, j=1,2,...,m(i).
[0032] Among them, bs i is the target failure identification group corresponding to the i-th data interface, n is the number of data interfaces; bs ij is the j-th target failure identifier returned by the target data source to the server through the i-th data interface between mom2 and mom1, mom2 is the second time, mom2<mom1; m(i) is the number of target failure identifiers returned by the target data source to the server through the i-th data interface between mom2 and mom1.
[0033] S300, according to BS, obtain the first reliability parameter par of the target data source = ∑ i=1 n [w i *s(i) / v(i)].
[0034] Among them, w i is the weight corresponding to the i-th data interface, s(i) is bs i The number of target failure identifiers that are the same as any target preset failure identifier in , v(i) is the number of data acquisition requests sent by the server to the target data source through the i-th data interface between mom2 and mom1.
[0035] Specifically, s(i) / v(i) is the failure rate of the server in acquiring data from the target data source through the i-th data interface between mom2 and mom1.
[0036] Specifically, the server stores a first preset failure information list A 1 =(a1 1 ,a2 1 ,...,a k1 ,...,a z 1 ), a k 1 =(a k1 1 ,a k2 1 ,...,a kx 1 ,...,a ky(k) 1 ), a kx 1 =(bs kx 1 ,ms kx 1 ), k = 1, 2, ..., z, x = 1, 2, ..., y (k); where a k 1 is the kth second preset failure flag c k a corresponding first preset failure information group, z is the number of second preset failure identifiers, the second preset failure identifiers are generated by the server, and each second preset failure identifier belongs to a critical category or a non-critical category; kx 1 c k The corresponding x-th first preset failure information, y(k) is c k The number of corresponding first preset failure information; bs kx 1 for a kx 1 The corresponding first preset failure flag, ms kx 1 for bs kx 1 The corresponding preset interpretation information; the first preset failure information is generated by the target data source; A 1 Any two first preset failure identifiers in are different.
[0037] The first preset failure flag and the second preset failure flag are both error codes, and the preset interpretation information is the error description text of the corresponding first preset failure flag. For example, if any first preset failure flag is 0044, the preset interpretation information corresponding to the first preset failure flag is "bad network status".
[0038] Each second preset failure identifier has a corresponding failure level. The second preset failure identifier whose failure level is greater than or equal to the preset level belongs to the critical category, and the second preset failure identifier whose failure level is less than the preset level belongs to the non-critical category. The specific failure level and preset level are set in this field according to the application scenario, and the embodiments of the present invention are not limited to this.
[0039] The higher the failure level, the greater the possibility that the failure indicated by the corresponding second preset failure identifier is caused by an abnormality in the target data source. For example, the first failure cause is a network failure, and the second failure cause is a damage to the interface on the target data source used to transmit the acquired data corresponding to the data acquisition request to the server. Based on this, the failure level corresponding to the second preset failure identifier corresponding to the first failure cause is higher than the second preset failure identifier corresponding to the second failure cause.
[0040] Each second preset failure identifier has a corresponding number of interpretation text data, and the interpretation text data is the error description text of the corresponding second preset failure identifier. Based on this, among all the first preset failure identifiers, if the semantic matching degree of the preset interpretation information corresponding to any first preset failure identifier and at least one interpretation text data corresponding to any second preset failure identifier exceeds the preset matching degree threshold, then it is determined that the first preset failure identifier corresponds to the second preset failure identifier; among all the first preset failure identifiers, if the semantic matching degree of the preset interpretation information corresponding to any first preset failure identifier and at least one interpretation text data corresponding to multiple second preset failure identifiers exceeds the preset matching degree threshold, then the second preset failure identifier corresponding to the interpretation text data with the greatest semantic matching degree with the preset interpretation information corresponding to the first preset failure identifier is determined to correspond to the first preset failure identifier.
[0041] Based on this, several methods for obtaining target preset failure flags include:
[0042] A 1 Each first preset failure identifier in the first preset failure information group corresponding to each second preset failure information belonging to the key category is used as a target preset failure identifier.
[0043] In a specific embodiment, w i is preset.
[0044] S400, determine whether par is greater than or equal to pre1; if so, determine the target data source as an abnormal data source; otherwise, if mom1=mom0, proceed to step S500.
[0045] Among them, pre1 is the first target threshold;
[0046] S500, determine whether par is greater than or equal to pre2; if so, proceed to step S600.
[0047] Wherein, pre2 is the second target threshold, and pre2<pre1.
[0048] S600 , in response to reaching the third time mom3 , updating mom1 to mom3 , updating pre1 to γ*pre1 , and entering step S200 .
[0049] Where mom3>mom0, γ is the reduction parameter, γ={[(mom0-mom2) / (mom3-mom2)]*(v 1 / v 2 )*H}^(1 / 3).
[0050] v 1 is the number of data acquisition requests sent by the server to the target data source between mom2 and mom0, v 1 =∑ i=1 n v(i);v 2 is the number of data acquisition requests sent by the server to the target data source between mom2 and mom3; H is the influence coefficient. ave ≤dur0, then H=(dur0-dur ave ) / dur0, if dur ave >dur0, then H=0; dur0 is the preset response time; dur ave is the reference response time, dur ave =∑ L=1 v(i) dur L ,dur L It is the difference between the time when the server sends the Lth data acquisition request to the target data source between mom2 and mom3 and the time when the server receives the acquired data or the target failure identifier corresponding to the data acquisition request.
[0051] Therefore, it can be known that in the present invention, the first reliability parameter par of the target data source is determined by the failure rate of the server in acquiring data from the target data source through the i-th data interface between mom2 and mom0, and the target data source is determined as an abnormal data source when par is greater than or equal to pre1. When par is less than pre1 and par is greater than or equal to pre2, the first reliability parameter par of the target data source is updated according to the failure rate of the server in acquiring data from the target data source through the i-th data interface between mom2 and mom3, and the target data source is determined as an abnormal data source when the updated par is greater than or equal to the updated pre1.
[0052] Compared with the related art in which the target data source is determined as an abnormal data source only based on the high failure rate of obtaining data from the target data source within a shorter test time period, in the present invention, when par is less than pre1 and par is greater than or equal to pre2, whether the target data source is an abnormal data source is determined based on the failure rate of obtaining data from the target data source between mom2 and mom3, that is, when par is larger, whether the target data source is an abnormal data source is determined based on a longer test time period, thereby reducing the possibility of determining a normally working target data source as an abnormal data source due to network fluctuations and the like, thereby achieving the purpose of improving the accuracy of determining abnormal data sources.
[0053] In addition, in the present invention, when determining whether the target data source is an abnormal data source based on mom2 and mom3, pre1 at this time is lower than pre1 based on mom2 and mom0 to determine whether the target data source is an abnormal data source. Since the target data source will be determined as an abnormal data source when par is greater than or equal to pre1, the present invention can reduce the influence of the low par caused by the large time interval between mom2 and mom3 on the determination of the abnormal data source when determining whether the target data source is an abnormal data source based on mom2 and mom3, thereby reducing the possibility of a normally operating target data source being determined as an abnormal data source, thereby achieving the purpose of further improving the accuracy of determining the abnormal data source.
[0054] Furthermore, the pre1 in the present invention when determining whether the target data source is an abnormal data source based on mom2 and mom3 not only takes into account the time interval between mom0 and mom2, the time interval between mom3 and mom2, the number of data acquisition requests sent by the server to the target data source between mom2 and mom0, and the number of requests v. 1 , the number of data acquisition requests sent by the server to the target data source between mom2 and mom3 v 2 , and also takes into account the difference between the time when the server sends the Lth data acquisition request to the target data source between mom2 and mom3 and the time when the server receives the acquired data or the target failure identifier corresponding to the data acquisition request. Therefore, the longer the response time of the target data source between mom2 and mom3, the smaller the pre1 is when determining whether the target data source is an abnormal data source based on mom2 and mom3. At this time, the target data source is more likely to be determined as an abnormal data source, that is, in the present invention, a target data source with too long a response time can be determined as an abnormal data source, which can improve the user experience when the user uses a target data source that works normally.
[0055] Optionally, step S400 includes the following steps:
[0056] S401, determine whether par is greater than or equal to pre1; if so, determine the target data source as an abnormal data source; otherwise, go to step S402.
[0057] S402, determine whether mom1 satisfies mom1=mom0; if so, proceed to step S500; otherwise, determine the target data source as a normal data source.
[0058] Optionally, some of the data interfaces are reserved interfaces and another part of the data interfaces are non-reserved interfaces.
[0059] In a specific implementation, if the business corresponding to any data interface belongs to the after-sales service category, the data interface is determined to be a reserved interface. For example, for the above-mentioned first data interface, second data interface, third data interface and fourth data interface, the first data interface and the second data interface are set as reserved interfaces, and the third data interface and the fourth data interface are set as non-reserved interfaces.
[0060] Based on this, step S401 includes the following steps:
[0061] S4011, determine whether par is greater than or equal to pre1; if so, determine the target data source as an abnormal data source and proceed to step S700; otherwise, proceed to step S402.
[0062] The determination method further comprises the following steps:
[0063] S700: disconnect each non-reserved interface of the server from the target data source, and determine the current time as the disconnection time mom 0 .
[0064] S800, connecting each data interface of the server to a standby data source corresponding to the target data source.
[0065] Exemplarily, if the server is an airline server, the target data source and the corresponding backup data source are data sources corresponding to different online ticketing platforms.
[0066] S900, in response to reaching the first detection time mom 1 , obtain the second reliability parameter par corresponding to the target data source 1 .
[0067] Among them, par 1 =∑ I=1 R [s′(I) / v′(I)] / R; s′(I) is the value at the second detection time mom 2 With mom 1The number of target failure flags sent by the target data source received by the server through the Ith reserved interface, mom 0 <mom 2 <mom 1 , v′(I) is the value at the second detection time mom 2 With mom 1 The number of data acquisition requests sent by the server to the target data source through the Ith reserved interface; R is the number of reserved interfaces, R<n.
[0068] Specifically, the end time of any detection sliding time window among several detection sliding time windows is taken as mom 1 .
[0069] In a specific implementation, the end time of each detection sliding time window in a plurality of preset detection sliding time windows is used as mom 1 , for any mom 1 The determination method of the embodiment of the present invention can be executed.
[0070] S1000, if par 1 ≥thr, disconnect each non-reserved interface from the backup data source and connect each non-reserved interface to the target data source; thr is a preset threshold.
[0071] It can be seen that in the present invention, after the target data source is determined to be an abnormal data source, the server can obtain data through the backup data source corresponding to the target data source. 1 When thr is reached, it indicates that the target data source has basically recovered to its normal working state. At this time, the server can disconnect from the backup data source and restore the connection with the target data source, so as to switch back to the target data source from the backup data source in time after the target data source recovers to its normal working state.
[0072] Optional, thr ≥ 0.8.
[0073] Optional, (mom3-mom2)≤(mom 1 -mom 2 ).
[0074] It can be seen that compared with (mom 1 -mom 2 )>(mom3-mom2), the duration of the time period for which the server determines whether to restore the connection with the target data source in the present invention (mom 1 -mom 2) is not less than the duration of the time period for determining whether the server is an abnormal server (mom3-mom2) or (mom0-mom2), which can reduce the par caused by the short duration of the time period for determining whether the server needs to restore the connection with the target data source when the target data source is still an abnormal data source. 1 The possibility of the server restoring the connection with the target data source is greater. Therefore, the possibility of the server and the target data source being normally connected when the target data source is an abnormal data source can be reduced, thereby achieving the purpose of improving the success rate of the server in obtaining data.
[0075] Optional, (mom3-mom0)≤(mom0-mom2).
[0076] It can be seen from this that compared to (mom3-mom0)>(mom0-mom2), in the present invention, (mom3-mom0)≤(mom0-mom2) can reduce the possibility of the target data source being determined as an abnormal data source a long time after being abnormal due to the excessive length of (mom3-mom0), thereby achieving the purpose of improving the timeliness of determining the abnormal data source.
[0077] Optional, 3 seconds ≤ dur0 ≤ 15 seconds.
[0078] Optionally, step S500 includes the following steps:
[0079] S501, determine whether par is greater than or equal to pre2; if so, proceed to step S600; otherwise, determine the target data source as a normal data source.
[0080] Optionally, the target data source is configured to, for each data acquisition request received, if data acquisition according to the data acquisition request is successful, return the acquired data corresponding to the data acquisition request to the server through the data interface corresponding to the data acquisition request; the target data source is also configured to, for each data acquisition request received, if data acquisition according to the data acquisition request fails, return a target failure identifier corresponding to the data acquisition request to the server through the data interface corresponding to the data acquisition request.
[0081] Optionally, step S200 includes the following steps:
[0082] S201, obtaining target failure identification lists BS and bs corresponding to the target data source ij The corresponding target interpretation information will bs ij With bs ij The corresponding target interpretation information is used as a target failure information.
[0083] Where BS=(bs1,bs2,...,bsi ,...,bs n ), bs i =(bs i1 ,bs i2 ,...,bs ij ,...,bs im(i) ), i=1,2,...,n, j=1,2,...,m(i).
[0084] The above-mentioned determination of the target data source as an abnormal data source includes the following steps:
[0085] S410, determine A 1 Whether there is a first preset failure information that is the same as each target failure information; if so, the target data source is determined to be an abnormal data source; otherwise, data update processing is performed on each target failure identifier, and after data update processing is performed on each target failure identifier, step S300 is entered.
[0086] The data update process includes the following steps:
[0087] S4101, taking the target failure flag of the current data update process as the current failure flag;
[0088] S4102, confirm A 1 whether there is a first preset failure identifier that is the same as the current failure identifier; if not, proceed to step S4103;
[0089] S4103, the first preset failure information a to be added 00 1 Add to A 1 in and pri max The corresponding first preset failure information group; a 00 1 =(bs 00 1 ,ms 00 1 ), bs 00 1 Current failure flag, ms 00 1 The target interpretation information corresponding to the current failure mark; the maximum priority pri max =max(pri1,pri2,...,pri k ,...,pri z ), max() is the preset maximum value determination function, a k 1 The corresponding priority pri k =(∑ x=1 y(k) d kx) / y(k), d kx The target interpretation information and ms corresponding to the current failure mark kx 1 The degree of match between them.
[0090] It can be seen that in the present invention, when the first time mom1 is reached, if the target failure information that meets the key conditions meets the preset abnormal conditions, and A 1 If there is no first preset failure information that is the same as any target failure information, data update processing is performed on each target failure identifier to update A 1 , in updating A 1 After that, re-determine whether the target failure information that meets the key conditions meets the preset abnormal conditions until the target data source is determined to be a normal data source, or due to A 1 The target data source is determined as an abnormal data source because the first preset failure information that is the same as each target failure information exists in the data.
[0091] Compared with the related art, when the first time mom1 is reached, if the target failure information that meets the key conditions meets the preset abnormal conditions, the target data source is determined as the abnormal data source. In the present invention, after determining that the target failure information that meets the key conditions meets the preset abnormal conditions, A is updated. 1 , and then determine whether the target data source is an abnormal data source, thereby reducing the A stored in the server 1 The possibility that a normal target data source is determined as an abnormal data source due to failure to update in full real time can be reduced, thereby improving the accuracy of determining the abnormal data source.
[0092] Further, in the present invention, when updating A 1 When a 00 1 Add to A 1 in and pri max The corresponding first preset failure information group, and pri max =max(pri1,pri2,...,pri k ,...,pri z ), due to pri k The determination takes into account the target interpretation information corresponding to the current failure mark and a k 1 The target interpretation information and the preset interpretation information used to determine the matching degree in the present invention are both generated by the target data source, that is, the target interpretation information and the preset interpretation information used to determine the matching degree in the present invention are both generated by the same subject, and then the a determined according to the matching degree in the present invention is 00 1The corresponding first preset failure information group is more accurate, which can improve the accuracy of data updating.
[0093] In addition, in the present invention, data update processing is performed when the target failure information that meets the key conditions meets the preset abnormal conditions. Compared with performing data update processing after reaching the first time mom1, in the present invention, data update processing will not be performed when the target failure information that meets the key conditions does not meet the preset abnormal conditions. At this time, the computing amount of the server can be reduced and computing resources can be saved.
[0094] Optionally, step S4102 includes the following steps:
[0095] S41021, confirm A 1 Whether there is a first preset failure identifier that is the same as the current failure identifier; if so, go to step S4104; otherwise, go to step S4103.
[0096] The data update process also includes the following steps:
[0097] S4104, determining whether the preset interpretation information of the first preset failure identifier that is the same as the current failure identifier is the same as the target interpretation information corresponding to the current failure identifier; if not, proceeding to step S4105.
[0098] S4105, confirm A 1 Whether there is preset interpretation information that is the same as the target interpretation information corresponding to the current failure identifier; if so, enter step S4106.
[0099] S4106, from A 1 The first preset failure information where the first preset failure identifier that is the same as the current failure identifier is located is deleted.
[0100] S4107, A 1 The first preset failure identifier corresponding to the preset interpretation information that is the same as the target interpretation information corresponding to the current failure identifier is replaced with the current failure identifier.
[0101] Optionally, step S4105 includes the following steps:
[0102] S41051, confirm A 1 Whether there is preset interpretation information that is the same as the target interpretation information corresponding to the current failure identifier; if so, go to step S4106; otherwise, go to step S4108.
[0103] The data update process also includes the following steps:
[0104] S4108, the first preset failure information a to be added 00 1 Add to A1 in and pri max The corresponding first preset failure information group.
[0105] Among them, a 00 1 =(bs 00 1 ,ms 00 1 ), bs 00 1 Current failure flag, ms 00 1 The target interpretation information corresponding to the current failure mark; the maximum priority pri max =max(pri1,pri2,...,pri k ,...,pri z ), max() is the preset maximum value determination function, a k 1 The corresponding priority pri k =(∑ x=1 y(k) d kx ) / y(k), d kx The target interpretation information and ms corresponding to the current failure mark kx 1 The degree of match between them.
[0106] S4109, A 1 Middle 00 1 Each first preset failure information other than 1 The candidate preset failure information where the first preset failure identifier that is the same as the current failure identifier is located is deleted.
[0107] Optional, w i is obtained by following the steps below:
[0108] S1001, A 1 Each first preset failure indicator corresponding to a key category is used as a target preset failure indicator.
[0109] Specifically, A 1 Each first preset failure identifier in the first preset failure information group corresponding to each second preset failure information belonging to the key category is a first preset failure identifier corresponding to the key category.
[0110] S1002, obtain the time list TIME=(time1, time2, ..., time i ,...,time n ), timei =(time i1 ,time i2 ,...,time iu ,...,time iv(i) ),u=1,2,...,v(i); where time i is the time group corresponding to the i-th data interface, n is the number of data interfaces, time iu is the time when the server sends the uth data acquisition request to the target data source through the i-th data interface between mom2 and mom1. i1 ≤time i2 ≤...≤time iu ≤...≤time iv(i) , mom2<time iu < mom1; v(i) is the number of data acquisition requests sent by the server to the target data source through the i-th data interface between mom2 and mom1.
[0111] S1003, according to TIME, obtain the time difference list ΔT=(Δt1, Δt2, ..., Δt Q ,...,Δt n ), Δt i =(Δt i1 ,Δt i2 ,...,Δt iu ,...,Δt iv(i) ).
[0112] Among them, Δt i for time i The corresponding time difference group, Δt iu for time iu The corresponding time difference; Δt iu The following conditions must be met:
[0113] If u=1, Δt i1 =[(time i1 -mom2)+(time i2 -time i1 )] / 2;
[0114] If u=q, Δt iq =[(time iq -time i(q-1) )+(time i(q+1) -time iq )] / 2,q=2,3,...,v(i)-1;
[0115] If u=v(i), Δt iv(i)=[(time iv(i) -time i(v(i)-1) )+(mom1-time iv(i) )] / 2.
[0116] S1004, obtain w i =α*v(i)+β / F i , α is the first preset coefficient, β is the second preset coefficient, F i is the time difference fluctuation parameter corresponding to the i-th data interface, F i =[∑ u=1 v(i) (Δt iu -Δt ave i ) 2 ] / v(i),Δt ave i is the reference time difference corresponding to the i-th data interface, Δt ave i =(∑ u=1 v(i) Δt iu ) / v(i).
[0117] The F in the present invention i The larger the w i The smaller the value, the smaller the influence of s(i) / v(i) on par. i =[∑ u=1 v(i) (Δt iu -Δt ave i ) 2 ] / v(i), therefore, the greater the fluctuation of the time difference of the data acquisition request sent by the server between mom2 and mom1 to the target data source through the i-th data interface, the smaller the impact of s(i) / v(i) on par, that is, the more concentrated the data acquisition requests sent by the server between mom2 and mom1 to the target data source through the i-th data interface, the smaller the impact of s(i) / v(i) on par, and further, when the time of the data acquisition requests sent by the server between mom2 and mom1 to the target data source through the i-th data interface is mostly concentrated before the target server starts to become abnormal, resulting in a smaller s(i) / v(i), the possibility of not determining the target server as an abnormal server can be reduced, thereby further improving the accuracy of determining the abnormal data source.
[0118] Compared with the related art in which the target data source is determined as an abnormal data source when the failure rate of acquiring data from the target data source within a preset time period is high, the present invention iConsidering the number of data acquisition requests v(i) sent by the server to the target data source through the i-th data interface between mom2 and mom1, the smaller v(i) is, the greater the weight w corresponding to the i-th data interface is. i The smaller it is, the smaller the impact of s(i) / v(i) on par, that is, the smaller v(i), the smaller the impact of the failure rate of the server in obtaining data from the target data source through the i-th data interface between mom2 and mom1 on determining whether the target data source is an abnormal data source. Therefore, when the target data source is an abnormal data source and the failure rate of obtaining data from the target data source through the i-th data interface between mom2 and mom1 is low due to the small v(i), the possibility of the failure rate being low and the par being small, and thus the target data source not being determined as an abnormal data source, can be reduced. Therefore, the accuracy of determining the abnormal data source can be improved.
[0119] Optional, 1≤α≤5, 100≤β≤10000.
[0120] Optional, pre1 = (∑ i=1 n w i )*coe, coe is the preset target coefficient, 0.5<coe≤0.9.
[0121] Optionally, step S4104 includes the following steps:
[0122] S41041, determine whether the preset interpretation information of the first preset failure identifier that is the same as the current failure identifier is the same as the target interpretation information corresponding to the current failure identifier; if so, end the data update process for the current failure identifier; otherwise, enter step S4105.
[0123] Optionally, the matching degree is the matching degree between the feature value of the corresponding target interpretation information and the feature value of the corresponding preset interpretation information, or the matching degree is the semantic matching degree between the corresponding target interpretation information and the corresponding preset interpretation information.
[0124] Specifically, the characteristic value is a hash value. In a specific implementation, the matching degree is the difference between the hash value of the corresponding target interpretation information and the hash value of the corresponding preset interpretation information. For example, if the hash value of any target interpretation information is 0110 and the hash value of any preset interpretation information is 0111, then according to 0111-0110=0001, it can be known that the matching degree between the target interpretation information and the preset interpretation information is 0001.
[0125] An embodiment of the present invention also provides a non-transitory computer-readable storage medium, which can be set in an electronic device to store at least one instruction or at least one program related to implementing a method in a method embodiment. The at least one instruction or the at least one program is loaded and executed by the processor to implement the determination method provided in the above embodiment.
[0126] An embodiment of the present invention further provides an electronic device, comprising a processor and the aforementioned non-transitory computer-readable storage medium.
[0127] An embodiment of the present invention further provides a computer program product, which includes program code. When the program product is run on an electronic device, the program code is used to enable the electronic device to execute the steps of the determination method according to various exemplary embodiments of the present invention described above in this specification.
[0128] Although some specific embodiments of the present invention have been described in detail by way of example, it should be understood by those skilled in the art that the above examples are for illustration only and are not intended to limit the scope of the present invention. It should also be understood by those skilled in the art that various modifications may be made to the embodiments without departing from the scope and spirit of the present invention. The scope of the present invention is defined by the appended claims.
Claims
1. A method for determining an abnormal data source, characterized in that: The determination method is applied to a server; the server is connected to a target data source, and the server is connected to the target data source through a plurality of data interfaces of the server; The server is used to send a number of corresponding data acquisition requests to the target data source through each of the data interfaces; the target data source is configured to, in response to receiving each of the data acquisition requests, return the acquired data or target failure flag corresponding to the data acquisition request to the server through the corresponding data interface; The determination method comprises the following steps: S100, in response to reaching the first time mom1, obtaining the target time mom0=mom1; S200, obtaining a target failure identifier list BS corresponding to the target data source = (bs1, bs2, ..., bs i ,...,bs n ), bs i =(bs i1 ,bs i2 ,...,bs ij ,...,bs im(i) ), i = 1, 2, ..., n, j = 1, 2, ..., m(i); where bs i is the target failure identification group corresponding to the i-th data interface, n is the number of data interfaces; bs ij is the jth target failure identifier returned by the target data source to the server through the i-th data interface between mom2 and mom1, mom2 is the second time, mom2<mom1; m(i) is the number of target failure identifiers returned by the target data source to the server through the i-th data interface between mom2 and mom1; S300, according to BS, obtain a first reliability parameter par of the target data source=∑ i=1 n [w i *s(i) / v(i)];w i is the weight corresponding to the i-th data interface, s(i) is bs i The number of target failure identifiers that are the same as any target preset failure identifier, v(i) is the number of data acquisition requests sent by the server to the target data source through the i-th data interface between mom2 and mom1; S400, determine whether par is greater than or equal to pre1; if so, determine the target data source as an abnormal data source; otherwise, if mom1=mom0, proceed to step S500; pre1 is the first target threshold; S500, determine whether par is greater than or equal to pre2; if so, proceed to step S600; pre2 is the second target threshold, pre2<pre1; S600, in response to reaching the third time mom3, mom1 is updated to mom3, pre1 is updated to γ*pre1, and the process goes to step S200; mom3>mom0, γ is the reduction parameter, γ={[(mom0-mom2) / (mom3-mom2)]*(v 1 / v 2 )*H}^(1 / 3);v 1 is the number of data acquisition requests sent by the server between mom2 and mom0 to the target data source, v 1 =∑ i=1 n v(i);v 2 is the number of data acquisition requests sent by the server between mom2 and mom3 to the target data source; H is the influence coefficient, if dur ave ≤dur0, then H=(dur0-dur ave ) / dur0, if dur ave >dur0, then H=0; dur0 is the preset response time; dur ave is the reference response time, dur ave =∑ L=1 v(i) dur L ,dur L It is the difference between the time when the server sends the Lth data acquisition request to the target data source between mom2 and mom3 and the time when the server receives the acquired data or the target failure mark corresponding to the data acquisition request.
2. The determination method according to claim 1, characterized in that: The step S400 includes the following steps: S401, determine whether par is greater than or equal to pre1; if so, determine the target data source as an abnormal data source; otherwise, proceed to step S402; S402, determine whether mom1 satisfies mom1=mom0; if so, proceed to step S500; otherwise, determine the target data source as a normal data source.
3. The determination method according to claim 2, characterized in that: Some of the data interfaces are reserved interfaces and another part of the data interfaces are non-reserved interfaces; The step S401 includes the following steps: S4011, determine whether par is greater than or equal to pre1; if so, determine the target data source as an abnormal data source and proceed to step S700; otherwise, proceed to step S402; The determination method further comprises the following steps: S700: disconnect each of the non-reserved interfaces of the server from the target data source, and determine the current time as the disconnection time mom 0 ; S800, connecting each of the data interfaces of the server to a standby data source corresponding to the target data source; S900, in response to reaching the first detection time mom 1 , obtain the second reliability parameter par corresponding to the target data source 1 =∑ I=1 R [s′(I) / v′(I)] / R; s′(I) is the value at the second detection time mom 2 With mom 1 The number of target failure identifiers sent by the target data source received by the server through the Ith reserved interface, mom 0 <mom 2 <mom 1 , v′(I) is the value at the second detection time mom 2 With mom 1 is the number of data acquisition requests sent by the server to the target data source through the Ith reserved interface; R is the number of the reserved interfaces; S1000, if par 1 ≥thr, disconnect each of the non-reserved interfaces from the backup data source and connect each of the non-reserved interfaces to the target data source; thr is a preset threshold.
4. The determination method according to claim 3, characterized in that: (mom3-mom2)≤(mom 1 -mom 2 )。 5. The determination method according to claim 1, characterized in that: (mom3-mom0)≤(mom0-mom2).
6. The determination method according to claim 1, characterized in that: 3 seconds ≤dur0 ≤15 seconds.
7. The determination method according to claim 1, characterized in that: The step S500 includes the following steps: S501, determine whether par is greater than or equal to pre2; if so, proceed to step S600; otherwise, determine the target data source as a normal data source.
8. The determination method according to claim 1, characterized in that: The target data source is configured to, for each received data acquisition request, return the acquired data corresponding to the data acquisition request to the server through the data interface corresponding to the data acquisition request if data acquisition is successful according to the data acquisition request; the target data source is also configured to, for each received data acquisition request, return the target failure identifier corresponding to the data acquisition request to the server through the data interface corresponding to the data acquisition request if data acquisition fails according to the data acquisition request.
9. A non-transitory computer-readable storage medium, wherein at least one instruction or at least one program is stored in the storage medium, characterized in that: The at least one instruction or the at least one program is loaded and executed by the processor to implement the determination method according to any one of claims 1-8.
10. An electronic device, characterized in that: Includes a processor and the non-transitory computer-readable storage medium of claim 9.
