Timing exception identification method, device and equipment and readable storage medium

Abstract

This invention discloses a method, apparatus, device, and readable storage medium for identifying time-series anomalies. The method includes: performing position-enhanced feature extraction processing on a business data sequence to obtain a position-enhanced feature vector matrix corresponding to the business data sequence; performing multi-layer feature extraction and transformation processing on the position-enhanced feature vector matrix through L transformer layers with chained connections in a time-series feature mining model to obtain intermediate feature representation vector matrices output by the L transformer layers respectively; performing time-series feature mining processing on the intermediate feature representation vector matrices output by the connected transformer layers through L time-series network layers to obtain a merged time-series feature vector; and performing time-series anomaly identification processing on the merged time-series feature vector to obtain the time-series anomaly identification result of the business data sequence. Using this invention can improve the utilization rate of sequence position information, thereby improving the accuracy of time-series anomaly identification.

Description

Technical Field

[0001] This application relates to the field of computer technology, and in particular to a method, apparatus, device and readable storage medium for identifying timing anomalies. Background Technology

[0002] Temporal anomaly identification refers to the process of detecting data points or data segments that are significantly different from or abnormal from a business data sequence with a temporal relationship. It has a wide range of applications in many businesses, such as abnormal user identification in payment scenarios and detection of extreme weather events or pollution sources in environmental monitoring.

[0003] Existing methods for identifying temporal anomalies typically rely on the traditional Transformer model to extract features from business data sequences, and then apply these features for anomaly detection. However, the traditional Transformer model has limitations in handling sequence order information; its attention mechanism is insensitive to the order of sequences. While it's possible to incorporate the positional encoding of the business data sequence into the features through addition, this approach is insufficient to fully preserve and extract the sequence's positional information. The addition operation may weaken the model's sensitivity to sequence positional information, thereby reducing the accuracy of temporal anomaly identification. Summary of the Invention

[0004] This application provides a method, apparatus, device, and readable storage medium for identifying timing anomalies, which can improve the utilization rate of sequence position information and thus improve the accuracy of timing anomaly identification.

[0005] This application provides a method for identifying timing anomalies, including:

[0006] Obtain a business data sequence with temporal relationship, perform location enhancement feature extraction processing on the business data sequence, and obtain the location enhancement feature vector matrix corresponding to the business data sequence;

[0007] The position enhancement feature vector matrix is ​​subjected to multi-layer feature extraction and transformation through L transformer layers in the temporal feature mining model, resulting in intermediate feature representation vector matrices output by each of the L transformer layers; L is a positive integer; the L transformer layers have a chain-like connection relationship; the temporal feature mining model also includes a temporal network layer connected to each of the L transformer layers.

[0008] The intermediate feature representation vector matrix output by the connected transformer layer is processed by L time-series network layers to obtain merged time-series feature vectors. Time-series anomaly identification is then performed on the merged time-series feature vectors to obtain the time-series anomaly identification results of the business data sequence.

[0009] The business data sequence includes business data B. i The location-enhanced feature vector matrix contains business data B. i The corresponding position-enhanced feature vector E i i is a positive integer less than or equal to the number of business data contained in the business data sequence;

[0010] The business data sequence is processed by location-enhanced feature extraction to obtain the location-enhanced feature vector matrix corresponding to the business data sequence, including:

[0011] For business data B i Perform data feature extraction processing to obtain business data B. i The corresponding data feature vector;

[0012] For business data B i Position encoding is performed on the position in the business data sequence to obtain business data B. i The corresponding positional encoding vector;

[0013] For business data B i The corresponding data feature vector and business data B i The corresponding location encoding vectors are fused to obtain business data B. i The corresponding position-enhanced feature vector E i ;

[0014] When the location-enhanced feature vector corresponding to each business data in the business data sequence is obtained, a location-enhanced feature vector matrix is ​​generated based on the location-enhanced feature vector corresponding to each business data.

[0015] Among them, business data B i Position encoding is performed on the position in the business data sequence to obtain business data B. i The corresponding positional encoding vector includes:

[0016] Obtain the feature vector dimension of the feature vector, and generate an initial vector with the same vector dimension as the feature vector dimension;

[0017] According to business data B i Sine coding is performed on the position and feature vector dimension in the business data sequence to obtain the even-numbered position coding value;

[0018] According to business data B i Cosine coding is performed on the position and feature vector dimension in the business data sequence to obtain the odd position coding value;

[0019] Replace the even-numbered elements in the initial vector with their coded values, and replace the odd-numbered elements with their coded values ​​to obtain business data B. i The corresponding positional encoding vector.

[0020] Among them, business data B i The corresponding data feature vector and business data B i The corresponding location encoding vectors are fused to obtain business data B. i The corresponding position-enhanced feature vector E i ,include:

[0021] In the row direction, business data B i The corresponding position encoding vector is concatenated in business data B i The business data B is obtained from the corresponding data feature vector. i The corresponding position-enhanced feature vector E i .

[0022] Specifically, the location-enhanced feature vector matrix is ​​subjected to multi-layer feature extraction and transformation through L transformer layers in the temporal feature mining model, resulting in intermediate feature representation vector matrices output by the L transformer layers, including:

[0023] Based on the chained connection relationship of L transformer layers in the time series feature mining model, traverse the L transformer layers to obtain the j-th transformer layer; j is a positive integer less than or equal to L;

[0024] The input data is processed by feature extraction and transformation through the j-th transformer layer to obtain the intermediate feature representation vector matrix output by the j-th transformer layer. This process continues to traverse the next connected transformer layer until the intermediate feature representation vector matrices output by L transformer layers are obtained respectively. If j is not 1, the input data is the intermediate feature representation vector matrix output by the (j-1)-th transformer layer. If j is 1, the input data is the position enhancement feature vector matrix.

[0025] Specifically, the input data undergoes feature extraction and transformation processing through the j-th transformer layer to obtain the intermediate feature representation vector matrix output by the j-th transformer layer, which includes:

[0026] The input data is processed by self-attention through the j-th transformer layer to obtain the attention feature vector matrix;

[0027] The attention feature vector matrix is ​​subjected to the first connection normalization process to obtain the connection normalized vector matrix;

[0028] The eigenvalue transformation is performed on the connection normalized vector matrix to obtain the eigenvalue transformation vector matrix;

[0029] The feature transformation vector matrix is ​​subjected to a second connection normalization process to obtain the intermediate feature representation vector matrix output by the j-th transformer layer.

[0030] Specifically, the input data undergoes self-attention processing through the j-th transformer layer to obtain the attention feature vector matrix, which includes:

[0031] The query weight, key weight, and value weight are obtained through the j-th transformer layer.

[0032] Generate a query vector matrix based on query weights and input data, generate a key vector matrix based on key weights and input data, and generate a value vector matrix based on value weights and input data.

[0033] Calculate the similarity score between the query vector matrix and the key vector matrix to obtain the similarity score matrix;

[0034] The similarity score matrix is ​​scaled to obtain the scaled similarity score matrix;

[0035] The scaling similarity score matrix is ​​normalized to obtain the attention weight matrix;

[0036] Multiplying the attention weight matrix and the value vector matrix yields the attention feature vector matrix.

[0037] The similarity score matrix is ​​scaled to obtain a scaled similarity score matrix, which includes:

[0038] Obtain the dimension of the location-enhanced feature vector; the dimension of the location-enhanced feature vector refers to the vector dimension of each location-enhanced feature vector contained in the location-enhanced feature vector matrix;

[0039] The scaled similarity score matrix is ​​obtained by dividing the scaled similarity score matrix by the square root of the dimension of the position-enhanced feature vector.

[0040] Specifically, L temporal feature mining processes are performed on L intermediate feature representation vector matrices through L temporal network layers to obtain merged temporal feature vectors, including:

[0041] Traverse the L temporal network layers to obtain the h-th temporal network layer; h is a positive integer less than or equal to L;

[0042] The intermediate feature representation vector matrix output by the transformer layer connected by the h-th temporal network layer is determined as the h-th intermediate feature representation vector matrix among the L intermediate feature representation vector matrices.

[0043] Temporal feature mining is performed on the h-th intermediate feature representation vector matrix through the h-th temporal network layer to obtain the h-th temporal feature vector;

[0044] After traversing all L temporal network layers, the temporal feature vectors output by the L temporal network layers are merged to obtain the merged temporal feature vector.

[0045] Specifically, the temporal feature vectors output by the L temporal network layers are merged to obtain the merged temporal feature vector, which includes:

[0046] In the row direction, according to the arrangement order of the L temporal network layers, the temporal feature vectors output by the L temporal network layers are concatenated in order to obtain the merged temporal feature vector.

[0047] The process involves performing time-series anomaly detection processing on the merged time-series feature vectors to obtain the time-series anomaly detection results for the business data sequence, including:

[0048] Obtain the time series anomaly detection model; the time series anomaly detection model includes a fully connected layer and a classification layer;

[0049] The merged temporal feature vector is processed by a fully connected layer to obtain an intermediate feature vector;

[0050] The intermediate feature vectors are classified by a classification layer to obtain the classification result. The result of the time-series anomaly identification of the business data sequence is determined based on the classification result.

[0051] One embodiment of this application provides a timing anomaly identification device, including:

[0052] The extraction module is used to acquire business data sequences with temporal relationships, perform location-enhanced feature extraction processing on the business data sequences, and obtain the location-enhanced feature vector matrix corresponding to the business data sequences.

[0053] The attention module is used to perform multi-layer feature extraction and transformation on the position enhancement feature vector matrix through L transformer layers in the temporal feature mining model, to obtain intermediate feature representation vector matrices output by L transformer layers respectively; L is a positive integer; the L transformer layers have a chain connection relationship; the temporal feature mining model also includes a temporal network layer connected to the L transformer layers respectively;

[0054] The temporal feature mining module is used to perform temporal feature mining on the intermediate feature representation vector matrix output by the connected transformer layer through L temporal network layers to obtain the merged temporal feature vector.

[0055] The timing anomaly detection module is used to perform timing anomaly detection processing on the merged timing feature vector to obtain the timing anomaly detection results of the business data sequence.

[0056] One embodiment of this application provides a computer device, including: a processor, a memory, and a network interface;

[0057] The processor is connected to the memory and the network interface. The network interface is used to provide a data communication network element, the memory is used to store a computer program, and the processor is used to call the computer program to execute the method in the embodiments of this application.

[0058] One aspect of this application provides a computer-readable storage medium storing a computer program adapted to be loaded by a processor and executed by the method described in this application.

[0059] One aspect of this application provides a computer program product or computer program that includes computer instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the computer device to perform the method described in this application.

[0060] In this embodiment, location-enhanced feature extraction processing can be performed on a business data sequence with temporal relationships to obtain a location-enhanced feature vector matrix corresponding to the business data sequence. Then, the location-enhanced feature vector matrix is ​​subjected to multi-layer feature extraction transformation processing through L transformer layers in the temporal feature mining model to obtain intermediate feature representation vector matrices output by the L transformer layers respectively. Further, temporal feature mining processing is performed on the intermediate feature representation vector matrices output by the connected transformer layers through temporal network layers in the temporal feature mining model to obtain a merged temporal feature vector. Finally, temporal anomaly detection processing is performed on the merged temporal feature vector to obtain the temporal anomaly detection result of the business data sequence. Using the method provided in this embodiment, a temporal network layer can be introduced for each transformer layer to learn and mine the temporal features of the business data sequence, thereby effectively improving the utilization rate of sequence location information and discovering richer temporal features, thus effectively improving the accuracy of temporal anomaly detection. Attached Figure Description

[0061] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0062] Figure 1 This is a schematic diagram of a network architecture provided in an embodiment of this application;

[0063] Figure 2This is a schematic diagram illustrating an application scenario of a timing anomaly identification method provided in an embodiment of this application;

[0064] Figure 3 This is a flowchart illustrating a timing anomaly identification method provided in an embodiment of this application;

[0065] Figure 4 This is a schematic diagram of the model structure of a time-series feature mining model provided in an embodiment of this application;

[0066] Figure 5 This is a schematic diagram of the structure of a timing anomaly identification model provided in an embodiment of this application;

[0067] Figure 6 This is a flowchart illustrating a timing anomaly identification method provided in an embodiment of this application;

[0068] Figure 7 This is a schematic diagram of the overall process of training a time-series anomaly recognition model provided in an embodiment of this application;

[0069] Figure 8 This is a schematic diagram of the application process of a timing anomaly identification model provided in an embodiment of this application;

[0070] Figure 9 This is a schematic diagram of the structure of a timing anomaly identification device provided in an embodiment of this application;

[0071] Figure 10 This is a schematic diagram of the structure of a computer device provided in an embodiment of this application. Detailed Implementation

[0072] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of this application.

[0073] For details, please see Figure 1 , Figure 1 This is a schematic diagram of a network architecture provided in an embodiment of this application. Figure 1 As shown, this network architecture may include a server 2000 and a terminal device cluster. The terminal device cluster may specifically include one or more terminal devices; the number of terminal devices in the cluster is not limited here. Figure 1As shown, the multiple terminal devices may specifically include terminal device 3000a, terminal device 3000b, terminal device 3000c, ..., terminal device 3000n; terminal device 3000a, terminal device 3000b, terminal device 3000c, ..., terminal device 3000n can communicate with server 2000 through a network (i.e., through a medium that provides a communication link via wired, wireless communication links or fiber optic cables, etc.) to facilitate data interaction between each terminal device and server 2000.

[0074] Each terminal device in the terminal device cluster can include: smartphones, tablets, laptops, desktop computers, smart voice interaction devices, smart home appliances (e.g., smart TVs), wearable devices, in-vehicle terminals, aircraft, and other smart terminals with data processing capabilities. It should be understood that, as... Figure 1 Each terminal device in the terminal device cluster shown can have an application client installed. When the application client runs on each terminal device, it can interact with the aforementioned... Figure 1 Data exchange occurs between the 2000 servers shown.

[0075] Specifically, the application client can include in-vehicle clients, smart home clients, entertainment clients (e.g., game clients), multimedia clients (e.g., video clients), social clients, and information clients (e.g., news clients) with data processing capabilities. In this embodiment, the application client can be integrated into a client (e.g., a social client), or it can be a standalone client (e.g., a news client). This embodiment does not limit the type of application client.

[0076] Among them, such as Figure 1 The server 2000 shown can be the server corresponding to the application client. The server 2000 can be an independent physical server, a server cluster or distributed system composed of multiple physical servers, or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, CDN (Content Delivery Network), and big data and artificial intelligence platforms.

[0077] For ease of understanding, the embodiments of this application may be described in detail below. Figure 1 From the multiple terminal devices shown, one terminal device is selected as the target terminal device. For example, in the embodiments of this application, a terminal device can be selected as the target terminal device. Figure 1The terminal device 3000a shown serves as the target terminal device, which can have an application client with data processing capabilities installed. In this case, the target terminal device can interact with the server 2000 through the application client.

[0078] like Figure 1 As shown, when the server 2000 receives a relevant request from the target terminal device, it can obtain a service data sequence with a temporal relationship, determine the temporal anomaly identification result of the service data sequence using the temporal anomaly identification method provided in this application embodiment, and then return it to the target terminal device. Specifically, the temporal anomaly identification method provided in this application can be as follows: performing position enhancement feature extraction processing on the service data sequence to obtain a position enhancement feature vector matrix corresponding to the service data sequence; performing multi-layer feature extraction transformation processing on the position enhancement feature vector matrix through L transformer layers in the temporal feature mining model to obtain intermediate feature representation vector matrices output by the L transformer layers respectively; the L transformer layers have a chain-like connection relationship; the temporal feature mining model also includes temporal network layers connected to the L transformer layers respectively; performing temporal feature mining processing on the intermediate feature representation vector matrices output by the connected transformer layers through the L temporal network layers to obtain a merged temporal feature vector; performing temporal anomaly identification processing on the merged temporal feature vector to obtain the temporal anomaly identification result of the service data sequence.

[0079] It is understood that the time-series anomaly identification method provided in this application can be applied to various business scenarios, such as user payment scenarios, environmental monitoring scenarios, and equipment maintenance scenarios. Specific business scenarios will not be listed here. For example, in a user payment scenario, the business data sequence may contain historical payment data (such as payment amount, payment time, payment method, etc.) corresponding to multiple payments received by the payee. In this case, the time-series anomaly identification method provided in this application can be used to predict whether the payee has any anomalies. As another example, in an environmental monitoring environment, the business data sequence may contain multiple historical environmental data (such as data collection time, environmental meteorological data, environmental quality data, etc.). In this case, the time-series anomaly identification method provided in this application can be used to predict whether extreme weather or pollution sources will occur.

[0080] For better understanding, please refer to [link / reference]. Figure 2 , Figure 2 This is a schematic diagram illustrating an application scenario of a timing anomaly identification method provided in an embodiment of this application. For example... Figure 2 The server 20a shown can be the one described above. Figure 1 The server 2000 in the corresponding embodiment, such as Figure 2 The terminal device 20b shown can be the one described above. Figure 1In the corresponding embodiment, the target terminal device 20b may have an application client installed, and the user corresponding to the terminal device 20b can be the business object U1. For ease of understanding, this embodiment uses the timing anomaly identification method executed by the server 20a as an example for illustration.

[0081] like Figure 2 As shown, business object U1 can initiate a transfer request for transfer object U2 to server 20a through terminal device 20b (e.g., ...). Figure 2 As shown in S211a), when server 20a receives the transfer request, it can perform a timing anomaly identification on the transfer object U2 to predict whether the transfer of business object U1 is safe. Specifically, server 20a can obtain the business data sequence S1 of the receiving object U2 (e.g., ...). Figure 2 As shown in 201a), the payee U2 can be an individual, enterprise, institution, etc. The business data sequence S1 can contain historical business data of multiple payees U2. The data attributes included in each business data can be the payee U2's payment method information (such as searching an account, scanning a QR code, etc.), payee identity information (such as personal identification), time information, and transfer amount information; these are not limited here. Then, server 20a can perform location-enhanced feature extraction processing on the business data sequence S1 to obtain a location-enhanced feature vector matrix X1 (such as...). Figure 2 As shown in 202a), a row vector in the position enhancement feature vector matrix X1 can be the position enhancement feature vector corresponding to a business data in the business data sequence S1. A position enhancement feature vector can be obtained by concatenating the data feature vector of the business data and its corresponding position encoding vector.

[0082] Furthermore, server 20a can obtain time-series feature mining model 203a, which may contain three transformer layers: transformer layer 204a, transformer layer 205a, and transformer layer 206a. These three transformer layers have a chained connection relationship, such as... Figure 2 As shown, transformer layer 204a is connected to transformer layer 205a, and transformer layer 205a is connected to transformer layer 206a. Based on this chain connection, the output data of the previous transformer layer is used as the input data of the next transformer layer for feature extraction and transformation processing. Figure 2 As shown, server 20a can input the location-enhanced feature vector matrix X1 into the temporal feature mining model 203a, which is equivalent to inputting it into the transformer layer 204a. The transformer layer 204a will perform feature extraction and transformation processing on it, and output the intermediate feature representation vector matrix T1 (e.g., ...). Figure 2As shown in 207a), the intermediate feature representation vector matrix T1 is the output data of transformer layer 204a. Therefore, server 20a will continue to use it as input data for transformer layer 205a. After performing feature extraction and transformation processing on the intermediate feature representation vector matrix T1, transformer layer 205a will output the intermediate feature representation vector matrix T2 (as shown in 207a). Figure 2 As shown in 208a). Similarly, transformer layer 206a performs feature extraction transformation on the intermediate feature representation vector matrix T2, outputting the intermediate feature representation vector matrix T3 (as shown in 208a). Figure 2 (As shown in 209a).

[0083] like Figure 2 As shown, the temporal feature mining model 203a also includes temporal network layers connected to three transformer layers: temporal network layer 210a connected to transformer layer 204a, temporal network layer 211a connected to transformer layer 205a, and temporal network layer 212a connected to transformer layer 206a. The server 20a also inputs the intermediate feature representation vector matrix output by each transformer layer into the temporal network layer connected to each transformer layer. Each temporal network layer performs temporal feature mining processing on the input intermediate feature representation vector matrix to obtain the temporal feature vector output by each temporal network layer. After merging the temporal feature vectors output by each temporal network layer, the merged temporal feature vector Y1 (e.g., ...) can be obtained. Figure 2 As shown in 213a). Server 20a can perform time-series anomaly identification processing on the merged time-series feature vector Y1 to obtain the time-series anomaly identification result Z1 of the business data sequence S1 (as shown in 213a). Figure 2 (As shown in 214a). If the timing anomaly identification result Z1 is a payment anomaly result, server 20a can temporarily not process the transfer request and send an anomaly alert to terminal device 20b (e.g., ...). Figure 2 As shown in S212a), this exception alert can be used to remind business object U1 that there may be risks in this transfer, and ask business object U1 to confirm whether to still execute the transfer request.

[0084] It should be noted that this application may display prompt interfaces, pop-ups, or output voice prompts before and during the collection of user data. These prompt interfaces, pop-ups, or voice prompts are used to inform the user that their data is being collected. This ensures that the application only begins the steps for collecting user data after receiving confirmation from the user regarding the prompt interface or pop-up; otherwise (i.e., without user confirmation), the steps for collecting user data end, meaning no user data is collected. In other words, all user data collected in this application is collected with the user's consent and authorization, and the collection, use, and processing of related user data must comply with the relevant laws, regulations, and standards of the relevant regions.

[0085] It is understood that the above scenarios are merely examples and do not constitute a limitation on the application scenarios of the technical solutions provided in the embodiments of this application. The technical solutions of this application can also be applied to other scenarios. For example, as those skilled in the art will know, with the evolution of system architecture and the emergence of new business scenarios, the technical solutions provided in the embodiments of this application are also applicable to similar technical problems.

[0086] Further, please see Figure 3 , Figure 3 This is a flowchart illustrating a timing anomaly identification method provided in an embodiment of this application. The method can be implemented using a computer device (e.g., the one described above). Figure 1 The method is executed by the server 2000 in the corresponding embodiment. The following description will take the execution of this method by a computer device as an example, wherein the timing anomaly identification method may include at least the following steps S101-S103:

[0087] Step S101: Obtain a business data sequence with temporal relationship, perform location enhancement feature extraction processing on the business data sequence, and obtain the location enhancement feature vector matrix corresponding to the business data sequence.

[0088] Specifically, a business data sequence with temporal relationships can be a sequence composed of multiple business data arranged in chronological order. The multiple business data contained in a business data sequence can originate from the business object to be identified for temporal anomaly detection. The business object can be an individual, group, location, device, etc.; temporal anomaly detection can be detecting whether extreme weather events are occurring in a weather forecast scenario, detecting whether payment recipients (individuals, institutions, enterprises, etc.) are experiencing payment anomalies in a payment scenario, or detecting whether equipment (such as cameras, production equipment, etc.) is malfunctioning in an equipment maintenance scenario. Therefore, business data can be data used for temporal anomaly detection of business objects. This business data can be data associated with the business object, and it can include timestamp information and word segmentation attribute data (i.e., data corresponding to one data attribute) on one or more data attributes. The timestamp information is used to represent the time information of collecting the business data, and one or more data attributes can be set based on different application scenarios and temporal anomaly detection requirements; this application does not impose any restrictions on this. For example, in a weather forecasting scenario, the data attributes of business data can include timestamp information, temperature information, precipitation information, humidity information, wind speed, and wind direction information for the business object (a specific location). Furthermore, multiple business data points can be collected at consecutive points in time or at equal time intervals. For instance, in a weather forecasting scenario, the weather conditions for the business object each day can be recorded, and this can be further refined to the hour, especially when analyzing complex weather changes within a day, such as recording hourly weather data to study short-term fluctuations in local climate. Of course, multiple business data points can also be collected whenever a business object has a transaction. For example, in a payment scenario, each time the business object receives a payment, the relevant payment details can be recorded.

[0089] Specifically, the location-enhanced feature vector matrix can be a matrix containing multiple location-enhanced feature vectors. Each location-enhanced feature vector can be a feature obtained after performing location-enhanced feature extraction on a piece of business data. Location-enhanced feature extraction can involve converting the business data into a corresponding vector representation and incorporating the location information of the business data within the business data sequence into this vector representation, thereby obtaining the location-enhanced feature vector. It is understandable that multiple business data points in a business data sequence are arranged in chronological order. Therefore, the positional information of business data within the business data sequence is crucial. For example, consider a simple temperature data sequence containing the highest temperature records for each day of the week: [10℃, 12℃, 14℃, 13℃, 11℃, 10℃, 9℃]. The vector representations of 10℃ on the first day and 10℃ on the sixth day are the same. However, after adding their respective positional information, the resulting positional enhancement feature vectors (assuming the positional enhancement feature vector corresponding to 10℃ on the first day is t11 and the positional enhancement feature vector corresponding to 10℃ on the sixth day is t12) allow the model to understand that 10℃ appearing on the first day and on the sixth day represent different scenarios. This helps the model understand the temperature trend, such as the temperature rising first and then falling, thus enabling better identification of temporal anomalies.

[0090] Step S102: The location enhancement feature vector matrix is ​​subjected to multi-layer feature extraction and transformation processing through L transformer layers in the temporal feature mining model to obtain intermediate feature representation vector matrices output by the L transformer layers respectively; the L transformer layers have a chain connection relationship; the temporal feature mining model also includes a temporal network layer connected to the L transformer layers respectively.

[0091] Specifically, the temporal feature mining model is used to extract features from location-enhanced feature vectors and further learn and mine temporal features to improve the utilization rate of sequence location information. This temporal feature mining model may include a transformer network layer, which may contain L transformer layers with linked connections. These L transformer layers are used to perform multi-layer feature extraction and transformation processing on the location-enhanced feature vector matrix to obtain an intermediate feature representation vector matrix that reflects the dynamic relationships between various business data in the business data sequence. Here, L is a positive integer. The linked connection relationship means that the transformer layers are connected sequentially in an output-input connection form. The output data of the previous transformer layer becomes the input data of the next transformer layer it connects to. Therefore, each layer can perform deeper feature processing based on the previous layer, achieving deep feature abstraction.

[0092] Specifically, a feasible implementation process for performing multi-layer feature extraction and transformation on the position enhancement feature vector matrix through L transformer layers in the temporal feature mining model to obtain intermediate feature representation vector matrices output by each of the L transformer layers can be as follows: Traverse the L transformer layers according to their chained connection relationship to obtain the j-th transformer layer; j is a positive integer less than or equal to L; perform feature extraction and transformation on the input data through the j-th transformer layer to obtain the intermediate feature representation vector matrix output by the j-th transformer layer, and continue traversing the next connected transformer layer until the intermediate feature representation vector matrices output by each of the L transformer layers are obtained; if j is not 1, the input data is the intermediate feature representation vector matrix output by the (j-1)-th transformer layer; if j is 1, the input data is the position enhancement feature vector matrix. For easier understanding, please refer to the above. Figure 2 The temporal feature mining model 203a shown has three transformer layers that are linked together in a chain. The output of transformer layer 204a is connected to the input of transformer layer 205a, and the output of transformer layer 205a is connected to the input of transformer layer 206a. At this time, the position enhancement feature vector matrix X1 will be used as the input data of transformer layer 204a. After feature extraction and transformation, transformer layer 204a outputs an intermediate feature representation vector matrix T1. This intermediate feature representation vector matrix T1 will be used as the input data of transformer layer 205a, and then transformer layer 205a will perform feature extraction and transformation, and so on.

[0093] In essence, a transformer network layer can be understood as a Transformer-structured model (transformer model). A Transformer is a deep learning model based on a self-attention mechanism, essentially an encoder-decoder structure. A Transformer model structure consists of multiple identical Transformer modules; the number of modules determines the number of layers. In other words, a single Transformer module can be considered a transformer layer. When Transformer modules are stacked in multiple layers (i.e., interconnected), the output data of each layer becomes the input data of the next. Therefore, each layer can perform deeper feature processing based on the previous layer, achieving deep feature abstraction.

[0094] Specifically, a feasible implementation of performing feature extraction and transformation on the input data through the j-th transformer layer to obtain the intermediate feature representation vector matrix output by the j-th transformer layer can be as follows: perform self-attention processing on the input data through the j-th transformer layer to obtain an attention feature vector matrix; perform a first connection normalization process on the attention feature vector matrix to obtain a connection normalized vector matrix; perform feature transformation processing on the connection normalized vector matrix to obtain a feature transformation vector matrix; and perform a second connection normalization process on the feature transformation vector matrix to obtain the intermediate feature representation vector matrix output by the j-th transformer layer.

[0095] The self-attention processing aims to capture the correlations between different elements in the input data from multiple perspectives in parallel. A feasible implementation of self-attention processing on the input data through the j-th transformer layer to obtain the attention feature vector matrix can be as follows: Obtain query weights, key weights, and value weights through the j-th transformer layer; generate a query vector matrix based on the query weights and input data; generate a key vector matrix based on the key weights and input data; generate a value vector matrix based on the value weights and input data; calculate similarity scores on the query vector matrix and key vector matrix to obtain a similarity score matrix; scale the similarity score matrix to obtain a scaled similarity score matrix; normalize the scaled similarity score matrix to obtain an attention weight matrix; multiply the attention weight matrix and the value vector matrix to obtain the attention feature vector matrix. A feasible implementation of calculating similarity scores on the query vector matrix and key vector matrix can be as follows: transpose the key vector matrix to obtain a transposed key vector matrix; multiply the query vector matrix and the transposed key vector matrix to determine the similarity score matrix. One feasible implementation process for scaling the similarity score matrix is ​​as follows: obtain the dimension of the position-enhanced feature vectors; the dimension of the position-enhanced feature vectors refers to the vector dimension of each position-enhanced feature vector contained in the position-enhanced feature vector matrix; divide the scaled similarity score matrix by the arithmetic square root of the dimension of the position-enhanced feature vectors to obtain the scaled similarity score matrix.

[0096] For ease of understanding, the above self-attention processing process can be represented by the following formula (1):

[0097]

[0098] Here, Attention refers to the output attention weight matrix, and Q, K, and V are the query vector matrix, key vector matrix, and value vector matrix, respectively. model This refers to the dimension of the location-enhanced feature vector, QK. T That is, the similarity score matrix mentioned above. This refers to scaling the similarity score matrix, and softmax refers to normalization.

[0099] The definitions of Q, K, and V can be expressed by the following formulas (2)-(4):

[0100]

[0101]

[0102]

[0103] in, This refers to the input data, W Q This refers to the query weight, W. K This refers to the key weight, W V These refer to value weights, and all three weights are learnable parameters.

[0104] The first connection normalization process refers to the connection normalization of the attention feature vector matrix, and the second connection normalization process refers to the connection normalization of the feature transformation vector matrix. The purpose of connection normalization is to improve model stability and enhance model performance. It can involve performing residual connection processing and layer normalization on the input (the attention feature vector matrix in the first connection normalization process, and the feature transformation vector matrix in the second connection normalization process). Residual connection processing involves adding the input to the transformed output; layer normalization involves normalizing the input feature dimension for each sample at each layer.

[0105] The role of feature transformation processing is to further extract and transform the features after the self-attention mechanism. Feature transformation processing can refer to the gradual and complex transformation of the input (i.e., the connection normalized vector matrix) through a combination of linear transformation and nonlinear activation, thereby extracting and transforming features to obtain a feature transformation vector matrix with enhanced representation capabilities.

[0106] Understandably, when the transformer layer is a Transformer module unit, it can consist of a self-attention mechanism (Multi-Head Attention) module, a feed-forward network module, and two residual connection (Residual Connection and Layer Normalization, add-norm) modules. In this case, the input data can be processed by the self-attention mechanism module to obtain the attention feature vector matrix; the attention feature vector matrix can be normalized by the first residual connection module to obtain the connection normalized vector matrix; the connection normalized vector matrix can be transformed by the feed-forward network module to obtain the feature transformation vector matrix; and the feature transformation vector matrix can be normalized by the second residual connection module to obtain the intermediate feature representation vector matrix output by the j-th transformer layer.

[0107] Step S103: Perform time-series feature mining processing on the intermediate feature representation vector matrix output by the connected converter layer through L time-series network layers to obtain merged time-series feature vectors. Perform time-series anomaly identification processing on the merged time-series feature vectors to obtain the time-series anomaly identification result of the business data sequence.

[0108] Specifically, the temporal feature mining model can also contain L temporal network layers, each connected to one of L transformer layers. That is, each transformer layer is followed by a temporal network layer. In this case, the output of the transformer layer is not only input to the next transformer layer but also input to the temporal network layer it is connected to. This structure establishes a direct connection between input sequences according to their order, overcoming the limitations of the transformer layer structure.

[0109] Specifically, a feasible implementation process for obtaining a merged temporal feature vector by performing temporal feature mining on the intermediate feature representation vector matrix output by the connected converter layer through L temporal network layers can be as follows: traverse L temporal network layers to obtain the h-th temporal network layer; h is a positive integer less than or equal to L; determine the h-th intermediate feature representation vector matrix as the intermediate feature representation vector matrix output by the converter layer connected to the h-th temporal network layer; perform temporal feature mining on the h-th intermediate feature representation vector matrix through the h-th temporal network layer to obtain the h-th temporal feature vector; after traversing all L temporal network layers, merge the temporal feature vectors output by the L temporal network layers respectively to obtain the merged temporal feature vector.

[0110] Temporal feature mining refers to discovering valuable, potential patterns, regularities, and temporal features from input data. This can be accomplished using deep learning methods. Therefore, the temporal network layer can be a model employing an LSTM (Long-Short Term Memory) structure. LSTM is a special type of recurrent neural network (RNN) primarily used for processing sequential data, such as text, speech, and time series. LSTM effectively addresses long-term dependency issues, meaning it can better capture the relationships between elements that are far apart in a sequence. Optionally, the temporal network layer can also employ other types of neural network structures for temporal data, such as RNN (Recurrent Neural Network). RNNs have a recurrent structure, allowing them to pass on previously processed information as they process each element in the sequence, thus enabling the modeling of temporal information in the sequential data.

[0111] One feasible implementation for merging the temporal feature vectors output by the L temporal network layers to obtain the merged temporal feature vector can be as follows: In the row direction, according to the arrangement order of the L temporal network layers, the temporal feature vectors output by the L temporal network layers are concatenated sequentially to obtain the merged temporal feature vector. For example, suppose the temporal feature vectors output by the L temporal network layers are y1, y2, ..., y... N Since all are horizontal quantities (i.e., row vectors), the merged time series feature vector can be represented by the following formula (5):

[0112] y output =[y1 y2…y N ] Formula (5)

[0113] Among them, y output That is, the merged temporal feature vector.

[0114] To facilitate understanding of the above time-series feature mining model, please also refer to... Figure 4 , Figure 4 This is a schematic diagram of the model structure of a time-series feature mining model provided in an embodiment of this application. For example... Figure 4As shown, the temporal feature mining model 4a can contain three transformer layers with a chained connection: transformer layer 42a, transformer layer 43a, and transformer layer 44a. It can also contain temporal network layers connected to the three transformer layers: temporal network layer 45a connected to transformer layer 42a, temporal network layer 46a connected to transformer layer 43a, and temporal network layer 47a connected to transformer layer 44a. After input data 41a is input into the temporal feature mining model 4a, it is first input into transformer layer 42a. Then, the output of transformer layer 42a is input into temporal network layers 45a and 43a for further processing. The corresponding outputs are then passed along the arrows shown in the diagram. Finally, after obtaining the outputs of temporal network layers 45a, 46a, and 47a, they can be merged into the output data 48a of the temporal feature mining model 4a.

[0115] As can be seen from the above, in this embodiment of the application, the temporal feature mining model 4a can be composed of a transformer network layer with a Transformer structure and a temporal network layer with an LSTM structure. That is, transformer layer 42a, transformer layer 43a and transformer layer 44a can be a transformer module unit. Therefore, a transformer layer can contain a self-attention mechanism module, a feedforward network module and two residual connection modules (add norm). For example, transformer layer 42a can contain a self-attention mechanism module 421a, a first residual connection module 422a, a feedforward network module 423a and a second residual connection module 424a. The data input to transformer layer 42a will be processed by the self-attention mechanism module 421a, the first residual connection module 422a, the feedforward network module 423a and the second residual connection module 424a in sequence. The data processed by the second residual connection module 424a is used as the output of transformer layer 42a. It is understandable that the data processing procedures of the various converter layers and timing network layers mentioned above can be referred to the relevant descriptions above, and will not be repeated here.

[0116] Furthermore, after obtaining the merged temporal feature vector with rich temporal features, corresponding temporal anomaly identification processing can be performed according to different temporal anomaly identification tasks to obtain the temporal anomaly identification results of the business data sequence. It can be understood that temporal anomaly identification tasks are usually pre-defined tasks. For example, in equipment maintenance scenarios, for the operating data sequence corresponding to industrial equipment, the temporal anomaly identification task could be to identify whether the operating status of the industrial equipment is abnormal. For different temporal anomaly identification tasks, corresponding temporal anomaly identification models can be pre-built and trained. These models can be models using fully connected network structures or models using other types of machine learning or deep learning methods; this application does not impose any restrictions on this.

[0117] To facilitate understanding, a detailed explanation of the time-series anomaly detection process using a fully connected network structure in a time-series anomaly detection model is provided as an example. A feasible implementation process for processing the merged time-series feature vectors to obtain the time-series anomaly detection result for the business data sequence can be as follows: Obtain a time-series anomaly detection model; the time-series anomaly detection model includes fully connected layers and classification layers; perform feature extraction processing on the merged time-series feature vectors through the fully connected layers to obtain intermediate feature vectors; perform classification processing on the intermediate feature vectors through the classification layer to obtain classification results; and determine the time-series anomaly detection result for the business data sequence based on the classification results. Here, the fully connected layer can refer to a fully-connected neural network layer (FC), and there can be multiple FC layers. These layers are used to perform linear transformations and non-linear activations on the input features, fusing feature information from different dimensions. After the output of the last fully connected layer, a classification layer can be connected. This classification layer is used to obtain the final output result (i.e., the classification result mentioned above) through an appropriate activation function, depending on the time-series anomaly detection task.

[0118] For ease of understanding, please refer to the following: Figure 5 , Figure 5 This is a schematic diagram of the structure of a timing anomaly identification model provided in an embodiment of this application. For example... Figure 5 As shown, the temporal anomaly detection model includes fully connected layers 51a, 52a, 53a, and a classification layer 54a, which merges temporal feature vectors (such as...). Figure 5 As shown in Figure 511a, the input to the time anomaly detection model will sequentially pass through fully connected layers 51a, 52a, 53a, and classification layer 54a. Finally, the time anomaly detection model 5a will output a classification result dependent on the time anomaly detection task (e.g., ...). Figure 5 (As shown in 512a).

[0119] To facilitate understanding that the output format of the classification results depends on the time-series anomaly identification task, we assume that the time-series anomaly identification task is a regression task. In this case, the activation function can be a linear function, and the output can be a single numerical value (a one-dimensional vector). For multi-class tasks, the activation function can be a softmax function, and the output is a multi-class vector after softmax. For example, the time-series anomaly identification task could be to identify whether the operation of industrial equipment is abnormal. After obtaining the operating sequence data of the industrial equipment, after processing it through steps S101-S103 to obtain the merged time-series feature vector of the operating sequence data, and then processing it through the time-series anomaly identification model, the resulting classification result can be [0,1]. The corresponding output of the time-series anomaly identification model has two categories: [normal, abnormal]. Therefore, based on the classification result, the time-series anomaly identification result of the industrial equipment can be determined as an abnormal equipment operation result.

[0120] Optionally, the time-series anomaly detection model can be trained using standard neural network training methods, specifically backpropagation training. Taking a binary classification task as an example, the label is whether a sample is anomaly; anomaly samples are labeled 1, and normal samples are labeled 0. The network parameters can be learned by optimizing the following binary classification objective function L:

[0121]

[0122] Where S is the total number of samples, p i It is the output of the time-series anomaly detection model for sample i, y i It is the model's label for sample i.

[0123] Optionally, after the time-series anomaly identification model has been trained, its performance can be evaluated using preset evaluation metrics to determine whether it has achieved the expected results. These evaluation metrics may include IV (Information Value), AUC (Area Under the Curve), or KS (Kolmogorov–Smirnov statistic), etc., and this application does not impose any restrictions on these metrics.

[0124] The method provided in this application embodiment can learn and mine the temporal features of business data sequences by introducing a temporal network layer to each transformer layer, thereby effectively improving the utilization rate of sequence position information and discovering richer temporal features, and thus effectively improving the accuracy of temporal anomaly identification.

[0125] Further, please see Figure 6 , Figure 6 This is a flowchart illustrating a timing anomaly identification method provided in an embodiment of this application. The method can be implemented using a computer device (e.g., the one described above). Figure 1The server 2000 in the corresponding embodiment executes the method. The following description uses the execution of this method by a computer device as an example. The timing anomaly identification method may include at least the following steps S201-S207:

[0126] Step S201: Obtain a service data sequence with a temporal relationship; the service data sequence includes service data B. i , where i is a positive integer less than or equal to the number of business data contained in the business data sequence.

[0127] Specifically, business data B i It refers to the i-th business data in the business data sequence, business data B. i It can contain word segmentation attribute data on one or more data attributes (i.e., the data corresponding to a data attribute). For example, in the weather forecasting scenario, the data attributes of business data can include timestamp information, temperature information of the business object (a certain place), precipitation information, humidity information, wind speed and wind direction information.

[0128] Step S202, for the business data B i Data feature extraction processing is performed to obtain the business data B. i The corresponding data feature vector.

[0129] Specifically, regarding business data B i The purpose of performing data feature extraction processing is to transform business data B i The features are converted into vector form so that they can be understood by the model. Data feature extraction and processing can employ methods such as word embedding and numerical normalization, depending on the type of business data to be processed. This application does not impose any restrictions on these methods.

[0130] Specifically, when using word embedding methods for data feature extraction and processing, for business data B... i Perform data feature extraction processing to obtain business data B. i A feasible implementation process for the corresponding data feature vector can be as follows: For business data B i By performing embedded feature vector extraction, business data B is obtained. i The corresponding embedded feature vector. Among them, the extraction of the embedded feature vector can be used to extract business data B. i Each segmentation attribute data in the data is converted into a corresponding vector representation, thus obtaining the embedded feature vector. For example, business data B iThe data includes word segmentation attribute data on data attributes s1, s2, s3, and s4. The word segmentation attribute data on data attribute s1 can be converted into vector v1, the word segmentation attribute data on data attribute s2 can be converted into vector v2, the word segmentation attribute data on data attribute s3 can be converted into vector v3, and the word segmentation attribute data on data attribute s4 can be converted into vector v4. Then, the embedding feature vector can be determined based on vectors v1-v4. For example, vectors v1-v4 can be directly combined (e.g., each vector is combined as a row of data in a matrix), and the resulting matrix is ​​determined as the embedding feature vector. Alternatively, vectors v1-v5 can be combined, and the resulting matrix can be reduced in dimensionality to obtain the embedding feature vector.

[0131] Step S203, for the business data B i The location in the business data sequence is subjected to position encoding processing to obtain the business data B. i The corresponding positional encoding vector.

[0132] Specifically, regarding business data B i Position encoding is performed on the position in the business data sequence to obtain business data B. i A feasible implementation process for the corresponding positional encoding vector can be as follows: obtain the feature vector dimension of the feature vector, generate an initial vector with the same vector dimension as the feature vector dimension; based on business data B... i Sine coding is performed on the position and feature vector dimension in the business data sequence to obtain the even-numbered position coding value; based on business data B i Cosine coding is performed on the position and feature vector dimension in the business data sequence to obtain the code value for odd-numbered positions. The element values ​​at even-numbered positions in the initial vector are replaced with the code values ​​at even-numbered positions, and the element values ​​at odd-numbered positions in the initial vector are replaced with the code values ​​at odd-numbered positions, resulting in business data B. i The corresponding positional encoding vector.

[0133] The process of calculating the sine code can be represented by the following formula (7):

[0134]

[0135] Where the feature vector dimension is d m Business data B i The position of 'a' in the business data sequence is 'pos', and its value ranges from 0 to n, where n is the total number of business data items contained in the business data sequence. The value of 'a' ranges from 0 to 'd'. m / 2-1.

[0136] The process of calculating the sine code can be represented by the following formula (8):

[0137]

[0138] Where the feature vector dimension is d m Business data B i The position of 'a' in the business data sequence is 'pos', and its value ranges from 0 to n, where n is the total number of business data items contained in the business data sequence. The value of 'a' ranges from 0 to 'd'. m / 2-1.

[0139] The final business data B i The corresponding positional encoding vector can be represented by the following formula (9):

[0140]

[0141] Where P represents business data B i The corresponding positional encoding vector.

[0142] Step S204, for the business data B i The corresponding data feature vector and the business data B i The corresponding location encoding vectors are fused to obtain the service data B. i The corresponding position-enhanced feature vector E i .

[0143] Specifically, regarding business data B i The corresponding data feature vector and business data B i The corresponding location encoding vectors are fused to obtain business data B. i The corresponding position-enhanced feature vector E i A feasible implementation process can be as follows: In the row direction, transfer business data B i The corresponding position encoding vector is concatenated in business data B i The business data B is obtained from the corresponding data feature vector. i The corresponding position-enhanced feature vector E i For ease of understanding, the splicing process can be represented by the following formula (10):

[0144] E i =[X i P i ] Formula (10)

[0145] Among them, E i Refers to business data B i The corresponding location-enhanced feature vector, X i This is business data B. i The corresponding data feature vector, Pi Business Data B i The corresponding positional encoding vector. It should be understood that X i With P i The vector dimension of E is the feature vector dimension. i The vector dimension is the same as the position enhancement feature vector dimension, and the relationship between the two can be expressed by the following formula (11):

[0146] d model =2d m Formula (11)

[0147] Where the feature vector dimension is d m The dimension of the location-enhanced feature vector is d. model .

[0148] Step S205: When the location enhancement feature vector corresponding to each business data in the business data sequence is obtained, a location enhancement feature vector matrix is ​​generated based on the location enhancement feature vector corresponding to each business data.

[0149] Specifically, the final generated location-enhanced feature vector matrix can be represented by the following formula (12):

[0150]

[0151] in, This is the final generated location-enhanced feature vector matrix, which is the input data described in formulas (2)-(4) above. E1 refers to the location-enhanced feature vector corresponding to the first business data in the business data sequence, ..., E m This refers to the enhanced feature vector corresponding to the m-th business data in the business data sequence. At this point, the business data sequence contains a total of m business data.

[0152] Step S206: The location enhancement feature vector matrix is ​​subjected to multi-layer feature extraction and transformation processing through L transformer layers in the temporal feature mining model to obtain intermediate feature representation vector matrices output by the L transformer layers respectively; the L transformer layers have a chain connection relationship; the temporal feature mining model also includes a temporal network layer connected to the L transformer layers respectively.

[0153] Step S207: Perform time-series feature mining processing on the intermediate feature representation vector matrix output by the connected converter layer through L time-series network layers to obtain merged time-series feature vectors. Perform time-series anomaly identification processing on the merged time-series feature vectors to obtain the time-series anomaly identification result of the service data sequence.

[0154] Specifically, the implementation process of steps S206-S207 can be found above. Figure 3 The specific descriptions of steps S102-S103 in the corresponding embodiments will not be repeated here.

[0155] The method provided in this application concatenates the location encoding vector corresponding to the business data with the data feature vector, preserving the original location encoding vector, which is equivalent to preserving the original location information. This is beneficial for the subsequent model to mine the location information. At the same time, the original feature vector is not affected by the fusion, which can better mine the original features. The concatenated location-enhanced feature vector matrix is ​​introduced into the temporal feature mining model of the temporal network layer, which can better learn and mine the temporal features of the business data sequence, making full use of the sequence location information and providing an effective solution for various temporal anomaly identification and analysis tasks.

[0156] Further, please see Figure 7 , Figure 7 This is a schematic diagram illustrating the overall process of training a temporal anomaly recognition model provided in an embodiment of this application. For example... Figure 7 As shown, the overall process of training a time-series anomaly detection model can include the following steps:

[0157] Step S71: Obtain the sample business data sequence.

[0158] Specifically, the sample business data sequence is a business data sequence with anomaly labels obtained in advance. The number of such sequences can be determined based on the training accuracy, and this application does not impose any restrictions on this.

[0159] Step S72: Determine the original feature vector matrix corresponding to the sample business data sequence.

[0160] Specifically, the original feature vector matrix refers to the matrix composed of the data feature vectors corresponding to the sample business data contained in the sample business data sequence. The process of determining the data feature vectors corresponding to the sample business data contained in the sample business data sequence can be found above. Figure 6 The relevant descriptions of step S202 in the corresponding embodiments will not be repeated here.

[0161] Step S73: Calculate the location encoding vector matrix corresponding to the sample business data sequence.

[0162] Specifically, the location coding vector matrix refers to the matrix composed of the location coding vectors corresponding to the sample service data contained in the sample service data sequence. The process of determining the location coding vectors can be found above. Figure 6 The relevant descriptions of step S203 in the corresponding embodiments will not be repeated here.

[0163] Step S74: Concatenate the original feature vector matrix and the position encoding vector matrix to obtain the position-enhanced feature vector matrix.

[0164] Specifically, concatenation refers to horizontally concatenating the position encoding vector matrix after the original feature vector matrix.

[0165] Step S75: Input the location-enhanced feature vector matrix into the temporal feature mining model to obtain the merged temporal feature vector.

[0166] Specifically, the temporal feature mining model can include a standard multi-layer transformer model, and for the output of each transformer layer, an LSTM layer is further connected to perform temporal feature mining. The specific model structure can be found above. Figure 4 The structure and processing of the temporal feature mining model in the corresponding embodiment can be found above. Figure 3 The relevant descriptions of steps S102-S103 in the corresponding embodiments will not be repeated here.

[0167] Step S76: Construct an initial time-series anomaly identification model based on the output features.

[0168] Specifically, the construction of the initial time-series anomaly detection model is determined by the different time-series anomaly detection tasks. A fully connected neural network structure can be selected to construct the initial time-series anomaly detection model. The output is also determined by the task type of the time-series anomaly detection task. For regression tasks, the output can be a single numerical value (one-dimensional vector); for multi-class tasks, the output is a multi-class vector after softmax.

[0169] Step S77: Backpropagation training of the neural network.

[0170] Specifically, the initial time-series anomaly identification model can be trained using standard neural network training methods, and this application does not impose any restrictions on this.

[0171] Step S78: Evaluate the model training effect.

[0172] Specifically, if the model performs as expected, it can be deployed and used for actual prediction tasks; if the model does not perform as expected, it is necessary to return to step S77 to retrain the model.

[0173] Furthermore, after the time-series anomaly detection model is trained, it can be applied to real-world business applications, such as anomaly user discovery in payment scenarios, and detection of extreme weather events or pollution sources in environmental monitoring. It can be used to identify and detect anomalous objects (e.g., abnormal payment recipients) and to issue alerts for potential anomalies. For easier understanding, please refer to [link to relevant documentation]. Figure 8 , Figure 8This is a schematic diagram illustrating the application process of a timing anomaly identification model provided in an embodiment of this application. For example... Figure 8 As shown, the entire application process includes the following steps:

[0174] Deployment phase:

[0175] Step S81: Deploy the model to the online system.

[0176] Specifically, a feasible implementation process for deploying the model to an online system can be as follows: deploy the structures and corresponding trained model parameters of the two neural network models, namely the first part, the temporal feature mining network (i.e., the aforementioned temporal feature mining model) and the second part, the fully connected neural network model for the target task (i.e., the aforementioned temporal anomaly recognition model), to the online system for subsequent recognition and prediction stages.

[0177] Identification and prediction phase:

[0178] Step S82: Acquire raw time series data.

[0179] Specifically, for a given object to be inferred and predicted (such as a payment user or historical weather data), its original time-series data is obtained, i.e., its historical time-series data, to generate the original time series data (i.e., the aforementioned). Figure 3 (The business data sequence shown).

[0180] Step S83: Calculate the position encoding vector.

[0181] Specifically, according to the above Figure 6 The method described in step S203 of the corresponding embodiment calculates the position encoding vector corresponding to each position in the sequence to obtain the corresponding position encoding matrix P. ’ .

[0182] Step S84: The position encoding vector and the data feature vector are merged to obtain the input data.

[0183] Specifically, according to the above Figure 6 The method described in step S202 of the corresponding embodiment can calculate the data feature vector corresponding to each element in the sequence to obtain the corresponding original time-series feature matrix X. ’ Then, after concatenating them, we can obtain the model's input data, that is, the model's input data = [original time series feature matrix X ... ’ Position encoding matrix P ’ ].

[0184] Step S85: Perform time series anomaly identification on the input data using the time series anomaly identification model to obtain the time series anomaly identification result.

[0185] Specifically, the assembled input data is fed into the time-series anomaly detection model deployed on the system for inference and prediction calculations. After the model completes the calculation, it can output a regression value or a classification probability vector based on the type of the target task. Taking binary classification as an example, the model output is a two-dimensional probability vector between 0 and 1, representing the probability of each class; the higher the probability value, the more likely the sample belongs to that class.

[0186] Abnormal output phase:

[0187] Step S86: Issue an anomaly alert based on the timing anomaly identification results.

[0188] Specifically, in real-time online systems, based on the results of time-series anomaly identification, corresponding alarms or reminders are output for detected abnormal objects. The methods vary depending on the task. Taking the identification of abnormal payees in a payment scenario as an example, if the model's output predicts that the payee has obvious anomalies, the system can take action against that payee. Specifically, the system can remind or block users who have made payments to that payee, informing them of the transaction risks and preventing financial losses; or it can restrict the payee's ability to receive payments, thereby reducing the risk of fraud and losses for the platform and users.

[0189] The method provided in this application no longer simply adds the positional encoding to the feature vector, but concatenates the positional encoding and feature vector, and then extracts features using a Transformer model. Subsequently, a Long Short-Term Memory (LSTM) structure is introduced for each Transformer layer to further learn and mine temporal features, thereby effectively improving the utilization rate of sequence positional information and discovering richer temporal features, thus effectively improving the accuracy of temporal anomaly identification. In summary, this application, while maintaining the superior performance of the Transformer model, fully utilizes sequence positional information by improving the temporal feature mining method, providing an effective solution for various temporal anomaly identification and analysis tasks.

[0190] Please see Figure 9 , Figure 9 This is a schematic diagram of a timing anomaly identification device provided in an embodiment of this application. The timing anomaly identification device can be a computer program (including program code) running on a computer device; for example, the timing anomaly identification device is an application software. The timing anomaly identification device 1 can be used to execute corresponding steps in the timing anomaly identification method provided in the embodiments of this application. Figure 9 As shown, the time series anomaly identification device 1 may include: an extraction module 110, an attention module 120, a time series feature mining module 130, and a time series anomaly identification module 140.

[0191] Extraction module 110 is used to acquire business data sequences with temporal relationships, perform location enhancement feature extraction processing on the business data sequences, and obtain the location enhancement feature vector matrix corresponding to the business data sequences;

[0192] Attention module 120 is used to perform multi-layer feature extraction and transformation on the position enhancement feature vector matrix through L transformer layers in the temporal feature mining model to obtain intermediate feature representation vector matrices output by the L transformer layers respectively; L is a positive integer; the L transformer layers have a chain connection relationship; the temporal feature mining model also includes a temporal network layer connected to the L transformer layers respectively;

[0193] The temporal feature mining module 130 is used to perform temporal feature mining processing on the intermediate feature representation vector matrix output by the connected transformer layer through L temporal network layers to obtain the merged temporal feature vector.

[0194] The timing anomaly identification module 140 is used to perform timing anomaly identification processing on the merged timing feature vector to obtain the timing anomaly identification result of the business data sequence.

[0195] The business data sequence includes business data B. i The location-enhanced feature vector matrix contains business data B. i The corresponding position-enhanced feature vector E i i is a positive integer less than or equal to the number of business data contained in the business data sequence;

[0196] In one possible implementation, the extraction module 110 is used to perform location enhancement feature extraction processing on the business data sequence. When obtaining the location enhancement feature vector matrix corresponding to the business data sequence, it is specifically used to perform the following operations:

[0197] For business data B i Perform data feature extraction processing to obtain business data B. i The corresponding data feature vector;

[0198] For business data B i Position encoding is performed on the position in the business data sequence to obtain business data B. i The corresponding positional encoding vector;

[0199] For business data B i The corresponding data feature vector and business data B i The corresponding location encoding vectors are fused to obtain business data B. i The corresponding position-enhanced feature vector E i ;

[0200] When the location-enhanced feature vector corresponding to each business data in the business data sequence is obtained, a location-enhanced feature vector matrix is ​​generated based on the location-enhanced feature vector corresponding to each business data.

[0201] In one possible implementation, the extraction module 110 is used to process business data B. i Position encoding is performed on the position in the business data sequence to obtain business data B. i When encoding the corresponding position vector, it is specifically used to perform the following operations:

[0202] Obtain the feature vector dimension of the feature vector, and generate an initial vector with the same vector dimension as the feature vector dimension;

[0203] According to business data B i Sine coding is performed on the position and feature vector dimension in the business data sequence to obtain the even-numbered position coding value;

[0204] According to business data B i Cosine coding is performed on the position and feature vector dimension in the business data sequence to obtain the odd position coding value;

[0205] Replace the even-numbered elements in the initial vector with their coded values, and replace the odd-numbered elements with their coded values ​​to obtain business data B. i The corresponding positional encoding vector.

[0206] In one possible implementation, the extraction module 110 is used to process business data B. i The corresponding data feature vector and business data B i The corresponding location encoding vectors are fused to obtain business data B. i The corresponding position-enhanced feature vector E i Specifically, it is used to perform the following operations:

[0207] In the row direction, business data B i The corresponding position encoding vector is concatenated in business data B i The business data B is obtained from the corresponding data feature vector. i The corresponding position-enhanced feature vector E i .

[0208] In one possible implementation, when the attention module 120 performs multi-layer feature extraction transformation on the position enhancement feature vector matrix through L transformer layers in the temporal feature mining model to obtain the intermediate feature representation vector matrices output by the L transformer layers respectively, it is specifically used to perform the following operations:

[0209] Based on the chained connection relationship of L transformer layers in the time series feature mining model, traverse the L transformer layers to obtain the j-th transformer layer; j is a positive integer less than or equal to L;

[0210] The input data is processed by feature extraction and transformation through the j-th transformer layer to obtain the intermediate feature representation vector matrix output by the j-th transformer layer. This process continues to traverse the next connected transformer layer until the intermediate feature representation vector matrices output by L transformer layers are obtained respectively. If j is not 1, the input data is the intermediate feature representation vector matrix output by the (j-1)-th transformer layer. If j is 1, the input data is the position enhancement feature vector matrix.

[0211] In one possible implementation, when the attention module 120 performs feature extraction and transformation on the input data through the j-th transformer layer to obtain the intermediate feature representation vector matrix output by the j-th transformer layer, it specifically performs the following operations:

[0212] The input data is processed by self-attention through the j-th transformer layer to obtain the attention feature vector matrix;

[0213] The attention feature vector matrix is ​​subjected to the first connection normalization process to obtain the connection normalized vector matrix;

[0214] The eigenvalue transformation is performed on the connection normalized vector matrix to obtain the eigenvalue transformation vector matrix;

[0215] The feature transformation vector matrix is ​​subjected to a second connection normalization process to obtain the intermediate feature representation vector matrix output by the j-th transformer layer.

[0216] In one possible implementation, when the attention module 120 performs self-attention processing on the input data through the j-th transformer layer to obtain the attention feature vector matrix, it specifically performs the following operations:

[0217] The query weight, key weight, and value weight are obtained through the j-th transformer layer.

[0218] Generate a query vector matrix based on query weights and input data, generate a key vector matrix based on key weights and input data, and generate a value vector matrix based on value weights and input data.

[0219] Calculate the similarity score between the query vector matrix and the key vector matrix to obtain the similarity score matrix;

[0220] The similarity score matrix is ​​scaled to obtain the scaled similarity score matrix;

[0221] The scaling similarity score matrix is ​​normalized to obtain the attention weight matrix;

[0222] Multiplying the attention weight matrix and the value vector matrix yields the attention feature vector matrix.

[0223] In one possible implementation, the attention module 120 is used to scale the similarity score matrix. Specifically, when obtaining the scaled similarity score matrix, it performs the following operations:

[0224] Obtain the dimension of the location-enhanced feature vector; the dimension of the location-enhanced feature vector refers to the vector dimension of each location-enhanced feature vector contained in the location-enhanced feature vector matrix;

[0225] The scaled similarity score matrix is ​​obtained by dividing the scaled similarity score matrix by the square root of the dimension of the position-enhanced feature vector.

[0226] In one possible implementation, the temporal feature mining module 130 is used to perform temporal feature mining processing on L intermediate feature representation vector matrices through L temporal network layers to obtain a merged temporal feature vector. Specifically, it is used to perform the following operations:

[0227] Traverse the L temporal network layers to obtain the h-th temporal network layer; h is a positive integer less than or equal to L;

[0228] The intermediate feature representation vector matrix output by the transformer layer connected by the h-th temporal network layer is determined as the h-th intermediate feature representation vector matrix among the L intermediate feature representation vector matrices.

[0229] Temporal feature mining is performed on the h-th intermediate feature representation vector matrix through the h-th temporal network layer to obtain the h-th temporal feature vector;

[0230] After traversing all L temporal network layers, the temporal feature vectors output by the L temporal network layers are merged to obtain the merged temporal feature vector.

[0231] In one possible implementation, the temporal feature mining module 130 is used to merge the temporal feature vectors output by the L temporal network layers respectively. When obtaining the merged temporal feature vector, it is specifically used to perform the following operations:

[0232] In the row direction, according to the arrangement order of the L temporal network layers, the temporal feature vectors output by the L temporal network layers are concatenated in order to obtain the merged temporal feature vector.

[0233] In one possible implementation, the timing anomaly detection module 140 is used to perform timing anomaly detection processing on the merged timing feature vector. When obtaining the timing anomaly detection result of the business data sequence, it is specifically used to perform the following operations:

[0234] Obtain the time series anomaly detection model; the time series anomaly detection model includes a fully connected layer and a classification layer;

[0235] The merged temporal feature vector is processed by a fully connected layer to obtain an intermediate feature vector;

[0236] The intermediate feature vectors are classified by a classification layer to obtain the classification result. The result of the time-series anomaly identification of the business data sequence is determined based on the classification result.

[0237] Please see Figure 10 , Figure 10 This is a schematic diagram of the structure of a computer device provided in an embodiment of this application. Figure 10 As shown above, Figure 9 The timing anomaly identification device 1 in the corresponding embodiment can be applied to a computer device 1000, which may include a processor 1001, a network interface 1004, and a memory 1005. Furthermore, the computer device 1000 may also include a user interface 1003 and at least one communication bus 1002. The communication bus 1002 is used to enable communication between these components. The user interface 1003 may include a display screen and a keyboard; optionally, the user interface 1003 may also include a standard wired interface or a wireless interface. The network interface 1004 may optionally include a standard wired interface or a wireless interface (such as a Wi-Fi interface). The memory 1005 may be high-speed RAM or non-volatile memory, such as at least one disk storage device. Optionally, the memory 1005 may also be at least one storage device located remotely from the processor 1001. Figure 10 As shown, the memory 1005, which is a computer-readable storage medium, may include an operating system, a network communication module, a user interface module, and a device control application.

[0238] In such Figure 10 In the computer device 1000 shown, the network interface 1004 provides network communication elements; the user interface 1003 is mainly used to provide an input interface for the user; and the processor 1001 can be used to call the device control application stored in the memory 1005 to achieve:

[0239] Obtain a business data sequence with temporal relationship, perform location enhancement feature extraction processing on the business data sequence, and obtain the location enhancement feature vector matrix corresponding to the business data sequence;

[0240] The temporal feature mining model uses L transformer layers to perform multi-layer feature extraction and transformation on the position enhancement feature vector matrix, resulting in intermediate feature representation vector matrices output by the L transformer layers respectively. The L transformer layers have a chain-like connection relationship. The temporal feature mining model also includes a temporal network layer that is connected to the L transformer layers respectively.

[0241] The intermediate feature representation vector matrix output by the connected transformer layer is processed by L time-series network layers to obtain merged time-series feature vectors. Time-series anomaly identification is then performed on the merged time-series feature vectors to obtain the time-series anomaly identification results of the business data sequence.

[0242] It should be understood that the computer device 1000 described in the embodiments of this application can execute the foregoing text. Figure 3 The description of the timing anomaly identification method in any corresponding embodiment will not be repeated here. Furthermore, the beneficial effects of using the same method will also not be repeated.

[0243] Furthermore, it should be noted that this application embodiment also provides a computer-readable storage medium, which stores a computer program executed by the timing anomaly identification device 1 mentioned above. The computer program includes program instructions, and when the processor executes the program instructions, it can execute the aforementioned... Figure 3 , Figure 6 The description of the timing anomaly identification method in any corresponding embodiment is already provided, and therefore will not be repeated here. Furthermore, the beneficial effects of using the same method will also not be repeated. For technical details not disclosed in the computer-readable storage medium embodiments related to this application, please refer to the description of the method embodiments of this application.

[0244] The aforementioned computer-readable storage medium can be the timing anomaly identification device provided in any of the foregoing embodiments or the internal storage unit of the aforementioned computer device, such as the hard disk or memory of the computer device. The computer-readable storage medium can also be an external storage device of the computer device, such as a plug-in hard disk, smart media card (SMC), secure digital (SD) card, flash card, etc., provided on the computer device. Furthermore, the computer-readable storage medium can include both internal storage units and external storage devices of the computer device. The computer-readable storage medium is used to store the computer program and other programs and data required by the computer device. The computer-readable storage medium can also be used to temporarily store data that has been output or will be output.

[0245] Furthermore, it should be noted that this application also provides a computer program product or computer program, which includes computer instructions stored in a computer-readable storage medium. The processor of a computer device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the computer device to perform the aforementioned... Figure 3 , Figure 6 The method provided in any of the corresponding embodiments.

[0246] The terms "first," "second," etc., in the specification, claims, and drawings of this application are used to distinguish different objects, not to describe a specific order. Furthermore, the term "comprising," and any variations thereof, are intended to cover non-exclusive inclusion. For example, a process, method, apparatus, product, or device that includes a series of steps or units is not limited to the listed steps or modules, but may optionally include steps or modules not listed, or may optionally include other step units inherent to these processes, methods, apparatuses, products, or devices.

[0247] In this application embodiment, the terms "module" or "unit" refer to a computer program or part of a computer program that has a predetermined function and works with other related parts to achieve a predetermined goal, and can be implemented wholly or partially using software, hardware (such as processing circuitry or memory), or a combination thereof. Similarly, a processor (or multiple processors or memory) can be used to implement one or more modules or units. Furthermore, each module or unit can be part of an overall module or unit that includes the functionality of that module or unit.

[0248] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of both. To clearly illustrate the interchangeability of hardware and software, the composition and steps of each example have been generally described in the foregoing description as a network element. Whether these network elements are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can implement the described network elements using different methods for each specific application, but such implementation should not be considered beyond the scope of this application.

[0249] The above-disclosed embodiments are merely preferred embodiments of this application and should not be construed as limiting the scope of this application. Therefore, any equivalent variations made in accordance with the claims of this application shall still fall within the scope of this application.

Claims

1. A method for identifying temporal anomalies, characterized in that, include: Obtain a business data sequence with temporal relationship, perform location enhancement feature extraction processing on the business data sequence, and obtain the location enhancement feature vector matrix corresponding to the business data sequence; The location enhancement feature vector matrix is ​​subjected to multi-layer feature extraction and transformation processing through L transformer layers in the temporal feature mining model to obtain intermediate feature representation vector matrices output by the L transformer layers respectively; L is a positive integer; the L transformer layers have a chain connection relationship; the temporal feature mining model also includes a temporal network layer connected to the L transformer layers respectively; The intermediate feature representation vector matrix output by the connected transformer layer is processed by L time-series network layers to obtain merged time-series feature vectors. The merged time-series feature vectors are then processed for time-series anomaly identification to obtain the time-series anomaly identification result of the business data sequence.

2. The method according to claim 1, characterized in that, The business data sequence includes business data B. i The location-enhanced feature vector matrix contains the service data B. i The corresponding position-enhanced feature vector E i i is a positive integer less than or equal to the number of business data contained in the business data sequence; The step of performing location enhancement feature extraction processing on the business data sequence to obtain the location enhancement feature vector matrix corresponding to the business data sequence includes: performing data feature extraction processing on the service data B i to obtain a corresponding data feature vector of the service data B i ​ The service data B i The position coding vector corresponding to the position of the service data B i The position coding vector corresponding to the position of the service data B The business data B i The corresponding data feature vector and the business data B i The corresponding position encoding vector is fused to obtain the business data B i The corresponding position enhanced feature vector E i ; When the location enhancement feature vector corresponding to each business data in the business data sequence is obtained, a location enhancement feature vector matrix is ​​generated based on the location enhancement feature vector corresponding to each business data.

3. The method according to claim 2, characterized in that, The position encoding processing is performed on the position in the service data sequence to obtain the service data B i The position encoding processing is performed on the position in the service data sequence to obtain the service data B i The corresponding position encoding vector comprises: Obtain the feature vector dimension of the feature vector, and generate an initial vector with the same vector dimension as the feature vector dimension; According to the service data B i The sine coding calculation is performed on the position in the service data sequence and the feature vector dimension to obtain an even position coding value. According to the service data B i The cosine coding calculation is performed on the position in the service data sequence and the feature vector dimension, and an odd position coding value is obtained. replacing the element value at the even position in the initial vector with the even position encoding value and replacing the element value at the odd position in the initial vector with the odd position encoding value, to obtain the service data B i a corresponding position encoding vector.

4. The method according to claim 2, characterized in that, The business data B i The corresponding data feature vector and the business data B i The corresponding location encoding vectors are fused to obtain the service data B. i The corresponding position-enhanced feature vector E i ,include: In the row direction, the business data B i The corresponding location encoding vector is concatenated into the service data B i The business data B is obtained from the corresponding data feature vector. i The corresponding position-enhanced feature vector E i .

5. The method according to claim 1, characterized in that, The step involves performing multi-layer feature extraction and transformation on the position-enhanced feature vector matrix through L transformer layers in the temporal feature mining model to obtain intermediate feature representation vector matrices output by the L transformer layers, including: Based on the chained connection relationship of the L transformer layers in the time series feature mining model, the j-th transformer layer is obtained by traversing the L transformer layers; j is a positive integer less than or equal to L. The input data is processed by feature extraction and transformation through the j-th transformer layer to obtain the intermediate feature representation vector matrix output by the j-th transformer layer. This process continues to traverse the next connected transformer layer until the intermediate feature representation vector matrices output by the L transformer layers are obtained respectively. If j is not 1, the input data is the intermediate feature representation vector matrix output by the (j-1)-th transformer layer. If j is 1, the input data is the position enhancement feature vector matrix.

6. The method according to claim 5, characterized in that, The step of performing feature extraction and transformation on the input data through the j-th transformer layer to obtain the intermediate feature representation vector matrix output by the j-th transformer layer includes: The input data is processed by self-attention through the j-th transformer layer to obtain the attention feature vector matrix; The attention feature vector matrix is ​​subjected to a first connection normalization process to obtain a connection normalized vector matrix; The connection normalized vector matrix is ​​subjected to feature transformation processing to obtain the feature transformation vector matrix; The feature transformation vector matrix is ​​subjected to a second connection normalization process to obtain the intermediate feature representation vector matrix output by the j-th transformer layer.

7. The method according to claim 6, characterized in that, The step of performing self-attention processing on the input data through the j-th transformer layer to obtain the attention feature vector matrix includes: The query weight, key weight, and value weight are obtained through the j-th transformer layer. A query vector matrix is ​​generated based on the query weights and input data; a key vector matrix is ​​generated based on the key weights and input data; and a value vector matrix is ​​generated based on the value weights and input data. A similarity score is calculated on the query vector matrix and the key vector matrix to obtain a similarity score matrix; The similarity score matrix is ​​scaled to obtain a scaled similarity score matrix; The scaling similarity score matrix is ​​normalized to obtain the attention weight matrix; Multiplying the attention weight matrix and the value vector matrix yields the attention feature vector matrix.

8. The method according to claim 7, characterized in that, The scaling process of the similarity score matrix to obtain the scaled similarity score matrix includes: Obtain the dimension of the location-enhanced feature vector; the dimension of the location-enhanced feature vector refers to the vector dimension of each location-enhanced feature vector contained in the location-enhanced feature vector matrix; The scaled similarity score matrix is ​​obtained by dividing the scaled similarity score matrix by the square root of the dimension of the position-enhanced feature vector.

9. The method according to claim 1, characterized in that, The process of performing temporal feature mining on L intermediate feature representation vector matrices through L temporal network layers to obtain merged temporal feature vectors includes: Traverse the L temporal network layers to obtain the h-th temporal network layer; h is a positive integer less than or equal to L; The intermediate feature representation vector matrix output by the converter layer connected to the h-th temporal network layer is determined as the h-th intermediate feature representation vector matrix among the L intermediate feature representation vector matrices. Temporal feature mining is performed on the h-th intermediate feature representation vector matrix through the h-th temporal network layer to obtain the h-th temporal feature vector; When all L temporal network layers have been traversed, the temporal feature vectors output by the L temporal network layers are merged to obtain merged temporal feature vectors.

10. The method according to claim 9, characterized in that, The process of merging the temporal feature vectors output by the L temporal network layers to obtain merged temporal feature vectors includes: In the row direction, according to the arrangement order of the L temporal network layers, the temporal feature vectors output by the L temporal network layers are sequentially concatenated to obtain the merged temporal feature vector.

11. The method according to claim 1, characterized in that, The step of performing time-series anomaly identification processing on the merged time-series feature vector to obtain the time-series anomaly identification result of the business data sequence includes: Obtain a time-series anomaly identification model; the time-series anomaly identification model includes a fully connected layer and a classification layer; The merged temporal feature vector is processed by the fully connected layer to obtain an intermediate feature vector. The intermediate feature vector is classified by the classification layer to obtain the classification result, and the time sequence anomaly identification result of the business data sequence is determined based on the classification result.

12. A timing anomaly identification device, characterized in that, include: The extraction module is used to acquire a business data sequence with a temporal relationship, perform location enhancement feature extraction processing on the business data sequence, and obtain the location enhancement feature vector matrix corresponding to the business data sequence. An attention module is used to perform multi-layer feature extraction and transformation processing on the position enhancement feature vector matrix through L transformer layers in the temporal feature mining model to obtain intermediate feature representation vector matrices output by the L transformer layers respectively; L is a positive integer; the L transformer layers have a chain connection relationship; the temporal feature mining model also includes a temporal network layer connected to the L transformer layers respectively; The temporal feature mining module is used to perform temporal feature mining on the intermediate feature representation vector matrix output by the connected transformer layer through L temporal network layers to obtain the merged temporal feature vector. The timing anomaly identification module is used to perform timing anomaly identification processing on the merged timing feature vector to obtain the timing anomaly identification result of the business data sequence.

13. A computer device, characterized in that, include: Processor, memory, and network interface; The processor is connected to the memory and the network interface, wherein the network interface is used to provide data communication functions, the memory is used to store program code, and the processor is used to call the program code to execute the method according to any one of claims 1-11.

14. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program adapted to be loaded by a processor and to execute the method according to any one of claims 1-11.

15. A computer program product comprising a computer program / instructions, characterized in that, When the computer program / instructions are executed by the processor, they can perform the method described in any one of claims 1-11.

Images