Model monitoring method and device, electronic equipment and computer readable storage medium

Abstract

The invention provides a model monitoring method and device, electronic equipment and a computer readable storage medium. The method comprises the following steps: loading model data of a network model from a preset data source end; performing data preprocessing on the model data of the loaded network model; performing data slicing processing on the preprocessed model data to obtain a plurality of data slices; for each data slice, carrying out data binning processing on the model data in the data slice to obtain a plurality of data buckets corresponding to the data slice; carrying out dimension reduction mapping processing of model data on the plurality of data buckets in parallel; a plurality of threads in a preset thread pool are utilized, monitoring index calculation is carried out in parallel according to the model data subjected to dimension reduction mapping processing in each data sub-bucket corresponding to each data slice, a monitoring index corresponding to each data sub-bucket is obtained, and the monitoring indexes are indexes reflecting the performance of the network model.

Description

Technical Field

[0001] The present disclosure relates to the field of computer technology, and in particular to a model monitoring method and device, an electronic device, and a computer-readable storage medium. Background Art

[0002] At present, when the industry implements model effect monitoring technology solutions or performs feature screening, due to the large scale of model data, most of the original data adopts distributed storage solutions. During calculation, the data needs to be pulled locally, and then loaded and preprocessed through Python (a computer programming language). Finally, multiple model data are serially iteratively calculated to obtain the model monitoring indicators. Summary of the invention

[0003] The present invention provides a model monitoring method and device, an electronic device, and a computer-readable storage medium.

[0004] In a first aspect, the present disclosure provides a model monitoring method, the model monitoring method comprising:

[0005] Loading model data of the network model from a preset data source end, wherein the preset data end is used to store the model data of the network model in a distributed storage manner;

[0006] Perform data preprocessing on the model data of the loaded network model;

[0007] Performing data slicing processing on the preprocessed model data to obtain multiple data slices, each data slice including the multiple preprocessed model data;

[0008] For each data slice, perform data binning processing on the model data in the data slice to obtain a plurality of data buckets corresponding to the data slice, each data bucket including a plurality of model data in the data slice;

[0009] Perform dimensionality reduction mapping processing on multiple data buckets in parallel;

[0010] Using multiple threads in a preset thread pool, monitoring indicators are calculated in parallel based on the model data processed by dimensionality reduction mapping in each data bucket corresponding to each data slice, so as to obtain monitoring indicators corresponding to each data bucket, wherein the monitoring indicators are indicators reflecting the performance of the network model.

[0011] In a second aspect, the present disclosure provides a model monitoring device, the model monitoring device comprising:

[0012] A loading unit, used to load model data of the network model from a preset data source end, wherein the preset data end is used to store the model data of the network model in a distributed storage manner;

[0013] A preprocessing unit, used for performing data preprocessing on model data of the loaded network model;

[0014] A slicing unit, used for performing data slicing processing on the preprocessed model data to obtain a plurality of data slices, each data slice including a plurality of preprocessed model data;

[0015] A binning unit, for binning the model data in each data slice to obtain a plurality of data buckets corresponding to the data slice, each data bucket including a plurality of model data in the data slice;

[0016] A dimension reduction unit is used to perform dimension reduction mapping processing on multiple data buckets in parallel;

[0017] The computing unit is used to utilize multiple threads in a preset thread pool to perform monitoring index calculations in parallel based on the model data processed by dimensionality reduction mapping in each data bucket corresponding to each data slice, and obtain monitoring indicators corresponding to each data bucket, wherein the monitoring indicators are indicators reflecting the performance of the network model.

[0018] In a third aspect, the present disclosure provides an electronic device, comprising: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores one or more computer programs executable by the at least one processor, and one or more of the computer programs are executed by the at least one processor so that the at least one processor can execute the above-mentioned model monitoring method.

[0019] In a fourth aspect, the present disclosure provides a computer-readable storage medium having a computer program stored thereon, wherein the computer program implements the above-mentioned model monitoring method when executed by a processor / processing core.

[0020] According to the technical solutions of the model monitoring method and device, electronic device, and computer-readable storage medium provided in the embodiments of the present disclosure, when performing model monitoring, distributed loading and reading of data are performed directly on the preset data end through the distributed data transmission and storage capabilities of multiple data nodes on the preset data end, without the need to read the data into local storage, which is conducive to breaking through computing bottlenecks such as slow data transmission, high bandwidth, and low storage capacity of a single machine, and can adapt to a variety of data structure types to ensure data integrity and consistency; before calculating the model monitoring index, preprocessing the model data of the loaded network model is conducive to reducing the subsequent introduction of deviations and errors due to the presence of abnormal data, and is conducive to ensuring the data precision, accuracy, and reliability of the data, thereby improving the computing effect and efficiency of using model data to calculate monitoring indicators; before calculating the model monitoring index, By slicing the model data, it is convenient to isolate and calculate the monitoring indicators for each data slice separately in the future, and to control the selection of appropriate data volume for calculation each time, it is compatible with data scales of different sizes, which is conducive to solving the calculation bottleneck problem of single-point large data sets; before calculating the model monitoring indicators, by performing dimensionality reduction mapping processing on the data of each data bucket, the data association is effectively reduced, which is conducive to reducing unnecessary merging of associated data when calculating the monitoring indicators, thereby helping to reduce additional resource waste and cost overhead; when calculating the model monitoring indicators, multiple threads in the preset thread pool are used to calculate the monitoring indicators in parallel according to the model data processed by dimensionality reduction mapping in each data bucket corresponding to each data slice, thereby realizing the concurrent computing capability of multiple data slices, which is conducive to improving the computational efficiency of the model monitoring indicator calculation.

[0021] It should be understood that the content described in this section is not intended to identify the key or important features of the embodiments of the present disclosure, nor is it intended to limit the scope of the present disclosure. Other features of the present disclosure will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] The accompanying drawings are used to provide a further understanding of the present disclosure and constitute a part of the specification. Together with the embodiments of the present disclosure, they are used to explain the present disclosure and do not constitute a limitation of the present disclosure. The above and other features and advantages will become more apparent to those skilled in the art by describing detailed example embodiments with reference to the accompanying drawings, in which:

[0023] Figure 1 A flowchart of a model monitoring method provided by an embodiment of the present disclosure;

[0024] Figure 2 It is a schematic diagram of a dimensionality reduction mapping process for model data;

[0025] Figure 3A block diagram of a model monitoring device provided by an embodiment of the present disclosure;

[0026] Figure 4 A block diagram of a composition of an electronic device provided in an embodiment of the present disclosure. DETAILED DESCRIPTION

[0027] In order to enable those skilled in the art to better understand the technical solutions of the present disclosure, the exemplary embodiments of the present disclosure are described below in conjunction with the accompanying drawings, including various details of the embodiments of the present disclosure to facilitate understanding, which should be considered as merely exemplary. Therefore, those of ordinary skill in the art should recognize that various changes and modifications can be made to the embodiments described herein without departing from the scope and spirit of the present disclosure. Similarly, for the sake of clarity and conciseness, the description of well-known functions and structures is omitted in the following description.

[0028] In the absence of conflict, the various embodiments of the present disclosure and the various features therein may be combined with each other.

[0029] As used herein, the term "and / or" includes any and all combinations of one or more of the associated listed items.

[0030] The terms used herein are only used to describe specific embodiments and are not intended to limit the present disclosure. As used herein, the singular forms "a" and "the" are also intended to include plural forms, unless the context clearly indicates otherwise. It will also be understood that when the terms "including" and / or "made of" are used in this specification, the presence of the features, wholes, steps, operations, elements and / or components is specified, but the presence or addition of one or more other features, wholes, steps, operations, elements, components and / or groups thereof is not excluded. "Connected" or "connected" and similar words are not limited to physical or mechanical connections, but can include electrical connections, whether direct or indirect.

[0031] Unless otherwise defined, all terms (including technical and scientific terms) used herein have the same meaning as commonly understood by those of ordinary skill in the art. It will also be understood that terms such as those defined in commonly used dictionaries should be interpreted as having a meaning consistent with their meaning in the context of the relevant art and the present disclosure, and will not be interpreted as having an idealized or overly formal meaning unless explicitly defined as such herein.

[0032] In related technologies, due to the large scale of model data, most of them use distributed storage solutions for raw data. During calculation, the data needs to be pulled locally, and then loaded and preprocessed through Python (a computer programming language). Finally, multiple model data are serially iteratively calculated to obtain the model's monitoring indicators.

[0033] However, the model monitoring solutions of related technologies usually have the following problems:

[0034] 1. End-to-end data transmission bottleneck: The amount of data transmitted from the distributed file system to the local storage system is huge, which makes the data transmission process very slow and takes a lot of time.

[0035] 2. Local data storage bottleneck: Storing a large amount of data in a local single machine will take up a lot of hard disk space. Once the local hard disk space is insufficient, storage will fail, causing the entire data transmission process to fail.

[0036] 3. Network bandwidth surges. Due to the large amount of data, massive files occupy a lot of network bandwidth during data transmission, seriously affecting the stability of online tasks and clusters.

[0037] 4. Data integrity issues. Due to the diversity of business and the complexity of data type structures, many problems occur during the transmission process, such as loss of data accuracy and inconsistent loaded data structures.

[0038] 5. Poor computing performance and versatility. Due to business complexity, the diversity of computing dimensions and monitoring indicators, the current business team is unable to perform multi-dimensional and multi-indicator calculations at the same time, and is unable to perform multi-model batch binning and batch calculations.

[0039] To this end, the embodiment of the present disclosure provides a model monitoring method. According to the technical solution of the model monitoring method provided by the embodiment of the present disclosure, when performing model monitoring, distributed loading and reading of data are performed directly on the preset data end through the distributed data transmission and storage capabilities of multiple data nodes on the preset data end, without the need to read the data to local storage, which is conducive to breaking through computing bottlenecks such as slow data transmission, high bandwidth, and low storage capacity of a single machine, and can adapt to a variety of data structure types to ensure data integrity and consistency; before calculating the model monitoring index, preprocessing the model data of the loaded network model is conducive to reducing subsequent deviations and errors introduced due to the presence of abnormal data, and is conducive to ensuring the data precision, accuracy and reliability of the data, thereby improving the computing effect and efficiency of using model data to calculate monitoring indicators; before calculating the model monitoring index, By slicing the model data, it is convenient to isolate and calculate the monitoring indicators for each data slice separately in the future, and to control the selection of appropriate data volume for calculation each time, it is compatible with data scales of different sizes, which is conducive to solving the calculation bottleneck problem of single-point large data sets; before calculating the model monitoring indicators, by performing dimensionality reduction mapping processing on the data of each data bucket, the data association is effectively reduced, which is conducive to reducing unnecessary merging of associated data when calculating the monitoring indicators, thereby helping to reduce additional resource waste and cost overhead; when calculating the model monitoring indicators, multiple threads in the preset thread pool are used to calculate the monitoring indicators in parallel according to the model data processed by dimensionality reduction mapping in each data bucket corresponding to each data slice, thereby realizing the concurrent computing capability of multiple data slices, which is conducive to improving the computational efficiency of the model monitoring indicator calculation.

[0040] The model monitoring method of the embodiment of the present disclosure can be performed by an electronic device. The electronic device can be a terminal device, and the terminal device can be, for example, a vehicle-mounted device, a user equipment (User Equipment, UE), a mobile device, a user terminal, a terminal, a cellular phone, a cordless phone, a personal digital assistant (Personal Digital Assistant, PDA), a handheld device, a computing device, a vehicle-mounted device, a wearable device, etc. The electronic device can be a server, and the server can be an independent physical server, a server cluster, or a cloud server capable of cloud computing. The model monitoring method can be implemented by a processor in an electronic device calling a computer-readable program instruction stored in a memory.

[0041] Figure 1 A flow chart of a model monitoring method provided by an embodiment of the present disclosure. Figure 1 , the model monitoring method includes:

[0042] Step S11: Loading model data of the network model from a preset data source end, where the preset data end is used to store the model data of the network model in a distributed storage manner.

[0043] Among them, the network model can be any neural network model, and the model data refers to the input data of the network model or the output data of the network model, wherein the input data includes input features or input variables used to input the network model for model inference calculation, and the output data is the output result of the model inference calculation, and the output data includes the model points output by the network model. The model data is stored in a preset data end in a distributed storage manner, and the preset data end can be a distributed storage system.

[0044] For example, the distributed storage system is a distributed storage system based on HIVE. HIVE is a data warehouse tool based on Hadoop, which is used for data extraction, transformation, and loading. This is a mechanism for storing, querying, and analyzing large-scale data stored in the Hadoop Distributed File System (HDFS). Hadoop is an open source distributed system infrastructure.

[0045] In the embodiment of the present disclosure, the model data of the network model can be distributedly stored in the preset data end in the form of a HIVE table. In step S11, the model data of the network model is loaded and read from the preset data end using a preset data engine, wherein the data engine can use a Spark engine, and Spark generally refers to Apache Spark, which is a fast and general computing engine designed specifically for large-scale data processing.

[0046] In the disclosed embodiments, the model scores, input features or input variables in the model data can be stored in the form of a model score wide table, a feature wide table or a variable wide table. The model score wide table is used to record the model scores, the feature wide table is used to record the input features, and the variable wide table is used to record the input variables. Among them, the wide table literally means a database table with more fields (columns), usually referring to a database table that is associated with indicators, dimensions, and attributes related to business topics.

[0047] In the embodiments of the present disclosure, a preset data engine is used to directly connect to a preset data terminal for data loading and reading. Distributed data loading is achieved through the distributed data transmission and storage capabilities of multiple data nodes of the preset data terminal, breaking through computing bottlenecks such as slow single-machine data transmission, high bandwidth, and low storage capacity. Distributed loading and reading of data is performed directly at the preset data terminal, which can adapt to a variety of data structure types and ensure data integrity and consistency.

[0048] In some embodiments, the step of loading the model data of the network model from the preset data source, that is, the above step S11, may further include:

[0049] The model data of the network model is loaded from the preset data source by using the preset data engine; and in the process of loading the model data of the network model, the model data of the network model is subjected to data merging and data compression processing.

[0050] Exemplarily, the data engine may adopt the Spark engine, directly connect to the preset data source through the Spark engine to read data, merge and compress the read data during the data loading process, and greatly reduce the network I / O load.

[0051] Step S12: preprocess the model data of the loaded network model.

[0052] Among them, data preprocessing may include at least one of the following: data missing value replacement processing, data cleaning processing, data discretization processing and data type conversion processing, among which data cleaning may include data cleaning of abnormal data such as invalid data, inaccurate data, duplicate data, inconsistent data, etc.

[0053] Correspondingly, the step of performing data preprocessing on the model data of the loaded network model, i.e., the above-mentioned step S12, may include at least one of the following steps: performing data cleaning processing on the model data of the loaded network model; performing data missing value replacement processing on the model data of the loaded network model; performing data discretization processing on the model data of the loaded network model; performing data type conversion processing on the model data of the loaded network model.

[0054] In the embodiments of the present disclosure, before using model data to calculate monitoring indicators, data preprocessing is performed on the model data of the loaded network model. This helps reduce the subsequent introduction of deviations and errors due to the presence of abnormal data, and helps ensure the data precision, accuracy and reliability, thereby improving the calculation effect and efficiency of using model data to calculate monitoring indicators.

[0055] Step S13: performing data slicing processing on the preprocessed model data to obtain a plurality of data slices, each data slice including one or more preprocessed model data.

[0056] In the embodiment of the present disclosure, there are multiple model data, and data slicing is performed on the preprocessed model data, that is, the multiple model data are divided into multiple batches of model data, each batch of model data is used as a data slice, and each data slice includes the multiple preprocessed model data.

[0057] In the disclosed embodiment, before calculating the monitoring indicators, the model data is sliced ​​to facilitate the subsequent isolation and separate calculation of monitoring indicators for each data slice, and the appropriate amount of data is selected as much as possible for calculation each time. This is compatible with data scales of different sizes and is conducive to solving the computational bottleneck problem of single-point large data sets.

[0058] Taking the model width table as an example, the model width table is filtered according to the field columns of different model scores recorded in the model width table, and the recorded model scores are divided into multiple batches of data to obtain multiple data slices.

[0059] In some embodiments, the step of performing data slicing processing on the preprocessed model data to obtain multiple data slices, i.e., the above-mentioned step S13, may further include: determining whether the ratio of the current remaining number of preprocessed model data to the preset number is greater than a preset threshold; in the case where the ratio of the current remaining number to the preset number is greater than the preset threshold, obtaining a preset number of model data from the currently remaining preprocessed model data to generate a data slice, and returning to execute the step of determining whether the ratio of the current remaining number of preprocessed model data to the preset number is greater than the preset threshold; in the case where the ratio of the current remaining number to the preset number is less than or equal to the preset threshold, generating a data slice based on the currently remaining preprocessed model data.

[0060] Initially, the ratio of the total number of preprocessed model data to the preset number is greater than a preset threshold.

[0061] Exemplarily, the default preset number is 5, and the preset threshold is 1.2. When the initial total number of model data is 11, that is, the current remaining number is 11, according to the segmentation rule, 11 / 5=2.2, which is greater than the preset threshold, 5 of the model data are used as a data slice, and 6 model data remain; when the current remaining number of model data is 6, according to the segmentation rule, 6 / 5=1.2, which is less than or equal to the preset threshold, 1 data slice is generated based on the 6 model data.

[0062] Step S14: for each data slice, perform data binning processing on the model data in the data slice to obtain multiple data buckets corresponding to the data slice, each data bucket including multiple model data in the data slice.

[0063] In some embodiments, the preprocessed model data in the data slice is binned with equal frequency or equal distance to obtain multiple data buckets corresponding to the data slice.

[0064] In the equal-frequency binning or equal-distance binning operation, the data boundary of the bucket can be specified according to the number of buckets, the size of the model score in the model data, the number of model scores, the number of input features, or the number of input variables. This enables multi-model data input, batch model data binning operations, and subsequent calculation operations, effectively solving the traditional iterative loop calculation method for each model data, greatly improving development efficiency.

[0065] In some embodiments, data can be bucketed through custom bucketing rules, thereby realizing multi-model data input, batch model data binning operations and subsequent calculation operations, effectively solving the traditional iterative loop method of calculating each model data, and greatly improving development efficiency.

[0066] Furthermore, the method of implementing data bucketing through custom bucketing rules may include the following steps:

[0067] 1. Preprocess the model data: First, the model data needs to be preprocessed so that the model data can be divided into multiple data types.

[0068] 2. Define different bucketing rules: Secondly, archive the data types according to the preprocessed data types, and define different bucketing rules for the different archived data types.

[0069] 3. Application of bucketing rules: Finally, by defining different bucketing rules, the model data is automatically matched and mapped according to the data type to distribute the model data to different data buckets.

[0070] Ultimately, the function of bucketing input data can be realized by implementing multi-dimensional batches of multiple types of data and multi-dimensional batches of input data.

[0071] In some embodiments, the step of performing data binning processing on the model data in the data slice to obtain multiple data buckets corresponding to the data slice, i.e., the above-mentioned step S14, may further include: performing data binning processing on the model data in the data slice using an equal-frequency binning method to obtain multiple data buckets corresponding to the data slice; or performing data binning processing on the model data in the data slice using an equal-distance binning method to obtain multiple data buckets corresponding to the data slice.

[0072] The steps of the equal frequency binning method include:

[0073] 1. Sort the data in ascending or descending order;

[0074] 2. Divide the sorted data into k equal parts, each part of the data is a subset, and each subset contains n / k data, where n is the total number of data;

[0075] 3. Determine the boundaries of each subset, that is, determine the score of each data so that the data values ​​in each subset are within the upper and lower range of the score;

[0076] 4. Allocate the data to corresponding subsets so that each subset has similar or equal amounts of data, and each subset serves as a data bucket.

[0077] The equal-interval binning method, also known as the equal-width binning method, is a method of mapping the data value of each data to intervals of equal size.

[0078] In some embodiments, a pre-trained neural network model prediction model data may be used to perform binning operations using an equal-distance binning method or an equal-frequency binning method.

[0079] Step S15: Perform dimensionality reduction mapping processing on the model data in parallel on the multiple data buckets.

[0080] In an embodiment of the present disclosure, by performing dimensionality reduction mapping processing on multiple data buckets in parallel, high-dimensional model data is dimensionality reduced and mapped into low-dimensional model data to reduce the data dimension of the model data. The data dimension of the model data after the dimensionality reduction mapping processing is smaller than the data dimension of the model data before the dimensionality reduction mapping processing.

[0081] For example, multiple similar dimension indicators of model data can be degraded by reverse perspective, and then multiple dimension reduction indicators of model data can be mapped and combined by custom methods. Dimension degradation can effectively reduce data association, which is conducive to reducing unnecessary merging of related data when calculating monitoring indicators, thereby helping to reduce additional resource waste and cost overhead.

[0082] In some embodiments, the step of performing dimensionality reduction mapping processing on multiple data buckets in parallel, i.e., the above step S15, may further include: for multiple data buckets, performing dimensionality reduction mapping processing on the model data in each data bucket in parallel by reverse perspective, to obtain the model data in each data bucket after dimensionality reduction mapping processing. The data dimension of the model data after dimensionality reduction mapping processing is lower than the data dimension of the model data before dimensionality reduction mapping processing.

[0083] Figure 2 It is a schematic diagram of a dimensionality reduction mapping process for model data, such as Figure 2As shown, illustratively, taking the model data as a feature wide table or a model sub-wide table as an example, the model data includes a primary key part and a dimension (dim) part, wherein the primary key is one or more fields in the table, and its value is used to uniquely identify a record in the table, and the dimension part records multiple model points or multiple feature fields in the feature wide table or the model sub-wide table. For the feature wide table or the model sub-wide table, you can reverse pivot according to the model points or feature fields to realize column-to-row conversion, convert multiple model points or multiple feature fields into name (name) columns and value (value) columns, thereby achieving the purpose of dimensionality reduction mapping, and then through the combination of key value and dimension mapping, you can finally get the following. Figure 2 The table on the left side is shown. Figure 2 The table on the left side is the model data after the final dimension reduction mapping processing.

[0084] In some embodiments, after performing dimensionality reduction mapping processing on multiple data buckets in parallel, it also includes: performing data evaluation verification on a sample of the model data after the dimensionality reduction mapping processing to verify the consistency of the model data before and after the dimensionality reduction mapping processing.

[0085] Step S16, using multiple threads in the preset thread pool, calculate the monitoring index in parallel according to the model data processed by dimensionality reduction mapping in each data bucket corresponding to each data slice, and obtain the monitoring index corresponding to each data bucket, which is an indicator reflecting the performance of the network model.

[0086] In the embodiment of the present disclosure, the model data processed by dimensionality reduction mapping is used, according to the data slicing results, by using multiple threads in a preset thread pool, to perform monitoring index calculations in parallel based on the model data processed by dimensionality reduction mapping in each data bucket corresponding to each data slice, thereby realizing the concurrent computing capability of multiple data slices.

[0087] Exemplary monitoring indicators include information value (IV), discrimination index (Kolmogorov-Smirnov, KS), model stability index (Population Stability Index, PSI), ROC (Receiver Operating Characteristic, receiver operating characteristic) curve and the area enclosed by the coordinate axis (Area Under Curve, AUC), lift index (Lift), Gini index (Gini), etc. In the embodiment of the present disclosure, any monitoring indicator can be calculated according to the calculation method of any monitoring indicator, and the embodiment of the present disclosure does not impose any special restrictions on this.

[0088] In some embodiments, for multiple data slices, each batch of data slices is marked to map the data slices of the same batch upstream and downstream in series, and the number of each batch of data slices is used as the thread pool capacity to realize dynamic expansion function, and isolated calculations of multiple batches of data slices are performed in batches, which is conducive to accelerating computing efficiency.

[0089] In some embodiments, in the above step S16, when the number of data slices is less than or equal to the number of threads in the preset thread pool, each data slice corresponds to a thread in the preset thread pool, and each thread in the preset thread pool is used to calculate monitoring indicators for each corresponding data slice.

[0090] Exemplarily, there are 5 threads preset in the thread pool, and there are 5 data slices in total. Then, each thread processes one data slice, and multiple data buckets corresponding to one data slice are input into the corresponding threads in sequence to calculate the monitoring indicators. For multiple threads, the monitoring indicators are calculated in parallel according to the model data in the currently input data bucket.

[0091] In some embodiments, in the above step S16, when the number of data slices is greater than the number of threads in the preset thread pool, and the ratio of the number of data slices to the number of threads in the preset thread pool is an integer, multiple data slices are divided into multiple batches of data slices, where the number of batches of data slices is the ratio, and each data slice in each batch of data slices corresponds to a thread in the preset thread pool, and each thread in the preset thread pool is used to calculate monitoring indicators for each data slice in each corresponding batch of data slices.

[0092] For example, there are 5 threads preset in the thread pool, and there are 10 data slices in total. Then, the multiple data slices are divided into two batches of data slices, each batch includes 5 data slices, and each data slice in each batch of data slices corresponds to a thread in the preset thread pool. For each data slice, the corresponding thread in the preset thread pool is used for processing. Multiple threads process each batch of data slices in turn, that is, after multiple threads finish processing the current batch of data slices, they process the next batch of data slices.

[0093] In some embodiments, in the above step S16, when the number of data slices is greater than the number of threads in the preset thread pool, and the ratio of the number of data slices to the number of threads in the preset thread pool is a non-integer, multiple data slices are divided into multiple batches of data slices, where the number of batches of data slices is the ratio rounded plus 1, and each data slice in each batch of data slices corresponds to a thread in the preset thread pool, and each thread in the preset thread pool is used to calculate monitoring indicators for each data slice in each corresponding batch of data slices.

[0094] For example, there are 5 threads preset in the thread pool, and there are 12 data slices in total. Then, the multiple data slices are divided into three batches of data slices, the first batch of data slices includes 5 data slices, the second batch of data slices includes 5 data slices, and the third batch of data slices includes 2 data slices. Each data slice in each batch of data slices corresponds to a thread in the preset thread pool, and each data slice is processed using the corresponding thread in the preset thread pool. Multiple threads process each batch of data slices in turn, that is, after multiple threads finish processing the current batch of data slices, they process the next batch of data slices.

[0095] According to the technical solution of the model monitoring method provided by the embodiment of the present disclosure, when performing model monitoring, distributed loading and reading of data are performed directly on the preset data end through the distributed data transmission and storage capabilities of multiple data nodes on the preset data end, without the need to read the data into local storage, which is conducive to breaking through computing bottlenecks such as slow data transmission, high bandwidth, and low storage capacity of a single machine, and can adapt to a variety of data structure types to ensure data integrity and consistency; before calculating the model monitoring index, preprocessing the model data of the loaded network model is conducive to reducing subsequent deviations and errors introduced due to the presence of abnormal data, and is conducive to ensuring the data precision, accuracy and reliability of the data, thereby improving the computing effect and efficiency of using model data to calculate monitoring indicators; before calculating the model monitoring index, preprocessing the model data is conducive to reducing subsequent deviations and errors introduced due to the presence of abnormal data, and is conducive to ensuring the data precision, accuracy and reliability of the data, thereby improving the computing effect and efficiency of using model data to calculate monitoring indicators; Slicing is used to isolate and calculate monitoring indicators for each data slice in the future, and to control the selection of appropriate data volume for calculation each time. It is compatible with data of different sizes and is conducive to solving the calculation bottleneck problem of single-point large data sets. Before calculating the model monitoring indicators, the data of each data bucket is processed by dimensionality reduction and mapping, which effectively reduces the data association, and is conducive to reducing unnecessary merging of associated data when calculating the monitoring indicators, thereby helping to reduce additional resource waste and cost overhead. When calculating the model monitoring indicators, multiple threads in the preset thread pool are used to calculate the monitoring indicators in parallel according to the model data processed by dimensionality reduction and mapping in each data bucket corresponding to each data slice, thereby realizing the concurrent computing capability of multiple data slices, which is conducive to improving the computational efficiency of the model monitoring indicator calculation.

[0096] In some embodiments, after the monitoring indicators of the model are calculated, the monitoring indicators are written into a monitoring indicator wide table for storage. The monitoring indicator wide table is used to record the various monitoring indicators obtained by calculation.

[0097] In some embodiments, after the monitoring indicators of the model are calculated, the data of the calculated monitoring indicators are stored in a disk and a table. In some embodiments, the monitoring indicators of the model can be stored in the form of a HIVE table.

[0098] In some embodiments, after the monitoring indicators of the model are calculated, the monitoring indicators are connected to a BI (Business Intelligence) system for BI visualization.

[0099] It can be understood that the above-mentioned various method embodiments mentioned in the present disclosure can be combined with each other to form a combined embodiment without violating the principle logic. Due to space limitations, the present disclosure will not repeat them. It can be understood by those skilled in the art that in the above-mentioned method of the specific implementation method, the specific execution order of each step should be determined according to its function and possible internal logic.

[0100] In addition, the present disclosure also provides a model monitoring device, an electronic device, and a computer-readable storage medium, all of which can be used to implement any model monitoring method provided by the present disclosure. The corresponding technical solutions and descriptions are referred to the corresponding records in the method part and will not be repeated here.

[0101] Figure 3 A block diagram of the composition of a model monitoring device provided in an embodiment of the present disclosure.

[0102] Reference Figure 3 The embodiment of the present disclosure provides a model monitoring device, the model monitoring device 300 includes:

[0103] The loading unit 301 is used to load the model data of the network model from a preset data source end, and the preset data end is used to store the model data of the network model in a distributed storage manner.

[0104] The preprocessing unit 302 is used to perform data preprocessing on the model data of the loaded network model.

[0105] The slicing unit 303 is used to perform data slicing processing on the preprocessed model data to obtain multiple data slices, each of which includes multiple preprocessed model data.

[0106] The binning unit 304 is used to perform data binning processing on the model data in each data slice to obtain multiple data buckets corresponding to the data slice, each data bucket including multiple model data in the data slice.

[0107] The dimension reduction unit 305 is used to perform dimension reduction mapping processing on the model data in parallel on the multiple data buckets.

[0108] The computing unit 306 is used to use multiple threads in a preset thread pool to calculate monitoring indicators in parallel based on the model data processed by dimensionality reduction mapping in each data bucket corresponding to each data slice, and obtain the monitoring indicators corresponding to each data bucket. The monitoring indicators are indicators that reflect the performance of the network model.

[0109] In some embodiments, the loading unit 301 is used to: use a preset data engine to load the model data of the network model from a preset data source; and in the process of loading the model data of the network model, perform data merging and data compression processing on the model data of the network model.

[0110] In some embodiments, the preprocessing unit 302 is used to perform at least one of the following: performing data cleaning processing on the model data of the loaded network model; performing data missing value replacement processing on the model data of the loaded network model; performing data discretization processing on the model data of the loaded network model; performing data type conversion processing on the model data of the loaded network model.

[0111] In some embodiments, the slicing unit 303 is used to: determine whether the ratio of the current remaining number of preprocessed model data to the preset number is greater than a preset threshold, and the ratio of the total number of preprocessed model data to the preset number is greater than the preset threshold; when the ratio of the current remaining number to the preset number is greater than the preset threshold, obtain the preset number of model data from the currently remaining preprocessed model data to generate a data slice, and return to execute the step of determining whether the ratio of the current remaining number of preprocessed model data to the preset number is greater than the preset threshold; when the ratio of the current remaining number to the preset number is less than or equal to the preset threshold, generate a data slice based on the currently remaining preprocessed model data.

[0112] In some embodiments, the binning unit 304 is used to: use the equal-frequency binning method to perform data binning processing on the model data in the data slice to obtain multiple data buckets corresponding to the data slice; or use the equal-distance binning method to perform data binning processing on the model data in the data slice to obtain multiple data buckets corresponding to the data slice.

[0113] In some embodiments, the dimensionality reduction unit 305 is used to perform dimensionality reduction mapping processing on the model data in each data bucket in parallel by reverse perspective for multiple data buckets, so as to obtain the model data in each data bucket after dimensionality reduction mapping processing.

[0114] In some embodiments, when the number of data slices is less than or equal to the number of threads in the preset thread pool, each data slice corresponds to a thread in the preset thread pool, v; when the number of data slices is greater than the number of threads in the preset thread pool, and the ratio of the number of data slices to the number of threads in the preset thread pool is an integer, multiple data slices are divided into multiple batches of data slices, wherein the number of batches of data slices is the ratio, and each data slice in each batch of data slices corresponds to a thread in the preset thread pool, and each thread in the preset thread pool is used to calculate monitoring indicators for each data slice in each corresponding batch of data slices; when the number of data slices is greater than the number of threads in the preset thread pool, and the ratio of the number of data slices to the number of threads in the preset thread pool is a non-integer, multiple data slices are divided into multiple batches of data slices, wherein the number of batches of data slices is the ratio rounded plus 1, and each data slice in each batch of data slices corresponds to a thread in the preset thread pool, and each thread in the preset thread pool is used to calculate monitoring indicators for each data slice in each corresponding batch of data slices.

[0115] Each module in the above-mentioned model monitoring device can be implemented in whole or in part by software, hardware or a combination thereof. Each module can be embedded in or independent of a processor in a computer device in the form of hardware, or can be stored in a memory in a computer device in the form of software, so that the processor can call and execute operations corresponding to each module above.

[0116] According to the technical solution of the model monitoring device provided by the embodiment of the present disclosure, when performing model monitoring, the loading unit directly performs distributed loading and reading of data on the preset data end through the distributed data transmission and storage capabilities of multiple data nodes of the preset data end, without the need to read the data to local storage, which is conducive to breaking through the computing bottlenecks such as slow data transmission, high bandwidth, and low storage capacity of a single machine, and can adapt to a variety of data structure types to ensure the integrity and consistency of the data; before calculating the model monitoring index, the preprocessing unit performs data preprocessing on the model data of the loaded network model, which is conducive to reducing the subsequent deviations and errors introduced due to the presence of abnormal data, and is conducive to ensuring the data precision, accuracy and reliability of the data, thereby improving the computing effect and efficiency of using model data to calculate monitoring indicators; before calculating the model monitoring index, the slicing unit performs data preprocessing on the model data of the loaded network model, which is conducive to reducing the subsequent deviations and errors introduced due to the presence of abnormal data, and is conducive to ensuring the data precision, accuracy and reliability of the data, thereby improving the computing effect and efficiency of using model data to calculate monitoring indicators; The data is sliced ​​in rows, so that each data slice can be isolated and calculated separately for monitoring indicators in the future, and the appropriate amount of data can be selected for calculation each time. It is compatible with data of different sizes and is conducive to solving the calculation bottleneck problem of single-point large data sets. Before calculating the model monitoring indicators, the dimension reduction unit effectively reduces the data association by performing dimension reduction mapping on the data of each data bucket, which is conducive to reducing unnecessary merging of associated data when calculating the monitoring indicators, thereby helping to reduce additional resource waste and cost overhead. When calculating the model monitoring indicators, the calculation unit uses multiple threads in the preset thread pool to calculate the monitoring indicators in parallel according to the model data processed by dimension reduction mapping in each data bucket corresponding to each data slice, thereby realizing the concurrent computing capability of multiple data slices, which is conducive to improving the computing efficiency of the model monitoring indicator calculation.

[0117] Figure 4 A block diagram of a composition of an electronic device provided in an embodiment of the present disclosure.

[0118] Reference Figure 4, an embodiment of the present disclosure provides an electronic device, the electronic device comprising: at least one processor 401; at least one memory 402, and one or more I / O interfaces 403, connected between the processor 401 and the memory 402; wherein the memory 402 stores one or more computer programs that can be executed by at least one processor 301, and the one or more computer programs are executed by at least one processor 401, so that at least one processor 401 can execute the above-mentioned model monitoring method, the model monitoring method comprising: loading model data of a network model from a preset data source end, the preset data end is used to store the model data of the network model in a distributed storage manner; performing digital processing on the loaded model data of the network model The invention relates to a method for preprocessing the model data; performing data slicing on the preprocessed model data to obtain multiple data slices, each data slice including multiple model data after preprocessing; for each data slice, performing data binning on the model data in the data slice to obtain multiple data buckets corresponding to the data slice, each data bucket including multiple model data in the data slice; performing dimensionality reduction mapping on the model data in parallel on the multiple data buckets; for multiple data slices, using multiple threads in a preset thread pool, performing monitoring indicator calculation in parallel according to the model data in each data bucket corresponding to each data slice after dimensionality reduction mapping, to obtain monitoring indicators corresponding to each data bucket, wherein the monitoring indicators are indicators reflecting the performance of the network model.

[0119] In some embodiments, the processor 401 is used to execute: using a preset data engine to load the model data of the network model from a preset data source; and in the process of loading the model data of the network model, performing data merging and data compression processing on the model data of the network model.

[0120] In some embodiments, the processor 401 is used to perform data preprocessing on the model data of the loaded network model, wherein the data preprocessing on the model data of the loaded network model includes at least one of the following: data cleaning processing on the model data of the loaded network model; data missing value replacement processing on the model data of the loaded network model; data discretization processing on the model data of the loaded network model; data type conversion processing on the model data of the loaded network model.

[0121] In some embodiments, processor 401 is used to execute: determining whether the ratio of the current remaining number of preprocessed model data to the preset number is greater than a preset threshold; if the ratio is greater than the preset threshold, obtaining a preset number of model data from the currently remaining preprocessed model data to generate a data slice, and returning to execute the step of determining whether the ratio of the current remaining number of preprocessed model data to the preset number is greater than the preset threshold; if the ratio is less than or equal to the preset threshold, generating a data slice based on the currently remaining preprocessed model data.

[0122] In some embodiments, the processor 401 is used to execute: using the equal-frequency binning method to perform data binning processing on the model data in the data slice to obtain multiple data buckets corresponding to the data slice; or using the equal-distance binning method to perform data binning processing on the model data in the data slice to obtain multiple data buckets corresponding to the data slice.

[0123] In some embodiments, the processor 401 is used to execute: for each data bucket, perform dimensionality reduction mapping processing on the model data in each data bucket by reverse perspective, so as to obtain the model data after dimensionality reduction mapping processing in each data bucket.

[0124] In some embodiments, when the number of data slices is less than or equal to the number of threads in the preset thread pool, each data slice corresponds to a thread in the preset thread pool, and each data slice is processed using the corresponding thread in the preset thread pool; when the number of data slices is greater than the number of threads in the preset thread pool, and the ratio of the number of data slices to the number of threads in the preset thread pool is an integer, multiple data slices are divided into multiple batches of data slices, wherein the number of batches of data slices is the ratio, and each data slice in each batch of data slices corresponds to a thread in the preset thread pool, and each data slice is processed using the corresponding thread in the preset thread pool; when the number of data slices is greater than the number of threads in the preset thread pool, and the ratio of the number of data slices to the number of threads in the preset thread pool is a non-integer, multiple data slices are divided into multiple batches of data slices, wherein the number of batches of data slices is the ratio rounded plus 1, and each data slice in each batch of data slices corresponds to a thread in the preset thread pool, and each data slice is processed using the corresponding thread in the preset thread pool.

[0125] Each module in the above electronic device can be implemented in whole or in part by software, hardware, or a combination thereof. Each module can be embedded in or independent of a processor in a computer device in the form of hardware, or can be stored in a memory in a computer device in the form of software, so that the processor can call and execute operations corresponding to each module.

[0126] The present disclosure also provides a computer-readable storage medium on which a computer program is stored, wherein the computer program implements the above-mentioned model monitoring method when executed by a processor / processing core. The computer-readable storage medium may be a volatile or non-volatile computer-readable storage medium.

[0127] The embodiment of the present disclosure also provides a computer program product, including a computer-readable code, or a non-volatile computer-readable storage medium carrying the computer-readable code. When the computer-readable code runs in a processor of an electronic device, the processor in the electronic device executes the above-mentioned model monitoring method.

[0128] It will be appreciated by those skilled in the art that all or some of the steps, systems, and functional modules / units in the methods disclosed above may be implemented as software, firmware, hardware, and appropriate combinations thereof. In a hardware implementation, the division between the functional modules / units mentioned in the above description does not necessarily correspond to the division of physical components; for example, a physical component may have multiple functions, or a function or step may be performed by several physical components in cooperation. Some or all physical components may be implemented as software executed by a processor, such as a central processing unit, a digital signal processor, or a microprocessor, or may be implemented as hardware, or may be implemented as an integrated circuit, such as an application-specific integrated circuit. Such software may be distributed on a computer-readable storage medium, which may include a computer storage medium (or a non-transitory medium) and a communication medium (or a temporary medium).

[0129] As is known to those of ordinary skill in the art, the term computer storage media includes volatile and nonvolatile, removable and non-removable media implemented in any method or technology for storing information (such as computer-readable program instructions, data structures, program modules or other data). Computer storage media include, but are not limited to, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM), static random access memory (SRAM), flash memory or other memory technology, portable compact disk read-only memory (CD-ROM), digital versatile disk (DVD) or other optical disk storage, magnetic cassettes, magnetic tapes, magnetic disk storage or other magnetic storage devices, or any other medium that can be used to store desired information and can be accessed by a computer. In addition, it is known to those of ordinary skill in the art that communication media typically contain computer-readable program instructions, data structures, program modules or other data in a modulated data signal such as a carrier wave or other transmission mechanism, and may include any information delivery medium.

[0130] The computer-readable program instructions described herein can be downloaded from a computer-readable storage medium to each computing / processing device, or downloaded to an external computer or external storage device via a network, such as the Internet, a local area network, a wide area network, and / or a wireless network. The network can include copper transmission cables, optical fiber transmissions, wireless transmissions, routers, firewalls, switches, gateway computers, and / or edge servers. The network adapter card or network interface in each computing / processing device receives the computer-readable program instructions from the network and forwards the computer-readable program instructions for storage in the computer-readable storage medium in each computing / processing device.

[0131] The computer program instructions for performing the operation of the present disclosure may be assembly instructions, instruction set architecture (ISA) instructions, machine instructions, machine-related instructions, microcode, firmware instructions, state setting data, or source code or object code written in any combination of one or more programming languages, including object-oriented programming languages, such as Smalltalk, C++, etc., and conventional procedural programming languages, such as "C" language or similar programming languages. Computer-readable program instructions may be executed completely on a user's computer, partially on a user's computer, as an independent software package, partially on a user's computer, partially on a remote computer, or completely on a remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer via any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., using an Internet service provider to connect via the Internet). In some embodiments, an electronic circuit, such as a programmable logic circuit, a field programmable gate array (FPGA), or a programmable logic array (PLA), may be customized by utilizing the state information of the computer-readable program instructions, and the electronic circuit may execute the computer-readable program instructions, thereby realizing various aspects of the present disclosure.

[0132] The computer program product described herein may be implemented in hardware, software, or a combination thereof. In one optional embodiment, the computer program product is embodied as a computer storage medium, and in another optional embodiment, the computer program product is embodied as a software product, such as a software development kit (SDK), etc.

[0133] Various aspects of the present disclosure are described herein with reference to the flowcharts and / or block diagrams of the methods, devices (systems) and computer program products according to the embodiments of the present disclosure. It should be understood that each box in the flowchart and / or block diagram and the combination of each box in the flowchart and / or block diagram can be implemented by computer-readable program instructions.

[0134] These computer-readable program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, thereby producing a machine, so that when these instructions are executed by the processor of the computer or other programmable data processing device, a device that implements the functions / actions specified in one or more boxes in the flowchart and / or block diagram is generated. These computer-readable program instructions can also be stored in a computer-readable storage medium, and these instructions cause the computer, programmable data processing device, and / or other equipment to work in a specific manner, so that the computer-readable medium storing the instructions includes a manufactured product, which includes instructions for implementing various aspects of the functions / actions specified in one or more boxes in the flowchart and / or block diagram.

[0135] Computer-readable program instructions may also be loaded onto a computer, other programmable data processing apparatus, or other device so that a series of operating steps are performed on the computer, other programmable data processing apparatus, or other device to produce a computer-implemented process, thereby causing the instructions executed on the computer, other programmable data processing apparatus, or other device to implement the functions / actions specified in one or more boxes in the flowchart and / or block diagram.

[0136] The flow chart and block diagram in the accompanying drawings show the possible architecture, function and operation of the system, method and computer program product according to multiple embodiments of the present disclosure. In this regard, each square box in the flow chart or block diagram can represent a part of a module, program segment or instruction, and a part of the module, program segment or instruction includes one or more executable instructions for realizing the specified logical function. In some alternative implementations, the function marked in the square box can also occur in a sequence different from that marked in the accompanying drawings. For example, two continuous square boxes can actually be executed substantially in parallel, and they can sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each square box in the block diagram and / or flow chart, and the combination of the square boxes in the block diagram and / or flow chart can be implemented with a dedicated hardware-based system that performs the specified function or action, or can be implemented with a combination of special hardware and computer instructions.

[0137] Example embodiments have been disclosed herein, and although specific terms are employed, they are used and should be interpreted only in a general illustrative sense and not for limiting purposes. In some instances, it will be apparent to those skilled in the art that, unless otherwise expressly noted, features, characteristics, and / or elements described in conjunction with a particular embodiment may be used alone or in combination with features, characteristics, and / or elements described in conjunction with other embodiments. Therefore, those skilled in the art will appreciate that various changes in form and detail may be made without departing from the scope of the present disclosure as set forth in the appended claims.

Claims

1. A model monitoring method, characterized in that: include: Loading model data of the network model from a preset data source end, wherein the preset data end is used to store the model data of the network model in a distributed storage manner; Perform data preprocessing on the model data of the loaded network model; Performing data slicing processing on the preprocessed model data to obtain a plurality of data slices, each data slice including the preprocessed model data; For each data slice, perform data binning processing on the model data in the data slice to obtain a plurality of data buckets corresponding to the data slice, each data bucket including a plurality of model data in the data slice; Perform dimensionality reduction mapping processing on multiple data buckets in parallel; Using multiple threads in a preset thread pool, monitoring indicators are calculated in parallel based on the model data processed by dimensionality reduction mapping in each data bucket corresponding to each data slice, so as to obtain monitoring indicators corresponding to each data bucket, wherein the monitoring indicators are indicators reflecting the performance of the network model.

2. The method according to claim 1, characterized in that: The step of loading the model data of the network model from the preset data source includes: Using a preset data engine to load model data of the network model from a preset data source; In the process of loading the model data of the network model, data merging and data compression are performed on the model data of the network model.

3. The method according to claim 1, characterized in that The data preprocessing includes at least one of the following: Data cleaning processing, data missing value replacement processing, data discretization processing, data type conversion processing.

4. The method according to claim 1, characterized in that The data slicing process is performed on the preprocessed model data to obtain a plurality of data slices, including: Determine whether the ratio of the current remaining quantity of the preprocessed model data to the preset quantity is greater than a preset threshold, and the ratio of the total quantity of the preprocessed model data to the preset quantity is greater than the preset threshold; In the case where the ratio of the current remaining number to the preset number is greater than the preset threshold, obtaining the preset number of model data from the current remaining preprocessed model data to generate a data slice, and returning to execute the step of determining whether the ratio of the current remaining number of preprocessed model data to the preset number is greater than the preset threshold; When the ratio of the current remaining number to the preset number is less than or equal to the preset threshold, a data slice is generated according to the current remaining preprocessed model data.

5. The method according to claim 1, characterized in that The performing data binning processing on the model data in the data slice to obtain a plurality of data buckets corresponding to the data slice includes: Using an equal frequency binning method to bin the model data in the data slice to obtain a plurality of data buckets corresponding to the data slice; or, The model data in the data slice is binned using an equidistant binning method to obtain a plurality of data buckets corresponding to the data slice.

6. The method according to claim 1, characterized in that The parallel dimensionality reduction mapping process of the model data on the multiple data buckets includes: For multiple data buckets, the model data in each data bucket is processed by dimensionality reduction mapping in parallel through the reverse perspective method to obtain the model data after dimensionality reduction mapping in each data bucket.

7. The method according to claim 1, characterized in that In the case where the number of data slices is less than or equal to the number of threads in the preset thread pool, each data slice corresponds to a thread in the preset thread pool, and each thread in the preset thread pool is used to calculate the monitoring indicator for each corresponding data slice; In the case where the number of the data slices is greater than the number of threads in the preset thread pool, and the ratio of the number of the data slices to the number of threads in the preset thread pool is an integer, the multiple data slices are divided into multiple batches of data slices, wherein the number of batches of the data slices is the ratio, and each data slice in each batch of data slices corresponds to a thread in the preset thread pool, and each thread in the preset thread pool is used to calculate the monitoring indicator for each data slice in each corresponding batch of data slices; When the number of data slices is greater than the number of threads in the preset thread pool, and the ratio of the number of data slices to the number of threads in the preset thread pool is a non-integer, the multiple data slices are divided into multiple batches of data slices, wherein the number of batches of the data slices is the ratio rounded plus 1, and each data slice in each batch of data slices corresponds to a thread in the preset thread pool, and each thread in the preset thread pool is used to calculate monitoring indicators for each data slice in each corresponding batch of data slices.

8. A model monitoring device, characterized in that: include: A loading unit, used to load model data of the network model from a preset data source end, wherein the preset data end is used to store the model data of the network model in a distributed storage manner; A preprocessing unit, used for performing data preprocessing on model data of the loaded network model; A slicing unit, used for performing data slicing processing on the preprocessed model data to obtain a plurality of data slices, each data slice including a plurality of preprocessed model data; A binning unit, for binning the model data in each data slice to obtain a plurality of data buckets corresponding to the data slice, each data bucket including a plurality of model data in the data slice; A dimension reduction unit is used to perform dimension reduction mapping processing on multiple data buckets in parallel; The computing unit is used to utilize multiple threads in a preset thread pool to perform monitoring index calculations in parallel based on the model data processed by dimensionality reduction mapping in each data bucket corresponding to each data slice, and obtain monitoring indicators corresponding to each data bucket, wherein the monitoring indicators are indicators reflecting the performance of the network model.

9. An electronic device, characterized in that: include: at least one processor; as well as a memory communicatively connected to the at least one processor; wherein, The memory stores one or more computer programs executable by the at least one processor, and the one or more computer programs are executed by the at least one processor to enable the at least one processor to perform the model monitoring method according to any one of claims 1 to 7.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the computer program implements the model monitoring method according to any one of claims 1 to 7.

Images