A big data display system and method based on artificial intelligence

Abstract

It is based on the above problems that the present invention proposes an artificial intelligence-based big data display system and method, which generates first target data based on collected original data, where the first target data is data corresponding to a preset data dimension that needs to be displayed, and inputs the first target data into a pre-trained data state recognition model to identify whether there is an unstable second target data in the first target data. When there is an unstable second target data in the first target data, a screening parameter of the second target data is obtained, where the screening parameter is a parameter for filtering the second target data from the first target data, and a display method of the second target data is determined, where the display method includes a data chart type, and the second target data is displayed according to the display method. The data display method is more flexible, the data display content is more practical, and no manual intervention is required for configuration.

Description

Technical Field

[0001] The present invention relates to the field of big data technology, and in particular to a big data display system and method based on artificial intelligence. Background Art

[0002] Big data technology refers to the technology of mining valuable data information from massive data content, and analyzing, processing, storing and displaying the acquired information data. With the development of data acquisition technology and Internet technology, big data technology has been applied in more and more fields. For example, big data technology is widely used in the field of marketing, through mining and analyzing massive customer data, market data, and transaction data to achieve precision marketing and improve marketing efficiency. For another example, big data technology is widely used in the field of warehousing and logistics, based on the tracking and positioning data of massive items, transportation tools, communication tools, weather data, traffic data, etc., to achieve functions such as tracking and early warning of logistics status.

[0003] At present, the common way to display big data is to visualize the pre-set data dimensions in various data charts on the visualization screen. These pre-set data dimensions can be real-time data directly collected, or processed data obtained by statistical analysis of the collected data. As mentioned above, the data dimensions displayed by the data charts in the visualization screen are all pre-set. Under normal conditions, after the massive data is converted into several fixed data dimensions that need to be displayed, the data of the displayed data dimensions changes regularly within the standard numerical range. Therefore, in most cases, the data charts on the visualization screen are not worth paying attention to and using. This data display method is too rigid and lacks flexibility. Although some visualization screens provide rich custom configuration functions, too many configuration items bring cumbersome configuration operations, and even if more configurable items are set, it is still not enough to cover the massive data collected in the data source and the various possible changes hidden in these data. Summary of the invention

[0004] Based on the above problems, the present invention proposes a big data display system and method based on artificial intelligence. The data display method is more flexible, the data display content is more practical and no manual intervention is required.

[0005] In view of this, a first aspect of the present invention proposes a big data display system based on artificial intelligence, including a data acquisition device, a data processing device, a data storage device and a data display device, wherein the data processing device is configured as follows:

[0006] Generate first target data based on the collected original data, where the first target data is data corresponding to a preset data dimension that needs to be displayed;

[0007] Inputting the first target data into a pre-trained data state recognition model to identify whether there is second target data in an unstable state in the first target data;

[0008] When the first target data has second target data in an unstable state, obtaining a screening parameter of the second target data, the screening parameter being a parameter for obtaining the second target data by screening the first target data;

[0009] Determining a display method of the second target data, wherein the display method includes a data chart type;

[0010] The second target data is displayed in the display manner.

[0011] The second aspect of the present invention proposes a big data display method based on artificial intelligence, comprising:

[0012] Generate first target data based on the collected original data, where the first target data is data corresponding to a preset data dimension that needs to be displayed;

[0013] Inputting the first target data into a pre-trained data state recognition model to identify whether there is second target data in an unstable state in the first target data;

[0014] When the first target data has second target data in an unstable state, obtaining a screening parameter of the second target data, the screening parameter being a parameter for obtaining the second target data by screening the first target data;

[0015] Determining a display method of the second target data, wherein the display method includes a data chart type;

[0016] The second target data is displayed in the display manner.

[0017] Preferably, the step of generating the first target data based on the collected original data specifically includes:

[0018] Acquire a data identifier in the original data to identify a data category of the original data according to the data identifier;

[0019] Acquire a pre-configured conversion function between the original data and the first target data according to the data category;

[0020] Performing standardization processing on the original data to obtain standardized data;

[0021] The normalized data is converted into the first target data based on the conversion function.

[0022] Preferably, the big data display method based on artificial intelligence also includes:

[0023] Read historical normalized data from the database;

[0024] Processing the historical standardized data to obtain first sample data and second sample data, wherein the first sample data is sample data that does not include second target data in an unstable state, and the second sample data is sample data that includes second target data in an unstable state;

[0025] Use the first sample data and the second sample data to construct positive sample data and negative sample data in the training sample data of the data state recognition model;

[0026] The data state recognition model is trained using the positive sample data and the negative sample data.

[0027] Preferably, the step of processing the historical standardized data to obtain the first sample data and the second sample data specifically includes:

[0028] Obtaining screening parameters of the historical standard data;

[0029] Constructing a target data vector sequence corresponding to each combination of the screening parameters;

[0030] determining a high instability screening parameter combination among the screening parameters according to the target data vector sequence;

[0031] The second sample data is extracted from the target data vector sequence corresponding to the high instability screening parameter combination.

[0032] Preferably, the step of constructing a target data vector sequence corresponding to each combination of the screening parameters specifically includes:

[0033] Determine the number of statistical periods n in the time range covered by the historical standard data t and the number of screening parameter combinations n com ;

[0034] Construct the first target data of the jth statistical period n com target data vector Where i is 1 to n com A positive integer between 1 and n. t A positive integer between ;

[0035] Construct the target data vector sequence corresponding to the i-th screening parameter combination in chronological order

[0036] Preferably, the number of statistical time periods n in the time range covered by the historical standard data is determined t and the number of screening parameter combinations n com The steps specifically include:

[0037] Get the time range covered by the historical standard data [t start , t end ] and the pre-set historical standard data segmentation time T div ;

[0038] Based on the historical standard data segmentation time length T div The time range covered by the historical standard data [t start , t end ] is divided into n t Statistical period;

[0039] Convert the historical standard data of each statistical period into corresponding first target data;

[0040] Get the number of fields n of the first target data dim and the number of values ​​n for each field val，k , where k is from 1 to n dim A positive integer between ;

[0041] Calculate the number of screening parameter combinations for the first target data:

[0042]

[0043] Preferably, the step of determining a high instability screening parameter combination in the screening parameters according to the target data vector sequence specifically comprises:

[0044] Obtaining a preset discreteness threshold S0 corresponding to the data category;

[0045] Calculate the target data vector sequence corresponding to the i-th screening parameter combination Discreteness of:

[0046]

[0047] Determine the target data vector sequence The dispersion S i Is it greater than the discreteness threshold S0?

[0048] The dispersion S i The screening parameter combination greater than the discreteness threshold S0 is determined as the high instability screening parameter combination.

[0049] Preferably, the step of extracting the second sample data from the target data vector sequence corresponding to the high instability screening parameter combination specifically includes:

[0050] Obtaining a preset average deviation threshold DEV0 corresponding to the data category;

[0051] Obtain the target data vector sequence corresponding to the high instability screening parameter combination

[0052] Calculate the target data vector sequence Each vector in The average deviation DEV j ;

[0053] Determine the target data vector sequence Vector in The average deviation DEV j Is it greater than the average deviation threshold DEV0?

[0054] The target data vector sequence Mean Deviation DEV j Vectors greater than the average deviation threshold DEV0 The corresponding data is determined as the second sample data.

[0055] Preferably, the target data vector sequence is calculated Each vector in The average deviation DEV j The steps specifically include:

[0056] Calculate the target data vector sequence The mean of :

[0057]

[0058] Calculate the target data vector sequence Each vector in Relative to the mean The average deviation of

[0059]

[0060] It is based on the above problems that the present invention proposes an artificial intelligence-based big data display system and method, which generates first target data based on collected original data, where the first target data is data corresponding to a preset data dimension that needs to be displayed, and inputs the first target data into a pre-trained data state recognition model to identify whether there is an unstable second target data in the first target data. When there is an unstable second target data in the first target data, a screening parameter of the second target data is obtained, where the screening parameter is a parameter for filtering the second target data from the first target data, and a display method of the second target data is determined, where the display method includes a data chart type, and the second target data is displayed according to the display method. The data display method is more flexible, the data display content is more practical, and no manual intervention is required for configuration. BRIEF DESCRIPTION OF THE DRAWINGS

[0061] Figure 1 is a schematic diagram of a big data display system based on artificial intelligence provided by an embodiment of the present invention;

[0062] Figure 2 It is a flowchart of a big data display method based on artificial intelligence provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0063] In order to more clearly understand the above-mentioned purpose, features and advantages of the present invention, the present invention is further described in detail below in conjunction with the accompanying drawings and specific embodiments. It should be noted that the embodiments of the present application and the features in the embodiments can be combined with each other without conflict.

[0064] In the following description, many specific details are set forth to facilitate a full understanding of the present invention. However, the present invention may also be implemented in other ways different from those described herein. Therefore, the protection scope of the present invention is not limited to the specific embodiments disclosed below.

[0065] In the description of the present invention, the term "multiple" refers to two or more. Unless otherwise clearly defined, the orientation or positional relationship indicated by the terms "upper", "lower", etc. is based on the orientation or positional relationship shown in the drawings. It is only for the convenience of describing the present invention and simplifying the description, and does not indicate or imply that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation. Therefore, it cannot be understood as a limitation of the present invention. The terms "connection", "installation", "fixation", etc. should be understood in a broad sense. For example, "connection" can be a fixed connection, a detachable connection, or an integral connection; it can be directly connected or indirectly connected through an intermediate medium. For ordinary technicians in this field, the specific meanings of the above terms in the present invention can be understood according to the specific circumstances. In addition, the terms "first", "second", etc. are only used for descriptive purposes, and cannot be understood as indicating or implying relative importance or implicitly indicating the number of technical features indicated. Therefore, the features defined as "first", "second", etc. can explicitly or implicitly include one or more of the features. In the description of the present invention, unless otherwise specified, the meaning of "multiple" is two or more.

[0066] In the description of this specification, the description of the terms "one embodiment", "some implementations", "specific embodiments", etc. means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present invention. In this specification, the schematic representation of the above terms does not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described can be combined in any one or more embodiments or examples in a suitable manner.

[0067] An artificial intelligence-based big data display system and method provided according to some embodiments of the present invention are described below with reference to the accompanying drawings.

[0068] like Figure 1As shown, the first aspect of the present invention proposes a big data display system based on artificial intelligence, including a data acquisition device, a data processing device, a data storage device and a data display device. Specifically, the data acquisition device can be a remote data acquisition device, including sensors such as positioning devices, acceleration sensors, temperature sensors, pressure sensors or flow sensors installed on goods, vehicles or mobile communication terminals. The data acquisition device can also be a personal computer or workstation that collects data from a public network through the Internet. The data processing device can be a computer device or a computer cluster that runs a data processing program, which is mainly used to process and analyze the collected data, and provide the processing and analysis results to the data storage device for storage, and to the data display device for display. A database program runs in the data storage device, and the data provided by the data processing device is persistently stored through the database program. The data display device can be a computer display, a projector or other display device.

[0069] like Figure 2 As shown, the data processing device is configured as follows:

[0070] Generate first target data based on the collected original data, where the first target data is data corresponding to a preset data dimension that needs to be displayed;

[0071] Inputting the first target data into a pre-trained data state recognition model to identify whether there is second target data in an unstable state in the first target data;

[0072] When the first target data has second target data in an unstable state, obtaining a screening parameter of the second target data, the screening parameter being a parameter for obtaining the second target data by screening the first target data;

[0073] Determining a display method of the second target data, wherein the display method includes a data chart type;

[0074] The second target data is displayed in the display manner.

[0075] Specifically, the original data is data collected by a data collection device, such as positioning data collected by a positioning device installed on goods, vehicles or mobile communication terminals, or weather data, road condition data, etc. collected by a computer responsible for executing data collection from a weather data source, a road condition data source on the Internet. The first target data is data obtained by standardizing, counting or converting data of one or more data dimensions in the original data. For example, when the original data is weather data, the first target data can be data directly collected from the weather data such as temperature data, humidity data, wind data, etc., and then standardized, or it can be data obtained by counting or converting the original data such as daily average temperature, regional average rainfall, day and night temperature difference, etc.

[0076] The screening parameters include object parameters, time parameters, and region parameters. The object parameters include object name parameters and object attribute parameters, such as customer name, customer type, and product name, product category, or product color. The time parameter may be the time when the transaction event occurs, the time when the positioning data is collected, the time when the weather data is collected, or the time when the road condition data is collected. The region parameter may be the location of the customer, the place where the product is shipped, or the place where the product is received.

[0077] In the step of determining the display mode of the second target data, the display mode of the second target data is determined by identifying the data type, data volume and / or data range of the second target data. Identifying the data type of the second target data mainly means identifying whether the second target data is numerical data, categorical data or relational data. For numerical data, a bar chart or a line chart can be used for display; for categorical data, a bar chart or a pie chart can be used for display; for relational data, a scatter chart or a bubble chart can be used for display. The data volume of the second target data refers to the amount of data contained in the second target data. For the second target data with a small amount of data, a bar chart or a pie chart can be used for display; for the second target data with a large amount of data, an area chart, a tree chart or a heat map can be used for display. The data range of the second target data mainly represents the data span of the second target data. For the second target data with a small data span, a bar chart or a line chart can be used for display; for the second target data with a large data span, a logarithmic coordinate chart can be used for display.

[0078] Preferably, in the step of generating the first target data based on the collected original data, the data processing device is configured to:

[0079] Acquire a data identifier in the original data to identify a data category of the original data according to the data identifier;

[0080] Acquire a pre-configured conversion function between the original data and the first target data according to the data category;

[0081] Performing standardization processing on the original data to obtain standardized data;

[0082] The normalized data is converted into the first target data based on the conversion function.

[0083] Specifically, the standardization process includes a denoising step, a quantization step, a dimensional conversion step and / or a normalization step. The denoising step is used to identify and filter obvious abnormal data in the original data. The quantization step is used to convert the non-quantized data in the original data into quantitative data, such as converting text data into vector data that can be quantified and evaluated. The dimensional conversion step is used to process the quantitative data in the original data into a unified dimension. The normalization step is used to map the quantized or dimensional converted data into a unified numerical range for subsequent processing.

[0084] The data category refers to the information category represented by the data itself, which varies according to the data source. The data category includes transaction data, positioning data, weather data, road condition data, etc. In the technical solution of the above implementation, the data acquisition device will carry a data identifier representing the data category of the original data in the data packet of the original data packaged and output.

[0085] In the technical solutions of some embodiments of the present invention, the first target data includes data of one or more data dimensions in the original data. For example, when the original data is weather data, the first target data may be temperature data and / or humidity data, etc. In this case, the first target data can be directly extracted after the original data is standardized without the need for conversion through the conversion function.

[0086] In the technical solutions of other embodiments of the present invention, the first target data includes data obtained after statistics or conversion processing of the original data. Still taking the above-mentioned weather data as an example, the original data includes real-time temperature monitoring data of several temperature monitoring points in a certain area, and the first target data includes real-time average temperature data of the area, which is calculated by the real-time temperature monitoring data of these temperature monitoring points through a conversion function. In this embodiment, the conversion function is an average function.

[0087] Preferably, after the step of performing standardization processing on the original data to obtain standardized data, the data processing device is further configured to persist the standardized data in a database.

[0088] Preferably, the data processing device is further configured to:

[0089] Read historical normalized data from the database;

[0090] Processing the historical standardized data to obtain first sample data and second sample data, wherein the first sample data is sample data that does not include second target data in an unstable state, and the second sample data is sample data that includes second target data in an unstable state;

[0091] Use the first sample data and the second sample data to construct positive sample data and negative sample data in the training sample data of the data state recognition model;

[0092] The data state recognition model is trained using the positive sample data and the negative sample data.

[0093] Specifically, the historical standardized data is the original data obtained from historical collection and standardized processing and stored in the database. The data state recognition model is a data classification model trained based on the historical standardized data. The data state recognition model can be any artificial intelligence model among CNN (Convolutional Neural Networks) model, SVM (Support Vector Machine) model, decision tree model and random forest model. The data state recognition model is a supervised learning model, that is, the data state recognition model is trained based on a labeled data set, and the so-called labeled data set includes a first sample data that does not contain the second target data of the unstable state and a second sample data that contains the second target data of the unstable state, wherein the first sample data is used to construct a positive sample data set, and the second sample data is used to construct a negative sample data set.

[0094] Preferably, in the step of processing the historical standardized data to obtain the first sample data and the second sample data, the data processing device is configured to:

[0095] Obtaining screening parameters of the historical standard data;

[0096] Constructing a target data vector sequence corresponding to each combination of the screening parameters;

[0097] determining a high instability screening parameter combination among the screening parameters according to the target data vector sequence;

[0098] The second sample data is extracted from the target data vector sequence corresponding to the high instability screening parameter combination.

[0099] Preferably, the step of obtaining the screening parameters of the historical standard data is specifically to screen the parameters of the second target data from the first target data after converting the historical standard data into the first target data. The high instability screening parameter combination is a combination of multiple screening parameters of the second target data having a high probability of instability obtained by screening the first target data converted from the historical standard data.

[0100] Preferably, in the step of constructing a target data vector sequence corresponding to each combination of the screening parameters, the data processing device is configured to:

[0101] Determine the number of statistical periods n in the time range covered by the historical standard data t and the number of screening parameter combinations n com ;

[0102] Construct the first target data of the jth statistical period n com target data vector Where i is 1 to n com A positive integer between 1 and n. t A positive integer between ;

[0103] Construct the target data vector sequence corresponding to the i-th screening parameter combination in chronological order

[0104] In the technical solution of the above implementation mode, the time range covered by the historical standard data is divided into n t statistical periods, and converting the historical standardized data in each statistical period into n of the first target data com The data records obtained by the combination of screening parameters are mapped to the vector space to obtain the corresponding n com Vector Thus, the n corresponding to each combination of screening parameters t Vector It is constructed as the target data vector sequence in chronological order.

[0105] Preferably, in determining the number of statistical time periods n in the time range covered by the historical standard data t and the number of screening parameter combinations n com In the step, the data processing device is configured to:

[0106] Get the time range covered by the historical standard data [t start , t end ] and the pre-set historical standard data segmentation time T div ;

[0107] Based on the historical standard data segmentation time length T div The time range covered by the historical standard data [t start , t end ] is divided into n t Statistical period;

[0108] Convert the historical standard data of each statistical period into corresponding first target data;

[0109] Get the number of fields n of the first target data dim and the number of values ​​n for each field val，k , where k is from 1 to n dim A positive integer between ;

[0110] Calculate the number of screening parameter combinations for the first target data:

[0111]

[0112] Specifically, start is the starting time of the time range covered by the historical standard data, t end Preferably, the historical standard data can be divided into time lengths T according to the different data categories of the historical standard data and the different data collection frequencies. div The time length T is configured in hours or days, for example, several hours or several days. div The time range covered by the historical standard data [t start , t end ] is divided into n t The step of composing a statistical period is specifically to convert the time range covered by the historical standard data [t start , t end ] is divided equally into n t The time length is the historical standard data segmentation time length T div The statistical period, that is, the number of statistical periods in the time range covered by the historical standard data The length of each statistical period is the historical standard data segmentation time length T div .

[0113] Preferably, in the step of determining a high instability screening parameter combination in the screening parameters according to the target data vector sequence, the data processing device is configured to:

[0114] Obtaining a preset discreteness threshold S0 corresponding to the data category;

[0115] Calculate the target data vector sequence corresponding to the i-th screening parameter combination Discreteness of:

[0116]

[0117] Determine the target data vector sequence The dispersion S i Is it greater than the discreteness threshold S0?

[0118] The dispersion S i The screening parameter combination greater than the discreteness threshold S0 is determined as the high instability screening parameter combination.

[0119] It should be noted that, since the discreteness performance of different categories of data in the time dimension is often different, in the technical solution of the present invention, a corresponding discreteness threshold S0 is set in advance for each different data category, so that the stability of the screening parameter combination corresponding to the data of different data categories can be evaluated separately according to the discreteness threshold S0.

[0120] Preferably, in the step of extracting the second sample data from the target data vector sequence corresponding to the high instability screening parameter combination, the data processing device is configured to:

[0121] Obtaining a preset average deviation threshold DEV0 corresponding to the data category;

[0122] Obtain the target data vector sequence corresponding to the high instability screening parameter combination

[0123] Calculate the target data vector sequence Each vector in The average deviation DEV j ;

[0124] Determine the target data vector sequence Vector in The average deviation DEV j Is it greater than the average deviation threshold DEV0?

[0125] The target data vector sequence Mean Deviation DEV j Vectors greater than the average deviation threshold DEV0 The corresponding data is determined as the second sample data.

[0126] In the technical solutions of some embodiments of the present invention, when determining the target data vector sequence The dispersion S iAfter the step of whether the discreteness is greater than the discreteness threshold S0, the method further includes:

[0127] The dispersion S i A screening parameter combination that is less than or equal to the discreteness threshold S0 is determined as a stable screening parameter combination;

[0128] The step of processing the historical standardized data to obtain the first sample data and the second sample data also includes:

[0129] The data corresponding to each vector in the target data vector sequence corresponding to the stable screening parameter combination is determined as the first sample data.

[0130] Preferably, when calculating the target data vector sequence Each vector in In the step of calculating the average deviation DEVj, the data processing device is configured to:

[0131] Calculate the target data vector sequence The mean of :

[0132]

[0133] Calculate the target data vector sequence Each vector in Relative to the mean The average deviation of

[0134]

[0135] like Figure 2 As shown, the second aspect of the present invention proposes a big data display method based on artificial intelligence, comprising:

[0136] Generate first target data based on the collected original data, where the first target data is data corresponding to a preset data dimension that needs to be displayed;

[0137] Inputting the first target data into a pre-trained data state recognition model to identify whether there is second target data in an unstable state in the first target data;

[0138] When the first target data has second target data in an unstable state, obtaining a screening parameter of the second target data, the screening parameter being a parameter for obtaining the second target data by screening the first target data;

[0139] Determining a display method of the second target data, wherein the display method includes a data chart type;

[0140] The second target data is displayed in the display manner.

[0141] Specifically, the original data is data collected by a data collection device, such as positioning data collected by a positioning device installed on goods, vehicles or mobile communication terminals, or weather data, road condition data, etc. collected by a computer responsible for executing data collection from a weather data source, a road condition data source on the Internet. The first target data is data obtained by standardizing, counting or converting data of one or more data dimensions in the original data. For example, when the original data is weather data, the first target data can be data directly collected from the weather data such as temperature data, humidity data, wind data, etc., and then standardized, or it can be data obtained by counting or converting the original data such as daily average temperature, regional average rainfall, day and night temperature difference, etc.

[0142] The screening parameters include object parameters, time parameters, and region parameters. The object parameters include object name parameters and object attribute parameters, such as customer name, customer type, and product name, product category, or product color. The time parameter may be the time when the transaction event occurs, the time when the positioning data is collected, the time when the weather data is collected, or the time when the road condition data is collected. The region parameter may be the location of the customer, the place where the product is shipped, or the place where the product is received.

[0143] In the step of determining the display mode of the second target data, the display mode of the second target data is determined by identifying the data type, data volume and / or data range of the second target data. Identifying the data type of the second target data mainly means identifying whether the second target data is numerical data, categorical data or relational data. For numerical data, a bar chart or a line chart can be used for display; for categorical data, a bar chart or a pie chart can be used for display; for relational data, a scatter chart or a bubble chart can be used for display. The data volume of the second target data refers to the amount of data contained in the second target data. For the second target data with a small amount of data, a bar chart or a pie chart can be used for display; for the second target data with a large amount of data, an area chart, a tree chart or a heat map can be used for display. The data range of the second target data mainly represents the data span of the second target data. For the second target data with a small data span, a bar chart or a line chart can be used for display; for the second target data with a large data span, a logarithmic coordinate chart can be used for display.

[0144] Preferably, the step of generating the first target data based on the collected original data specifically includes:

[0145] Acquire a data identifier in the original data to identify a data category of the original data according to the data identifier;

[0146] Acquire a pre-configured conversion function between the original data and the first target data according to the data category;

[0147] Performing standardization processing on the original data to obtain standardized data;

[0148] The normalized data is converted into the first target data based on the conversion function.

[0149] Specifically, the standardization process includes a denoising step, a quantization step, a dimensional conversion step and / or a normalization step. The denoising step is used to identify and filter obvious abnormal data in the original data. The quantization step is used to convert the non-quantized data in the original data into quantitative data, such as converting text data into vector data that can be quantified and evaluated. The dimensional conversion step is used to process the quantitative data in the original data into a unified dimension. The normalization step is used to map the quantized or dimensional converted data into a unified numerical range for subsequent processing.

[0150] The data category refers to the information category represented by the data itself, which varies according to the data source. The data category includes transaction data, positioning data, weather data, road condition data, etc. In the technical solution of the above implementation, the data acquisition device will carry a data identifier representing the data category of the original data in the data packet of the original data packaged and output.

[0151] In the technical solutions of some embodiments of the present invention, the first target data includes data of one or more data dimensions in the original data. For example, when the original data is weather data, the first target data may be temperature data and / or humidity data, etc. In this case, the first target data can be directly extracted after the original data is standardized without the need for conversion through the conversion function.

[0152] In the technical solutions of other embodiments of the present invention, the first target data includes data obtained after statistics or conversion processing of the original data. Still taking the above-mentioned weather data as an example, the original data includes real-time temperature monitoring data of several temperature monitoring points in a certain area, and the first target data includes real-time average temperature data of the area, which is calculated by the real-time temperature monitoring data of these temperature monitoring points through a conversion function. In this embodiment, the conversion function is an average function.

[0153] Preferably, after the step of performing standardization processing on the original data to obtain standardized data, the method further includes persisting the standardized data in a database.

[0154] Preferably, the big data display method based on artificial intelligence also includes:

[0155] Read historical normalized data from the database;

[0156] Processing the historical standardized data to obtain first sample data and second sample data, wherein the first sample data is sample data that does not include second target data in an unstable state, and the second sample data is sample data that includes second target data in an unstable state;

[0157] Use the first sample data and the second sample data to construct positive sample data and negative sample data in the training sample data of the data state recognition model;

[0158] The data state recognition model is trained using the positive sample data and the negative sample data.

[0159] Specifically, the historical standardized data is the original data obtained from historical collection and standardized processing and stored in the database. The data state recognition model is a data classification model trained based on the historical standardized data. The data state recognition model can be any artificial intelligence model among CNN (Convolutional Neural Networks) model, SVM (Support Vector Machine) model, decision tree model and random forest model. The data state recognition model is a supervised learning model, that is, the data state recognition model is trained based on a labeled data set, and the so-called labeled data set includes a first sample data that does not contain the second target data of the unstable state and a second sample data that contains the second target data of the unstable state, wherein the first sample data is used to construct a positive sample data set, and the second sample data is used to construct a negative sample data set.

[0160] Preferably, the step of processing the historical standardized data to obtain the first sample data and the second sample data specifically includes:

[0161] Obtaining screening parameters of the historical standard data;

[0162] Constructing a target data vector sequence corresponding to each combination of the screening parameters;

[0163] determining a high instability screening parameter combination among the screening parameters according to the target data vector sequence;

[0164] The second sample data is extracted from the target data vector sequence corresponding to the high instability screening parameter combination.

[0165] Preferably, the step of obtaining the screening parameters of the historical standard data is specifically to screen the parameters of the second target data from the first target data after converting the historical standard data into the first target data. The high instability screening parameter combination is a combination of multiple screening parameters of the second target data having a high probability of instability obtained by screening the first target data converted from the historical standard data.

[0166] Preferably, the step of constructing a target data vector sequence corresponding to each combination of the screening parameters specifically includes:

[0167] Determine the number of statistical periods n in the time range covered by the historical standard data t and the number of screening parameter combinations n com ;

[0168] Construct the first target data of the jth statistical period n com target data vector Where i is 1 to n com A positive integer between 1 and n. t A positive integer between ;

[0169] Construct the target data vector sequence corresponding to the i-th screening parameter combination in chronological order

[0170] In the technical solution of the above implementation mode, the time range covered by the historical standard data is divided into n t statistical periods, and converting the historical standardized data in each statistical period into n of the first target data com The data records obtained by the combination of screening parameters are mapped to the vector space to obtain the corresponding n com Vector Thus, the n corresponding to each combination of screening parameters t Vector It is constructed as the target data vector sequence in chronological order.

[0171] Preferably, the number of statistical time periods n in the time range covered by the historical standard data is determined t and the number of screening parameter combinations n com The steps specifically include:

[0172] Get the time range covered by the historical standard data [t start , t end ] and the pre-set historical standard data segmentation time Tdiv ;

[0173] Based on the historical standard data segmentation time length T div The time range covered by the historical standard data [t start , t end ] is divided into n t Statistical period;

[0174] Convert the historical standard data of each statistical period into corresponding first target data;

[0175] Get the number of fields n of the first target data dim and the number of values ​​n for each field val，k , where k is from 1 to n dim A positive integer between ;

[0176] Calculate the number of screening parameter combinations for the first target data:

[0177]

[0178] Specifically, start is the starting time of the time range covered by the historical standard data, t end Preferably, the historical standard data can be divided into time lengths T according to the different data categories of the historical standard data and the different data collection frequencies. div The time length T is configured in hours or days, for example, several hours or several days. div The time range covered by the historical standard data [t start , t end ] is divided into n t The step of composing a statistical period is specifically to convert the time range covered by the historical standard data [t start , t end ] is divided equally into n t The time length is the historical standard data segmentation time length T div The statistical period, that is, the number of statistical periods in the time range covered by the historical standard data The length of each statistical period is the historical standard data segmentation time length T div .

[0179] Preferably, the step of determining a high instability screening parameter combination in the screening parameters according to the target data vector sequence specifically comprises:

[0180] Obtaining a preset discreteness threshold S0 corresponding to the data category;

[0181] Calculate the target data vector sequence corresponding to the i-th screening parameter combination Discreteness of:

[0182]

[0183] Determine the target data vector sequence The dispersion S i Is it greater than the discreteness threshold S0?

[0184] The dispersion S i The screening parameter combination greater than the discreteness threshold S0 is determined as the high instability screening parameter combination.

[0185] It should be noted that, since the discreteness performance of different categories of data in the time dimension is often different, in the technical solution of the present invention, a corresponding discreteness threshold S0 is set in advance for each different data category, so that the stability of the screening parameter combination corresponding to the data of different data categories can be evaluated separately according to the discreteness threshold S0.

[0186] Preferably, the step of extracting the second sample data from the target data vector sequence corresponding to the high instability screening parameter combination specifically includes:

[0187] Obtaining a preset average deviation threshold DEV0 corresponding to the data category;

[0188] Obtain the target data vector sequence corresponding to the high instability screening parameter combination

[0189] Calculate the target data vector sequence Each vector in The average deviation DEV j ;

[0190] Determine the target data vector sequence Vector in The average deviation DEV j Is it greater than the average deviation threshold DEV0?

[0191] The target data vector sequence Mean Deviation DEV j Vectors greater than the average deviation threshold DEV0 The corresponding data is determined as the second sample data.

[0192] In the technical solutions of some embodiments of the present invention, when determining the target data vector sequence The dispersion S iAfter the step of whether the discreteness is greater than the discreteness threshold S0, the method further includes:

[0193] The dispersion S i A screening parameter combination that is less than or equal to the discreteness threshold S0 is determined as a stable screening parameter combination;

[0194] The step of processing the historical standardized data to obtain the first sample data and the second sample data also includes:

[0195] The data corresponding to each vector in the target data vector sequence corresponding to the stable screening parameter combination is determined as the first sample data.

[0196] Preferably, the target data vector sequence is calculated Each vector in The average deviation DEV j The steps specifically include:

[0197] Calculate the target data vector sequence The mean of :

[0198]

[0199] Calculate the target data vector sequence Each vector in Relative to the mean The average deviation of

[0200]

[0201] It should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the statement "comprise a ..." do not exclude the presence of other identical elements in the process, method, article or device including the elements.

[0202] According to the embodiments of the present invention as described above, these embodiments do not describe all the details in detail, nor do they limit the invention to the specific embodiments described. Obviously, many modifications and changes can be made based on the above description. This specification selects and specifically describes these embodiments in order to better explain the principles and practical applications of the present invention, so that those skilled in the art can make good use of the present invention and the modified use based on the present invention. The present invention is limited only by the claims and their full scope and equivalents.

Claims

1. A big data display system based on artificial intelligence, characterized in that: It includes a data acquisition device, a data processing device, a data storage device and a data display device, and the data processing device is configured as follows: Read historical normalized data from the database; Processing the historical standardized data to obtain first sample data and second sample data, wherein the first sample data is sample data that does not include second target data in an unstable state, and the second sample data is sample data that includes second target data in an unstable state; constructing positive sample data and negative sample data in training sample data of a data state recognition model using the first sample data and the second sample data; Using the positive sample data and the negative sample data to train a data state recognition model; In the step of processing the historical standardized data to obtain the first sample data and the second sample data, the data processing device is configured to: Obtaining screening parameters of the historical standard data; Constructing a target data vector sequence corresponding to each combination of the screening parameters; determining a high instability screening parameter combination among the screening parameters according to the target data vector sequence; Extracting the second sample data from the target data vector sequence corresponding to the high instability screening parameter combination; Generate first target data based on the collected original data, where the first target data is data corresponding to a preset data dimension that needs to be displayed; Inputting the first target data into a pre-trained data state recognition model to identify whether there is second target data in an unstable state in the first target data; When the first target data has second target data in an unstable state, obtaining a screening parameter of the second target data, the screening parameter being a parameter for obtaining the second target data by screening the first target data; Determining a display method of the second target data, wherein the display method includes a data chart type; Displaying the second target data in the display manner; In the step of determining a high instability screening parameter combination in the screening parameters according to the target data vector sequence, the data processing device is configured to: Obtain a preset discreteness threshold S0 corresponding to the data category; Calculate the target data vector sequence corresponding to the i-th screening parameter combination Discreteness of: Determine the target data vector sequence The dispersion S i Is it greater than the discreteness threshold S0? The dispersion S i The screening parameter combination greater than the discreteness threshold S0 is determined as the high instability screening parameter combination; In the step of extracting the second sample data from the target data vector sequence corresponding to the high instability screening parameter combination, the data processing device is configured to: Obtain a preset average deviation threshold DEV0 of the corresponding data category; Obtain the target data vector sequence corresponding to the high instability screening parameter combination Calculate the target data vector sequence Each vector in The average deviation DEV j ; Determine the target data vector sequence Vector in The average deviation DEV j Is it greater than the average deviation threshold EDV0? The target data vector sequence Mean Deviation DEV j Vectors greater than the average deviation threshold DEV0 The corresponding data is determined as the second sample data; In calculating the target data vector sequence Each vector in The average deviation DEV j In the step, the data processing device is configured to: Calculate the target data vector sequence The mean of : Calculate the target data vector sequence Each vector in Relative to the mean The average deviation of 2. A big data display method based on artificial intelligence, characterized in that: include: Read historical normalized data from the database; Processing the historical standardized data to obtain first sample data and second sample data, wherein the first sample data is sample data that does not include second target data in an unstable state, and the second sample data is sample data that includes second target data in an unstable state; constructing positive sample data and negative sample data in training sample data of a data state recognition model using the first sample data and the second sample data; Using the positive sample data and the negative sample data to train a data state recognition model; The step of processing the historical standardized data to obtain the first sample data and the second sample data specifically includes: Obtaining screening parameters of the historical standard data; Constructing a target data vector sequence corresponding to each combination of the screening parameters; determining a high instability screening parameter combination among the screening parameters according to the target data vector sequence; Extracting the second sample data from the target data vector sequence corresponding to the high instability screening parameter combination; Generate first target data based on the collected original data, where the first target data is data corresponding to a preset data dimension that needs to be displayed; Inputting the first target data into a pre-trained data state recognition model to identify whether there is second target data in an unstable state in the first target data; When the first target data has second target data in an unstable state, obtaining a screening parameter of the second target data, the screening parameter being a parameter for obtaining the second target data by screening the first target data; Determining a display method of the second target data, wherein the display method includes a data chart type; Displaying the second target data in the display manner; The step of determining a high instability screening parameter combination in the screening parameters according to the target data vector sequence specifically comprises: Obtain a preset discreteness threshold S0 corresponding to the data category; Calculate the target data vector sequence corresponding to the i-th screening parameter combination Discreteness of: Determine the target data vector sequence The dispersion S i Is it greater than the discreteness threshold S0? The dispersion S i The screening parameter combination greater than the discreteness threshold S0 is determined as the high instability screening parameter combination; The step of extracting the second sample data from the target data vector sequence corresponding to the high instability screening parameter combination specifically includes: Obtain a preset average deviation threshold DEV0 of the corresponding data category; Obtain the target data vector sequence corresponding to the high instability screening parameter combination Calculate the target data vector sequence Each vector in The average deviation DEV j ; Determine the target data vector sequence Vector in The average deviation DEV j Is it greater than the average deviation threshold EDV0? The target data vector sequence Mean Deviation DEV j Vectors greater than the average deviation threshold DEV0 The corresponding data is determined as the second sample data; Calculate the target data vector sequence Each vector in The average deviation DEV j The steps specifically include: Calculate the target data vector sequence The mean of : Calculate the target data vector sequence Each vector in Relative to the mean The average deviation of 3. The big data display method based on artificial intelligence according to claim 2 is characterized in that: The step of generating the first target data based on the collected original data specifically includes: Acquire a data identifier in the original data to identify a data category of the original data according to the data identifier; Acquire a pre-configured conversion function between the original data and the first target data according to the data category; Performing standardization processing on the original data to obtain standardized data; The normalized data is converted into the first target data based on the conversion function.

4. The big data display method based on artificial intelligence according to claim 2 is characterized in that: The steps of constructing a target data vector sequence corresponding to each combination of the screening parameters specifically include: Determine the number of statistical periods n in the time range covered by the historical standard data t and the number of screening parameter combinations n com ; Construct the first target data of the jth statistical period n com target data vector Where i is 1 to n com A positive integer between 1 and n. t A positive integer between ; Construct the target data vector sequence corresponding to the i-th screening parameter combination in chronological order 5. The big data display method based on artificial intelligence according to claim 4 is characterized in that: Determine the number of statistical periods n in the time range covered by the historical standard data t and the number of screening parameter combinations n com The steps specifically include: Get the time range covered by the historical standard data [t stsrt ,t end ] and the pre-set historical standard data segmentation time T div ; Based on the historical standard data segmentation time length T div The time range covered by the historical standard data [t start ,t end ] is divided into n t Statistical period; Convert the historical standard data of each statistical period into corresponding first target data; Get the number of fields n of the first target data dim and the number of values ​​n for each field val,k , where j is from 1 to n dim A positive integer between ; Calculate the number of screening parameter combinations for the first target data:

Images