Media intelligent decision operation method and system and medium

Abstract

The embodiment of the invention provides a media intelligent decision operation method and system and a medium, and the method comprises the steps: obtaining operation data, carrying out the preprocessing of the operation data, and obtaining the preprocessing data; inputting the preprocessed data into an AI large model, and outputting user demand information; comparing the user demand information with set condition information to obtain a demand deviation rate; judging whether the demand deviation rate is greater than or equal to a set demand deviation rate threshold value; if yes, generating conversion information, and performing format conversion on the preprocessed data based on the conversion information to obtain converted operation data; if yes, transmitting the preprocessed data to the terminal in real time; through an advanced data processing technology, efficient processing and intelligent analysis of large-scale multimedia data and business data are realized, so that a decision maker and an operator can acquire desired data through an interactive page, decision reports and operation automation are realized, and the operation efficiency and decision quality of provincial media are improved.

Description

Technical Field

[0001] The present application relates to the field of media decision technology, and more specifically, to a media intelligent decision-making operation method, system and medium. Background Art

[0002] With the rapid development of information technology, provincial media generate various types of data during operation, including multimedia data such as text, pictures, and videos, as well as user behavior data and business operation data. The scale of these data is huge and growing rapidly, which brings great challenges to data processing and analysis. Large amount of data: The amount of data that needs to be processed is huge, and traditional data processing methods are difficult to cope with. Complex data types: The data types are diverse, requiring more complex data processing technology, and high real-time requirements: Decisions in the media industry often require rapid responses, and have high requirements for the real-time nature of data processing. Before the needs initiated by decision makers and operators, it is impossible to quickly verify the logic, verify the needs, and obtain data results through the existing data.

[0003] The media industry is faced with the need to process and analyze massive amounts of data and then make decisions and operate. Traditional data processing, analysis, decision-making, and operation methods are mainly based on tools such as ETL and BI reports and rely on manual operations. This operation mode has poor timeliness and data timeliness, and the demand conversion link is long and inefficient. Understanding the degree of conversion determines the final result and is easily affected by subjective judgment, resulting in inaccurate decisions. Existing ETL systems usually use fixed data processing models when processing media data, which lack sufficient flexibility and scalability. In addition, these systems often require a lot of manual intervention to complete data pre-processing and post-processing, and cannot meet the high requirements of provincial media for real-time, accuracy and intelligence of data processing. The existing traditional big data intelligent decision-making operation system does not have an interactive demand page. When initiating a demand, decision makers and operators need to conduct multiple rounds of demand reviews with relevant personnel, and before the demand is initiated, decision makers and operators cannot preliminarily verify the demand. In the existing intelligent decision-making operation system, when decision makers and operators want to submit demands temporarily and return results quickly, they cannot respond in time and need other relevant personnel to access, which is relatively inefficient and cumbersome. Effective technical solutions are urgently needed for the above problems. Summary of the invention

[0004] The purpose of the embodiments of the present application is to provide a media intelligent decision-making operation method, system and medium, which, through advanced data processing technology, can achieve efficient processing and intelligent analysis of large-scale multimedia data and business data, so that decision makers and operators can obtain the desired data through interactive pages, realize decision-making reports and operation automation, thereby improving the operational efficiency and decision-making quality of provincial media.

[0005] The present application also provides a media intelligent decision-making operation method, including:

[0006] Acquire operation data, pre-process the operation data, and obtain pre-processed data;

[0007] Input preprocessed data into the AI ​​big model and output user demand information;

[0008] Compare the user demand information with the set condition information to obtain the demand deviation rate;

[0009] Determining whether the demand deviation rate is greater than or equal to a set demand deviation rate threshold;

[0010] If it is greater than or equal to, conversion information is generated, and the format of the pre-processed data is converted based on the conversion information to obtain the converted operation data;

[0011] If it is less than, the preprocessed data will be transmitted to the terminal in real time.

[0012] Optionally, in the media intelligent decision-making operation method described in the embodiment of the present application, obtaining operation data, preprocessing the operation data to obtain preprocessed data specifically includes:

[0013] Obtain network collection data, production business data and tracking point log data, and transmit them to offline ETL;

[0014] Based on offline ETL, data features are extracted from network collection data, production business data and tracking point log data to obtain network collection data features, production business data features and tracking point log data features;

[0015] Analyze whether the characteristics of network collection data, production business data and tracking point log data meet the set data standard information;

[0016] If it meets the requirements, an ETL data log is generated to obtain preprocessed data;

[0017] If it does not meet the requirements, the analysis results are obtained, and based on the analysis results, the network collection data, production business data and tracking point log data are cleaned and converted to obtain pre-processed data.

[0018] Optionally, in the media intelligent decision-making operation method described in the embodiment of the present application, data cleaning and data conversion are performed on network collection data, production business data and embedding log data based on the analysis results to obtain pre-processed data, specifically including:

[0019] Obtain analysis results, analyze the consistency of network collection data based on the analysis results to obtain a first similarity, analyze the consistency of production business data based on the analysis results to obtain a second similarity, and analyze the consistency of tracking point log data based on the analysis results to obtain a third similarity;

[0020] Determine whether the first similarity, the second similarity, and the third similarity all reach a set similarity threshold;

[0021] If the similarity threshold is reached, the duplicate data between network collection data, production business data and tracking point log data will be analyzed based on the analysis results, and the duplicate data will be cleaned;

[0022] If one of them does not reach the set similarity threshold, the network collection data, production business data and tracking log data are converted, and the converted network collection data, production business data and tracking log data are cleaned to obtain preprocessed data.

[0023] Optionally, in the media intelligent decision-making operation method described in the embodiment of the present application, network collection data, production business data and embedding log data are obtained and transmitted to the offline ETL, specifically including:

[0024] Synchronize network collected data to Hybase database based on API;

[0025] Synchronize production business data to offline ETL using Sqoop / DataX synchronization tools, and synchronize data to real-time ETL using FlinkCDC;

[0026] Synchronize the tracking log data to SLS based on Logfile or ilogtail tools, deliver the data to offline ETL based on SLS, send the tracking log data to Kafka based on API, and consume Kafka data based on real-time ETL;

[0027] Maxcomputer+dataworks components and a five-layer data warehouse architecture are used for layered ETL processing. The processed result data and embedded point log data are exported to Greenplume, Clickhouse, and Hbase.

[0028] The public cloud version of Kafka+Flink+Hbase components and the five-layer data warehouse architecture are used for layered ETL processing. The processed result data is directly returned through the API and landed in Hbase.

[0029] The Python program calls the response data source model to read and load the data, segment the text, divide the documents by type, vectorize the documents, and store the vectorized documents in the Milvus vector database.

[0030] Optionally, in the media intelligent decision-making operation method described in the embodiment of the present application, the pre-processed data is input into the AI ​​big model, and the user demand information is output, which specifically includes:

[0031] Establish a training set based on big data, input the training set into the initial model for iterative training, and obtain the training results;

[0032] Determine whether the training results converge;

[0033] If convergence occurs, an AI big model is generated, and preprocessed data is analyzed based on the AI ​​big model to obtain user demand information;

[0034] If it does not converge, feedback information is generated, and model parameters of the initial model are adjusted based on the feedback information until the model converges.

[0035] Optionally, in the media intelligent decision-making operation method described in the embodiment of the present application, if greater than or equal to, then generating conversion information, based on the conversion information, converting the format of the preprocessed data to obtain the converted operation data, specifically includes:

[0036] Preprocessing data text based on interactive page to obtain input;

[0037] Perform segmentation, word segmentation, semantic understanding and word meaning analysis on the input pre-processed data text to obtain the input content;

[0038] Determine whether the input content needs to be converted into SQL;

[0039] If SQL needs to be converted, call the Test-To-Sql component or plug-in to convert SQL into the prompt corresponding to the AI ​​big model, and output the converted operation data;

[0040] If SQL conversion is not required, the preprocessed data text can be directly converted into documents and vectors, and Milvus vector data can be retrieved to output the converted operational data.

[0041] In a second aspect, an embodiment of the present application provides a media intelligent decision-making operation system, the system comprising: a memory and a processor, the memory comprising a program of a media intelligent decision-making operation method, and the program of the media intelligent decision-making operation method is executed by the processor to implement the following steps:

[0042] Acquire operation data, pre-process the operation data, and obtain pre-processed data;

[0043] Input preprocessed data into the AI ​​big model and output user demand information;

[0044] Compare the user demand information with the set condition information to obtain the demand deviation rate;

[0045] Determining whether the demand deviation rate is greater than or equal to a set demand deviation rate threshold;

[0046] If it is greater than or equal to, conversion information is generated, and the format of the pre-processed data is converted based on the conversion information to obtain the converted operation data;

[0047] If it is less than, the preprocessed data will be transmitted to the terminal in real time.

[0048] Optionally, in the media intelligent decision-making operation system described in the embodiment of the present application, obtaining operation data, preprocessing the operation data, and obtaining preprocessed data specifically includes:

[0049] Obtain network collection data, production business data and tracking point log data, and transmit them to offline ETL;

[0050] Based on offline ETL, data features are extracted from network collection data, production business data and tracking point log data to obtain network collection data features, production business data features and tracking point log data features;

[0051] Analyze whether the characteristics of network collection data, production business data and tracking point log data meet the set data standard information;

[0052] If it meets the requirements, an ETL data log is generated to obtain preprocessed data;

[0053] If it does not meet the requirements, the analysis results are obtained, and based on the analysis results, the network collection data, production business data and tracking point log data are cleaned and converted to obtain pre-processed data.

[0054] Optionally, in the media intelligent decision-making operation system described in the embodiment of the present application, data cleaning and data conversion are performed on network collection data, production business data and embedding log data based on the analysis results to obtain pre-processed data, specifically including:

[0055] Obtain analysis results, analyze the consistency of network collection data based on the analysis results to obtain a first similarity, analyze the consistency of production business data based on the analysis results to obtain a second similarity, and analyze the consistency of tracking point log data based on the analysis results to obtain a third similarity;

[0056] Determine whether the first similarity, the second similarity, and the third similarity all reach a set similarity threshold;

[0057] If the similarity threshold is reached, the duplicate data between network collection data, production business data and tracking point log data will be analyzed based on the analysis results, and the duplicate data will be cleaned;

[0058] If one of them does not reach the set similarity threshold, the network collection data, production business data and tracking log data are converted, and the converted network collection data, production business data and tracking log data are cleaned to obtain preprocessed data.

[0059] In a third aspect, an embodiment of the present application further provides a computer-readable storage medium, wherein the computer-readable storage medium includes a media intelligent decision-making operation method program. When the media intelligent decision-making operation method program is executed by a processor, the steps of the media intelligent decision-making operation method as described in any one of the above items are implemented.

[0060] As can be seen from the above, a media intelligent decision-making operation method, system and medium provided in the embodiment of the present application obtains operation data, pre-processes the operation data, and obtains pre-processed data; inputs the pre-processed data into the AI ​​big model, and outputs user demand information; compares the user demand information with the set condition information to obtain the demand deviation rate; determines whether the demand deviation rate is greater than or equal to the set demand deviation rate threshold; if greater than or equal to, generates conversion information, and converts the format of the pre-processed data based on the conversion information to obtain the converted operation data; if less than, transmits the pre-processed data to the terminal in real time; through advanced data processing technology, efficient processing and intelligent analysis of large-scale multimedia data and business data are realized, so that decision makers and operators can obtain the desired data through interactive pages, realize decision-making reports and operation automation, thereby improving the operation efficiency and decision-making quality of provincial media. BRIEF DESCRIPTION OF THE DRAWINGS

[0061] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings required for use in the embodiments of the present application will be briefly introduced below. It should be understood that the following drawings only show certain embodiments of the present application and therefore should not be regarded as limiting the scope. For ordinary technicians in this field, other related drawings can be obtained based on these drawings without paying creative work.

[0062] Figure 1 A flowchart of the media intelligent decision-making operation method provided in an embodiment of the present application;

[0063] Figure 2 A flow chart of obtaining pre-processed data for the media intelligent decision-making operation method provided in an embodiment of the present application;

[0064] Figure 3 A flow chart for cleaning and converting network collected data, production business data, and embedded point log data of the media intelligent decision-making operation method provided in an embodiment of the present application;

[0065] Figure 4 A block diagram of the media intelligent decision-making operation system provided in an embodiment of the present application. DETAILED DESCRIPTION

[0066] The technical solutions in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all of the embodiments. The components of the embodiments of the present application described and shown in the drawings here can be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of the present application provided in the drawings is not intended to limit the scope of the application claimed for protection, but merely represents the selected embodiments of the present application. Based on the embodiments of the present application, all other embodiments obtained by those skilled in the art without making creative work belong to the scope of protection of the present application.

[0067] It should be noted that similar reference numerals and letters represent similar items in the following drawings, so once an item is defined in one drawing, it does not need to be further defined and explained in subsequent drawings. At the same time, in the description of this application, the terms "first", "second", etc. are only used to distinguish the description and cannot be understood as indicating or implying relative importance.

[0068] Please refer to Figure 1 , Figure 1 This is a flow chart of a media intelligent decision-making operation method in some embodiments of the present application. The media intelligent decision-making operation method is used in a terminal device, and the media intelligent decision-making operation method includes the following steps:

[0069] S101, acquiring operation data, and preprocessing the operation data to obtain preprocessed data;

[0070] S102, input the preprocessed data into the AI ​​big model and output user demand information;

[0071] S103, comparing the user demand information with the set condition information to obtain the demand deviation rate;

[0072] S104, determining whether the demand deviation rate is greater than or equal to a set demand deviation rate threshold;

[0073] S105, if it is greater than or equal to, generate conversion information, convert the format of the pre-processed data based on the conversion information to obtain the converted operation data; if it is less than, transmit the pre-processed data to the terminal in real time.

[0074] It should be noted that the AI ​​big model obtains the result data of ETL from the data middle platform through the data middle platform, or directly obtains the buried data, business data, etc. for model training, and generates a local data decision-making and operation knowledge base. Decision makers and operators can obtain the desired data through interactive pages (intelligent Q&A, fixed page display, etc.), and can raise their own needs through questions and answers, automatically convert needs, and verify whether the needs are what they expect. They can directly return the demand results to realize decision-making reports and operation automation.

[0075] Please refer to Figure 2 , Figure 2 This is a flow chart of preprocessing data acquisition in a media intelligent decision-making operation method in some embodiments of the present application. According to an embodiment of the present invention, acquiring operation data, preprocessing the operation data, and obtaining preprocessed data specifically includes:

[0076] S201, obtaining network collection data, production business data and tracking point log data, and transmitting them to offline ETL;

[0077] S202, extracting data features from network collection data, production business data, and embedding log data based on offline ETL to obtain network collection data features, production business data features, and embedding log data features;

[0078] S203, analyzing whether the network collection data characteristics, production business data characteristics, and point log data characteristics meet the set data standard information;

[0079] S204, if it meets the requirements, then generate ETL data log to obtain pre-processed data;

[0080] S205, if it does not meet the requirements, then obtain the analysis results, and perform data cleaning and data conversion on the network collection data, production business data and embedded point log data based on the analysis results to obtain pre-processed data.

[0081] It should be noted that the network collection data, production business data and tracking log data are analyzed according to the set data standard information to see whether they meet the requirements, so as to obtain the analysis results, and the data is accurately cleaned and converted according to different analysis results.

[0082] Please refer to Figure 3 , Figure 3 This is a flow chart of cleaning and converting network data, production business data, and point-of-care log data in a media intelligent decision-making operation method in some embodiments of the present application. According to an embodiment of the present invention, data cleaning and data conversion are performed on network data, production business data, and point-of-care log data based on the analysis results to obtain preprocessed data, specifically including:

[0083] S301, obtaining analysis results, analyzing the consistency of network collection data based on the analysis results to obtain a first similarity, analyzing the consistency of production business data based on the analysis results to obtain a second similarity, and analyzing the consistency of tracking point log data based on the analysis results to obtain a third similarity;

[0084] S302, determining whether the first similarity, the second similarity, and the third similarity all reach a set similarity threshold;

[0085] S303: If the similarity threshold is reached, duplicate data among network collection data, production business data and tracking log data are analyzed based on the analysis result, and the duplicate data is cleaned;

[0086] S304: If one of them does not reach the set similarity threshold, the network collection data, production business data and tracking log data are converted, and the converted network collection data, production business data and tracking log data are cleaned to obtain preprocessed data.

[0087] It should be noted that the data is cleaned by analyzing the consistency of network collection data, production business data and tracking point log data, and the cleaned data is converted to improve the data processing effect.

[0088] According to an embodiment of the present invention, network collection data, production business data and tracking log data are obtained and transmitted to offline ETL, specifically including:

[0089] Synchronize network collected data to Hybase database based on API;

[0090] Synchronize production business data to offline ETL using Sqoop / DataX synchronization tools, and synchronize data to real-time ETL using FlinkCDC;

[0091] Synchronize the tracking log data to SLS based on Logfile or ilogtail tools, deliver the data to offline ETL based on SLS, send the tracking log data to Kafka based on API, and consume Kafka data based on real-time ETL;

[0092] Maxcomputer+dataworks components and a five-layer data warehouse architecture are used for layered ETL processing. The processed result data and embedded point log data are exported to Greenplume, Clickhouse, and Hbase.

[0093] The public cloud version of Kafka+Flink+Hbase components and the five-layer data warehouse architecture are used for layered ETL processing. The processed result data is directly returned through the API and landed in Hbase.

[0094] The Python program calls the response data source model to read and load the data, segment the text, divide the documents by type, vectorize the documents, and store the vectorized documents in the Milvus vector database.

[0095] It should be noted that if the content does not need to be combined with real-time data, the SQL needs to be converted, the vector needs to be converted, and the Milvus vector database needs to be searched, and then the relevant documents are obtained. The matched content is added to the prompt as the context and the question, and the prompt template is filled with context and query (the template here needs to be adjusted according to the different large models called), and the corresponding large model Prompt is obtained, and then it is given to LLM, and the results are displayed. There is no need to convert SQL, directly convert the text to a document, and then to a quantity, and the Milvus vector database needs to be searched, and then the relevant documents are obtained. The matched content is added to the prompt as the context and the question, and the prompt template is filled with context and query (the template here needs to be adjusted according to the different large models called), and the corresponding large model Prompt is obtained, and then it is given to LLM, and the results are displayed.

[0096] According to an embodiment of the present invention, the preprocessed data is input into the AI ​​big model, and the user demand information is output, which specifically includes:

[0097] Establish a training set based on big data, input the training set into the initial model for iterative training, and obtain the training results;

[0098] Determine whether the training results converge;

[0099] If convergence occurs, an AI big model is generated, and preprocessed data is analyzed based on the AI ​​big model to obtain user demand information;

[0100] If it does not converge, feedback information is generated, and model parameters of the initial model are adjusted based on the feedback information until the model converges.

[0101] It should be noted that by training the initial model through big data, the model parameters are continuously adjusted to obtain an accurate AI big model and improve the accuracy of the output results of the AI ​​big model.

[0102] According to an embodiment of the present invention, if it is greater than or equal to, conversion information is generated, and the pre-processed data is format-converted based on the conversion information to obtain converted operation data, which specifically includes:

[0103] Preprocessing data text based on interactive page to obtain input;

[0104] Perform segmentation, word segmentation, semantic understanding and word meaning analysis on the input pre-processed data text to obtain the input content;

[0105] Determine whether the input content needs to be converted into SQL;

[0106] If SQL needs to be converted, call the Test-To-Sql component or plug-in to convert SQL into the prompt corresponding to the AI ​​big model, and output the converted operation data;

[0107] If SQL conversion is not required, the preprocessed data text can be directly converted into documents and vectors, and Milvus vector data can be retrieved to output the converted operational data.

[0108] It should be noted that by converting information, decision makers and operators can raise temporary demands through a visual interactive page, convert them into actual SQL execution, and return data, thereby improving the efficiency of decision makers and operators. In particular, it allows operators to track activities in real time and customize the analysis of operational activities from multiple dimensions at any time, so that operational activity strategies can be adjusted more timely.

[0109] Please refer to Figure 4 , Figure 4 The block diagram of a media intelligent decision-making operation system in some embodiments of the present application. In a second aspect, the embodiments of the present application provide a media intelligent decision-making operation system, the system comprising: a memory and a processor, the memory including a program of a media intelligent decision-making operation method, and when the program of the media intelligent decision-making operation method is executed by the processor, the following steps are implemented:

[0110] Acquire operation data, pre-process the operation data, and obtain pre-processed data;

[0111] Input preprocessed data into the AI ​​big model and output user demand information;

[0112] Compare the user demand information with the set condition information to obtain the demand deviation rate;

[0113] Determine whether the demand deviation rate is greater than or equal to a set demand deviation rate threshold;

[0114] If it is greater than or equal to, conversion information is generated, and the format of the pre-processed data is converted based on the conversion information to obtain the converted operation data;

[0115] If it is less than, the preprocessed data will be transmitted to the terminal in real time.

[0116] It should be noted that the media intelligent decision-making and operation system can reduce the number of rounds of demand review by decision makers, operators and related personnel, thereby improving efficiency. Decision makers and operators can verify the needs through the system in advance, or directly obtain the results of the needs.

[0117] According to an embodiment of the present invention, obtaining operation data and preprocessing the operation data to obtain preprocessed data specifically includes:

[0118] Obtain network collection data, production business data and tracking point log data, and transmit them to offline ETL;

[0119] Based on offline ETL, data features are extracted from network collection data, production business data and tracking point log data to obtain network collection data features, production business data features and tracking point log data features;

[0120] Analyze whether the characteristics of network collection data, production business data and tracking point log data meet the set data standard information;

[0121] If it meets the requirements, an ETL data log is generated to obtain preprocessed data;

[0122] If it does not meet the requirements, the analysis results are obtained, and based on the analysis results, the network collection data, production business data and tracking point log data are cleaned and converted to obtain pre-processed data.

[0123] It should be noted that the network collection data, production business data and tracking log data are analyzed according to the set data standard information to see whether they meet the requirements, so as to obtain the analysis results, and the data is accurately cleaned and converted according to different analysis results.

[0124] According to an embodiment of the present invention, data cleaning and data conversion are performed on network collection data, production business data and tracking log data based on the analysis results to obtain preprocessed data, which specifically includes:

[0125] Obtain analysis results, analyze the consistency of network collection data based on the analysis results to obtain a first similarity, analyze the consistency of production business data based on the analysis results to obtain a second similarity, and analyze the consistency of tracking point log data based on the analysis results to obtain a third similarity;

[0126] Determine whether the first similarity, the second similarity, and the third similarity all reach a set similarity threshold;

[0127] If the similarity threshold is reached, the duplicate data between network collection data, production business data and tracking point log data will be analyzed based on the analysis results, and the duplicate data will be cleaned;

[0128] If one of them does not reach the set similarity threshold, the network collection data, production business data and tracking log data are converted, and the converted network collection data, production business data and tracking log data are cleaned to obtain preprocessed data.

[0129] It should be noted that the data is cleaned by analyzing the consistency of network collection data, production business data and tracking point log data, and the cleaned data is converted to improve the data processing effect.

[0130] According to an embodiment of the present invention, network collection data, production business data and tracking log data are obtained and transmitted to offline ETL, specifically including:

[0131] Synchronize network collected data to Hybase database based on API;

[0132] Synchronize production business data to offline ETL using Sqoop / DataX synchronization tools, and synchronize data to real-time ETL using FlinkCDC;

[0133] Synchronize the tracking log data to SLS based on Logfile or ilogtail tools, deliver the data to offline ETL based on SLS, send the tracking log data to Kafka based on API, and consume Kafka data based on real-time ETL;

[0134] Maxcomputer+dataworks components and a five-layer data warehouse architecture are used for layered ETL processing. The processed result data and embedded point log data are exported to Greenplume, Clickhouse, and Hbase.

[0135] The public cloud version of Kafka+Flink+Hbase components and the five-layer data warehouse architecture are used for layered ETL processing. The processed result data is directly returned through the API and landed in Hbase.

[0136] The Python program calls the response data source model to read and load the data, segment the text, divide the documents by type, vectorize the documents, and store the vectorized documents in the Milvus vector database.

[0137] It should be noted that if the content does not need to be combined with real-time data, the SQL needs to be converted, the vector needs to be converted, and the Milvus vector database needs to be searched, and then the relevant documents are obtained. The matched content is added to the prompt as the context and the question, and the prompt template is filled with context and query (the template here needs to be adjusted according to the different large models called), and the corresponding large model Prompt is obtained, and then it is given to LLM, and the results are displayed. There is no need to convert SQL, directly convert the text to a document, and then to a quantity, and the Milvus vector database needs to be searched, and then the relevant documents are obtained. The matched content is added to the prompt as the context and the question, and the prompt template is filled with context and query (the template here needs to be adjusted according to the different large models called), and the corresponding large model Prompt is obtained, and then it is given to LLM, and the results are displayed.

[0138] According to an embodiment of the present invention, the preprocessed data is input into the AI ​​big model, and the user demand information is output, which specifically includes:

[0139] Establish a training set based on big data, input the training set into the initial model for iterative training, and obtain the training results;

[0140] Determine whether the training results converge;

[0141] If convergence occurs, an AI big model is generated, and preprocessed data is analyzed based on the AI ​​big model to obtain user demand information;

[0142] If it does not converge, feedback information is generated, and model parameters of the initial model are adjusted based on the feedback information until the model converges.

[0143] It should be noted that by training the initial model through big data, the model parameters are continuously adjusted to obtain an accurate AI big model and improve the accuracy of the output results of the AI ​​big model.

[0144] According to an embodiment of the present invention, if it is greater than or equal to, conversion information is generated, and the pre-processed data is format-converted based on the conversion information to obtain converted operation data, which specifically includes:

[0145] Preprocessing data text based on interactive page to obtain input;

[0146] Perform segmentation, word segmentation, semantic understanding and word meaning analysis on the input pre-processed data text to obtain the input content;

[0147] Determine whether the input content needs to be converted into SQL;

[0148] If SQL needs to be converted, call the Test-To-Sql component or plug-in to convert SQL into the prompt corresponding to the AI ​​big model, and output the converted operation data;

[0149] If SQL conversion is not required, the preprocessed data text can be directly converted into documents and vectors, and Milvus vector data can be retrieved to output the converted operational data.

[0150] It should be noted that by converting information, decision makers and operators can raise temporary demands through a visual interactive page, convert them into actual SQL execution, and return data, thereby improving the efficiency of decision makers and operators. In particular, it allows operators to track activities in real time and customize the analysis of operational activities from multiple dimensions at any time, so that operational activity strategies can be adjusted more timely.

[0151] A third aspect of the present invention provides a computer-readable storage medium, which includes a media intelligent decision-making operation method program. When the media intelligent decision-making operation method program is executed by a processor, the steps of the media intelligent decision-making operation method as described above are implemented.

[0152] The present invention discloses a media intelligent decision-making operation method, system and medium, which obtains operation data, pre-processes the operation data to obtain pre-processed data; inputs the pre-processed data into an AI big model, and outputs user demand information; compares the user demand information with set condition information to obtain a demand deviation rate; determines whether the demand deviation rate is greater than or equal to a set demand deviation rate threshold; if greater than or equal to, generates conversion information, and converts the format of the pre-processed data based on the conversion information to obtain converted operation data; if less than, transmits the pre-processed data to a terminal in real time; through advanced data processing technology, efficient processing and intelligent analysis of large-scale multimedia data and business data are achieved, so that decision makers and operators can obtain desired data through interactive pages, realize decision reporting and operation automation, and thus improve the operation efficiency and decision quality of provincial media.

[0153] In the several embodiments provided in the present application, it should be understood that the disclosed devices and methods can be implemented in other ways. The device embodiments described above are only schematic. For example, the division of units is only a logical function division. There may be other division methods in actual implementation, such as: multiple units or components can be combined, or can be integrated into another system, or some features can be ignored or not executed. In addition, the coupling, direct coupling, or communication connection between the components shown or discussed can be through some interfaces, and the indirect coupling or communication connection of devices or units can be electrical, mechanical or other forms.

[0154] The units described above as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units; they may be located in one place or distributed on multiple network units; some or all of the units may be selected according to actual needs to achieve the purpose of the present embodiment.

[0155] In addition, all functional units in the embodiments of the present invention may be integrated into one processing unit, or each unit may be separately used as a unit, or two or more units may be integrated into one unit; the above-mentioned integrated units may be implemented in the form of hardware or in the form of hardware plus software functional units.

[0156] Those skilled in the art can understand that: all or part of the steps of implementing the above method embodiments can be completed by hardware related to program instructions, the aforementioned program can be stored in a readable storage medium, and when the program is executed, it executes the steps of the above method embodiments; and the aforementioned storage medium includes: mobile storage devices, read-only memories (ROM, Read-Only Memory), random access memories (RAM, Random Access Memory), disks or optical disks, and other media that can store program codes.

[0157] Or, if the above-mentioned integrated unit of the present invention is implemented in the form of a software function module and sold or used as an independent product, it can also be stored in a readable storage medium. Based on this understanding, the technical solution of the embodiment of the present invention can be essentially or partly reflected in the form of a software product that contributes to the prior art. The software product is stored in a storage medium and includes several instructions for a computer device (which can be a personal computer, server, or network device, etc.) to execute all or part of the methods described in each embodiment of the present invention. The aforementioned storage medium includes: various media that can store program codes, such as mobile storage devices, ROM, RAM, magnetic disks or optical disks.

Claims

1. A media intelligent decision-making operation method, characterized in that: include: Acquire operation data, pre-process the operation data, and obtain pre-processed data; Input preprocessed data into the AI ​​big model and output user demand information; Compare the user demand information with the set condition information to obtain the demand deviation rate; Determining whether the demand deviation rate is greater than or equal to a set demand deviation rate threshold; If it is greater than or equal to, conversion information is generated, and the format of the pre-processed data is converted based on the conversion information to obtain the converted operation data; If it is less than, the preprocessed data will be transmitted to the terminal in real time.

2. The media intelligent decision-making operation method according to claim 1, characterized in that: Obtain operational data, and preprocess the operational data to obtain preprocessed data, including: Obtain network collection data, production business data and tracking point log data, and transmit them to offline ETL; Based on offline ETL, data features are extracted from network collection data, production business data and tracking point log data to obtain network collection data features, production business data features and tracking point log data features; Analyze whether the characteristics of network collection data, production business data and tracking point log data meet the set data standard information; If it meets the requirements, an ETL data log is generated to obtain preprocessed data; If it does not meet the requirements, the analysis results are obtained, and based on the analysis results, the network collection data, production business data and tracking point log data are cleaned and converted to obtain pre-processed data.

3. The media intelligent decision-making operation method according to claim 2 is characterized in that: Based on the analysis results, the network collection data, production business data and tracking point log data are cleaned and converted to obtain pre-processed data, including: Obtain analysis results, analyze the consistency of network collection data based on the analysis results to obtain a first similarity, analyze the consistency of production business data based on the analysis results to obtain a second similarity, and analyze the consistency of tracking point log data based on the analysis results to obtain a third similarity; Determine whether the first similarity, the second similarity, and the third similarity all reach a set similarity threshold; If the similarity threshold is reached, the duplicate data between network collection data, production business data and tracking point log data will be analyzed based on the analysis results, and the duplicate data will be cleaned; If one of them does not reach the set similarity threshold, the network collection data, production business data and tracking log data are converted, and the converted network collection data, production business data and tracking log data are cleaned to obtain preprocessed data.

4. The media intelligent decision-making operation method according to claim 3 is characterized in that: Obtain network collection data, production business data, and tracking log data, and transmit them to offline ETL, including: Synchronize network collected data to Hybase database based on API; Synchronize production business data to offline ETL using Sqoop / DataX synchronization tools, and synchronize data to real-time ETL using FlinkCDC; Synchronize the tracking log data to SLS based on Logfile or ilogtail tools, deliver the data to offline ETL based on SLS, send the tracking log data to Kafka based on API, and consume Kafka data based on real-time ETL; Maxcomputer+dataworks components and a five-layer data warehouse architecture are used for layered ETL processing. The processed result data and embedded point log data are exported to Greenplume, Clickhouse, and Hbase. The public cloud version of Kafka+Flink+Hbase components and the five-layer data warehouse architecture are used for layered ETL processing. The processed result data is directly returned through the API and landed in Hbase. The Python program calls the response data source model to read and load the data, segment the text, divide the documents by type, vectorize the documents, and store the vectorized documents in the Milvus vector database.

5. The media intelligent decision-making operation method according to claim 4 is characterized in that: Input the preprocessed data into the AI ​​big model and output the user demand information, including: Establish a training set based on big data, input the training set into the initial model for iterative training, and obtain the training results; Determine whether the training results converge; If convergence occurs, an AI big model is generated, and preprocessed data is analyzed based on the AI ​​big model to obtain user demand information; If it does not converge, feedback information is generated, and model parameters of the initial model are adjusted based on the feedback information until the model converges.

6. The media intelligent decision-making operation method according to claim 4 is characterized in that: If it is greater than or equal to, then conversion information is generated, and the pre-processed data is format-converted based on the conversion information to obtain the converted operational data, which specifically includes: Preprocessing data text based on interactive page to obtain input; Perform segmentation, word segmentation, semantic understanding and word meaning analysis on the input pre-processed data text to obtain the input content; Determine whether the input content needs to be converted into SQL; If SQL needs to be converted, call the Test-To-Sql component or plug-in to convert SQL into the prompt corresponding to the AI ​​big model, and output the converted operation data; If SQL conversion is not required, the preprocessed data text can be directly converted into documents and vectors, and Milvus vector data can be retrieved to output the converted operational data.

7. A media intelligent decision-making operation system, characterized in that: The system includes: a memory and a processor, wherein the memory includes a program of a media intelligent decision-making operation method, and when the program of the media intelligent decision-making operation method is executed by the processor, the following steps are implemented: Acquire operation data, pre-process the operation data, and obtain pre-processed data; Input preprocessed data into the AI ​​big model and output user demand information; Compare the user demand information with the set condition information to obtain the demand deviation rate; Determining whether the demand deviation rate is greater than or equal to a set demand deviation rate threshold; If it is greater than or equal to, conversion information is generated, and the format of the pre-processed data is converted based on the conversion information to obtain the converted operation data; If it is less than, the preprocessed data will be transmitted to the terminal in real time.

8. The media intelligent decision-making operation system according to claim 7, characterized in that: Obtain operational data, and preprocess the operational data to obtain preprocessed data, including: Obtain network collection data, production business data and tracking point log data, and transmit them to offline ETL; Based on offline ETL, data features are extracted from network collection data, production business data and tracking point log data to obtain network collection data features, production business data features and tracking point log data features; Analyze whether the characteristics of network collection data, production business data and tracking point log data meet the set data standard information; If it meets the requirements, an ETL data log is generated to obtain preprocessed data; If it does not meet the requirements, the analysis results are obtained, and based on the analysis results, the network collection data, production business data and tracking point log data are cleaned and converted to obtain pre-processed data.

9. The media intelligent decision-making operation system according to claim 8, characterized in that: Based on the analysis results, the network collection data, production business data and tracking point log data are cleaned and converted to obtain pre-processed data, including: Obtain analysis results, analyze the consistency of network collection data based on the analysis results to obtain a first similarity, analyze the consistency of production business data based on the analysis results to obtain a second similarity, and analyze the consistency of tracking point log data based on the analysis results to obtain a third similarity; Determine whether the first similarity, the second similarity, and the third similarity all reach a set similarity threshold; If the similarity threshold is reached, the duplicate data between network collection data, production business data and tracking point log data will be analyzed based on the analysis results, and the duplicate data will be cleaned; If one of them does not reach the set similarity threshold, the network collection data, production business data and tracking log data are converted, and the converted network collection data, production business data and tracking log data are cleaned to obtain preprocessed data.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium includes a media intelligent decision-making operation method program, and when the media intelligent decision-making operation method program is executed by a processor, the steps of the media intelligent decision-making operation method as described in any one of claims 1 to 6 are implemented.

Images