Information acquisition method based on big data and artificial intelligence
By selecting, preprocessing and judging information sources in big data and artificial intelligence information collection methods, the problems of information collection permissions, abandonment and reliability are solved, the accuracy and comprehensiveness of information collection are achieved, and the processing efficiency is improved .
Patent Information
- Application Number
- CN202411113783.4
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-08-14
- Publication Date
- 2024-11-01
- Estimated Expiration
- Not applicable · inactive patent
AI Technical Summary
The existing big data and artificial intelligence information collection methods are collected directly after the target is selected, which is prone to encounter information source permission problems, abandoned information sources and unreliable information sources, resulting in inaccurate and incomplete information collection, affecting processing efficiency.
A method of information collection based on big data and artificial intelligence was designed, including the steps of information source selection, preprocessing, collection, processing, analysis, integration and storage, through permission judgment, abandonment judgment and reliability before the information source selection Judgment, ensure the effectiveness and reliability of information sources, and avoid confusion and unexpected phenomena in the information collection process.
It improves the accuracy and comprehensiveness of information collection, reduces the difficulty of collection and the probability of handling accidents, and improves the processing efficiency of big data and artificial intelligence information collection.
Smart Images

Figure CN118885474A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the technical field of information collection, and in particular to an information collection method based on big data and artificial intelligence. Background Art
[0002] In the existing technology, the common information collection methods in life are usually manual collection or big data and artificial intelligence collection. In the common big data and artificial intelligence collection methods, in the process of collecting information, most of them select one or more information sources as the target of information collection, and directly collect information after the target selection is completed. In this way, it is very easy to encounter information sources that require permission to collect information during information collection, and directly collect information from the selected information sources, which is very easy to encounter abandoned information sources and unreliable information sources, resulting in difficulties in information collection, and the collected information is prone to inaccuracy and incompleteness, resulting in unexpected phenomena during the processing of the collected information, which seriously affects the processing efficiency of big data and artificial intelligence information collection. Summary of the invention
[0003] In view of the above-mentioned shortcomings of the prior art, most common big data and artificial intelligence collect information directly after target selection, which can easily cause difficulties in information collection and the collected information is inaccurate and incomplete, which can easily affect the processing efficiency after information collection. The first purpose of the present invention is to provide an information collection method based on big data and artificial intelligence to solve the problems in the above-mentioned background technology.
[0004] To achieve the above object, the present invention provides the following technical solutions:
[0005] An information collection method based on big data and artificial intelligence comprises the following steps:
[0006] S1. Information source selection, which is the step of selecting targets through information collection using big data and artificial intelligence;
[0007] S2, information source preprocessing, the information source preprocessing is a step of preprocessing the selected target before information collection;
[0008] S3, information collection, the information collection is a step of collecting information on the target after completing preprocessing;
[0009] S4, information processing, which is a step of processing the information after it is collected;
[0010] S5, information analysis, wherein the information analysis is a step of analyzing the information after the processing is completed;
[0011] S6, information integration, wherein the information integration is a step of integrating the information after the analysis is completed;
[0012] S7: Information storage, which is a step of storing the information after integration.
[0013] The present invention is further configured as follows: in the step S1, target selection is performed according to the direction required for information collection, and a corresponding target is selected from multiple targets. After the target selection is completed, the target is used as an information source for information collection.
[0014] By adopting the above technical solution: by designing information source selection, information sources are selected as information collection targets before information collection, so as to avoid collecting multiple information sources at the same time during the information collection process, which may cause confusion in the collected information.
[0015] The present invention is further configured as follows: in the step S2, after the information source is determined, the selected information source is preprocessed, and the selected information source is tried for the first information collection to confirm whether the selected information source can collect information normally. If the information collection is carried out normally, the latest information release time of the information source is obtained after the initial information collection is completed, and the selected information source is judged by using the latest information release time of the obtained information source to judge whether the selected information source is discarded. If the latest information release time of the information source is relatively recent, the reliability of the selected information source is relatively large, and the information release interval of the information source is further obtained. If the information release interval is relatively short, the selected information source is confirmed as the target of information collection;
[0016] S2.1. Authority judgment: the authority judgment is a step to judge whether the selected information source has the authority to collect information. The selected information source is first collected. If the information source requires authority to collect information normally, it is difficult to collect information from the information source, and a new information source is selected;
[0017] S2.2, abandonment judgment, the abandonment judgment is a step of judging whether the selected information source is abandoned, obtaining the time when the latest information of the information source is released, and judging the information source according to the latest information release time of the selected information source. If the latest information release time is far away, the information source is regarded as abandoned, and a new information source is selected;
[0018] S2.3, reliability judgment, the reliability judgment is a step to judge whether the selected information source is reliable, obtain the interval time of information release of the information source, and judge the information source in combination with the time of the latest information release of the information source. If the latest information is released recently, but the interval time of information release is large, the reliability of the information source is poor, and a new information source is selected;
[0019] S2.4, information source confirmation. The information source confirmation is a step to confirm the selected information source as an information collection target. After the selected information source completes the authority judgment, abandonment judgment and reliability judgment in sequence, when the information source meets the requirements, the selected information source is confirmed as an information collection target.
[0020] By adopting the above technical solution: by designing information source preprocessing, various judgments are made after the information source selection is completed to ensure that the information in the selected information source can be collected normally, avoid unexpected phenomena during the information processing process after information collection, reduce the probability of unexpected phenomena in the subsequent information processing, and improve the efficiency of subsequent information processing.
[0021] The present invention is further configured as follows: in the step S3, after the selected information source is confirmed as an information collection target, information collection is started on the information source to obtain various information in the information source.
[0022] By adopting the above technical solution: by designing information collection, after the information source is confirmed, the information in the information source is collected so that the information can be collected normally.
[0023] The present invention is further configured as follows: in step S4, after the information is acquired, the information is processed, the packaged information is unpacked, and the approximate content of various information is obtained based on the collected information, and the duplicate and known invalid information is deleted.
[0024] By adopting the above technical solution: by designing information processing, the collected information is unpacked, and duplicate and known invalid information is deleted according to the approximate content of the information, so as to avoid the mixing of duplicate and known invalid information, which may affect the accuracy of the information.
[0025] The present invention is further configured as follows: in the step S5, after the collected information is processed, the processed information is analyzed and mined to obtain the data present in the information.
[0026] By adopting the above technical solution: by designing information analysis, the processed information is analyzed and mined, so that the data in the information can be normally obtained, and the efficiency of information processing is improved.
[0027] The present invention is further configured as follows: in the step S6, various collected information is integrated based on the data obtained after the information is analyzed, and information of the same category is integrated together.
[0028] By adopting the above technical solution: by designing information integration, information of the same category is integrated together, so that the information has good integration measures after processing and analysis, avoiding information confusion.
[0029] The present invention is further configured as follows: in step S7, the information that has completed the integration process is stored in an information database.
[0030] By adopting the above technical solution: by designing information storage, the integrated information is stored, thereby reducing the probability of loss of the collected information after processing.
[0031] Beneficial Effects
[0032] Compared with the known public technology, the technical solution provided by the present invention has the following beneficial effects:
[0033] In the present invention, by designing information source preprocessing, after the information source selection is completed, the selected information source is sequentially subjected to authority judgment, abandonment judgment and reliability judgment, so as to avoid information collection directly from the information source with authority, thereby reducing the difficulty of information collection, and through the information source abandonment judgment, it is ensured that the selected information source is not abandoned, and the information collected is valid, thereby reducing the probability of invalid information collection and improving the accuracy of information collection, and through the information source reliability judgment, the information release time and interval time of the selected information source are obtained to judge whether the selected information source is reliable, and the reliability judgment measures of the selected information source are improved, thereby avoiding information collection from unreliable information sources, resulting in missing collected information, improving the comprehensiveness of information collection, allowing the collected valid information to be processed normally, and improving the efficiency of big data and artificial intelligence information collection and processing. BRIEF DESCRIPTION OF THE DRAWINGS
[0034] Figure 1 The present invention is a flowchart of an information collection method based on big data and artificial intelligence. DETAILED DESCRIPTION
[0035] In order to make the purpose, technical solution and advantages of the embodiments of the present invention clearer, the technical solution in the embodiments of the present invention will be clearly and completely described below. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.
[0036] The present invention will be further described below in conjunction with the embodiments.
[0037] Example 1
[0038] like Figure 1 As shown, the present invention provides a technical solution: an information collection method based on big data and artificial intelligence, comprising the following steps:
[0039] Information source selection, which is the step of selecting targets for information collection using big data and artificial intelligence. Target selection is performed according to the direction required for information collection, and the corresponding target is selected from multiple targets. After the target selection is completed, the target is used as the information source for information collection. By designing information source selection, information sources are selected as information collection targets before information collection, so as to avoid collecting multiple information sources at the same time during the information collection process, which may cause confusion in the collected information;
[0040] Information source preprocessing, the information source preprocessing is a step of preprocessing the selected target before information collection. After the information source is determined, the preprocessing of the selected information source begins. The initial information collection attempt is made on the selected information source to confirm whether the selected information source can collect information normally. If the information collection is carried out normally, the latest information release time of the information source is obtained after the initial information collection is completed. The selected information source is judged by using the latest information release time of the obtained information source to determine whether the selected information source is abandoned. If the latest information release time of the information source is relatively recent, the reliability of the selected information source is relatively high. The information release interval of the information source is further obtained. If the information release interval is relatively short, the selected information source is confirmed as the target of information collection; authority judgment, the authority judgment is a step of judging whether the selected information source has information collection authority. The initial information collection is carried out on the selected information source. If the information source requires authority to collect information normally, the information collection of the information source is difficult, and the information source is reselected; abandonment judgment, the abandonment judgment is a judgment on whether the selected information source is abandoned The step of obtaining the time of the latest information release of the information source, judging the information source according to the latest information release time of the selected information source, if the latest information release time is far away, the information source is regarded as abandoned, and the information source is reselected; reliable judgment, the reliable judgment is a step of judging whether the selected information source is reliable, obtaining the interval time of information release of the information source, and judging the information source in combination with the latest information release time of the information source, if the latest information release time is close, but the information release interval is large, the reliability of the information source is poor, and the information source is reselected; information source confirmation, the information source confirmation is a step of confirming the selected information source as an information collection target, after the selected information source completes the authority judgment, the abandonment judgment and the reliability judgment in turn, when the information source meets the requirements, the selected information source is confirmed as an information collection target, by designing information source preprocessing, after the information source selection is completed, various judgments are performed to ensure that the information in the selected information source can be collected normally, avoid unexpected phenomena in the information source during the information processing after information collection, reduce the probability of unexpected information subsequent processing, and improve the efficiency of information subsequent processing;
[0041] Information collection is a step of collecting information on the target after preprocessing is completed. After the selected information source is confirmed as the information collection target, information collection is started on the information source to obtain various information in the information source. By designing information collection, after the information source is confirmed, information in the information source is collected so that the information can be collected normally;
[0042] Information processing, which is a step of processing information after it is collected. After the information is collected, the information is processed, the packaged information is unpacked, and the approximate content of various information is obtained based on the collected information, and duplicate and known invalid information is deleted. By designing information processing, the collected information is unpacked, and duplicate and known invalid information is deleted based on the approximate content of the information, so as to avoid mixing duplicate and known invalid information into the information, which may affect the accuracy of the information;
[0043] Information analysis is a step of analyzing information after it has been processed. After the collected information has been processed, the processed information is analyzed and mined to obtain the data in the information. By designing information analysis, the processed information is analyzed and mined so that the data in the information can be obtained normally, thereby improving the efficiency of information processing.
[0044] Information integration is the step of integrating information after analysis. According to the data obtained after the information is analyzed, various collected information is integrated and processed, and information of the same category is integrated together. By designing information integration, information of the same category is integrated together, so that the information has good integration measures after processing and analysis, avoiding information confusion;
[0045] Information storage, the information storage is the step of storing the information after the integration is completed, storing the information that has completed the integration processing in the information database, and storing the integrated information through the design of information storage, thereby reducing the probability of loss of the collected information after processing.
[0046] The working principle of this embodiment:
[0047] During the collection, target selection is performed according to the direction required for information collection, and the corresponding target is selected from multiple targets. After the target selection is completed, the target is used as the information source for information collection, and the selected information source is preprocessed. The initial information collection attempt is performed on the selected information source to confirm whether the selected information source can collect information normally. If the information source requires permission to collect information normally, then information collection from this information source is difficult, and the information source is reselected. If the information collection is carried out normally, the latest information release time of the information source is obtained after the initial information collection is completed. The selected information source is judged using the latest information release time of the obtained information source to determine whether the selected information source is obsolete. If the latest information release time is far away, the information source is regarded as obsolete, and the information source is reselected. If the latest information release time of the information source is recent, then the reliability of the selected information source is relatively high, and further information is collected. The information release interval of the source is obtained, and the information source is judged in combination with the time when the latest information is released by the information source. If the latest information is released recently, but the information release interval is large, the reliability of the information source is poor, and the information source is re-selected. If the information release interval is short, the selected information source is confirmed as the target of information collection, and information collection is started on the information source to obtain various information in the information source, process the information, unpack the packaged information, and obtain the approximate content of various information based on the collected information, delete the duplicate and known invalid information, and start analyzing the processed information, analyze and mine the information, obtain the data existing in the information, integrate various collected information according to the data obtained after the information is analyzed, integrate the information of the same category together, store the information that has completed the integration processing, and store it in the information database.
[0048] Comparative Example 1
[0049] The method of this embodiment is substantially the same as that of the embodiment 1 provided, and the main difference is that: in the step S2, the authority determination process is not performed on the information source;
[0050] Comparative Example 2
[0051] The method of this embodiment is substantially the same as that of the embodiment 1 provided, with the main difference being that in step S2, no discard determination process is performed on the information source.
[0052] Comparative Example 3
[0053] The method of this embodiment is substantially the same as that of the embodiment 1 provided, with the main difference being that in step S2, no reliable judgment process is performed on the information source.
[0054] Performance Testing
[0055] The difficulty, accuracy and comprehensiveness of information collection provided by Example 1 and Comparative Examples 1 to 3 are as follows:
[0056] Difficulty accuracy Comprehensiveness Example 1 93.9% 19.9% 25.9% Comparative Example 1 80% 39.9% 45% Comparative Example 2 63% 50% 72% Comparative Example 3 27.9% 86.9% 93%
[0057] By analyzing the relevant data in the above tables, it can be known that through the present invention, information source preprocessing is designed. After the information source selection is completed, the selected information sources are sequentially subjected to authority judgment, abandonment judgment and reliability judgment, so as to avoid information collection directly in the information source with authority, thereby reducing the difficulty of information collection. Through the information source abandonment judgment, it is ensured that the selected information source is not abandoned, and the information collected is valid, thereby reducing the probability of invalid information collection and improving the accuracy of information collection. Through the information source reliability judgment, the information release time and interval time of the selected information source are obtained to judge whether the selected information source is reliable, and the reliability judgment measures of the selected information source are improved, thereby avoiding information collection in unreliable information sources, resulting in missing collected information, improving the comprehensiveness of information collection, allowing the collected valid information to be processed normally, and improving the efficiency of big data and artificial intelligence information collection and processing.
[0058] The above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit the same. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that the technical solutions described in the aforementioned embodiments may still be modified, or some of the technical features may be replaced by equivalents. Such modifications or replacements will not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.
Claims
1. An information collection method based on big data and artificial intelligence, It is characterized in that The following steps are involved: S1. Information source selection, which is the step of selecting targets through information collection using big data and artificial intelligence; S2, information source preprocessing, the information source preprocessing is a step of preprocessing the selected target before information collection; S3, information collection, the information collection is a step of collecting information on the target after completing preprocessing; S4, information processing, which is a step of processing the information after it is collected; S5, information analysis, wherein the information analysis is a step of analyzing the information after the processing is completed; S6, information integration, wherein the information integration is a step of integrating the information after the analysis is completed; S7: Information storage, which is a step of storing the information after integration.
2. According to claim 1, an information collection method based on big data and artificial intelligence, Features: In the step S1, target selection is performed according to the direction required for information collection, and a corresponding target is selected from multiple targets. After the target selection is completed, the target is used as an information source for information collection.
3. According to claim 1, the information collection method based on big data and artificial intelligence, Features: In step S2, after the information source is determined, the selected information source is preprocessed, and the selected information source is tried for the first information collection to confirm whether the selected information source can collect information normally. If the information collection is normal, the latest information release time of the information source is obtained after the initial information collection is completed, and the selected information source is judged by using the latest information release time of the obtained information source to determine whether the selected information source is discarded. If the latest information release time of the information source is relatively recent, the reliability of the selected information source is relatively high, and the information release interval of the information source is further obtained. If the information release interval is relatively short, the selected information source is confirmed as the target of information collection; S2.
1. Authority judgment: the authority judgment is a step to judge whether the selected information source has the authority to collect information. The selected information source is first collected. If the information source requires authority to collect information normally, it is difficult to collect information from the information source, and a new information source is selected; S2.2, abandonment judgment, the abandonment judgment is a step of judging whether the selected information source is abandoned, obtaining the time when the latest information of the information source is released, and judging the information source according to the latest information release time of the selected information source. If the latest information release time is far away, the information source is regarded as abandoned, and a new information source is selected; S2.3, reliability judgment, the reliability judgment is a step to judge whether the selected information source is reliable, obtain the interval time of information release of the information source, and judge the information source in combination with the time of the latest information release of the information source. If the latest information is released recently, but the interval time of information release is large, the reliability of the information source is poor, and a new information source is selected; S2.4, information source confirmation. The information source confirmation is a step to confirm the selected information source as an information collection target. After the selected information source completes the authority judgment, abandonment judgment and reliability judgment in sequence, when the information source meets the requirements, the selected information source is confirmed as an information collection target.
4. According to claim 1, the information collection method based on big data and artificial intelligence, Features: In the step S3, after the selected information source is confirmed as the information collection target, information collection is started on the information source to obtain various information in the information source.
5. According to claim 1, the information collection method based on big data and artificial intelligence, Features: In step S4, after the information is acquired, the information is processed, the packaged information is unpacked, and the approximate content of various information is obtained based on the collected information, and the duplicated and known invalid information is deleted.
6. According to claim 1, the information collection method based on big data and artificial intelligence, It is characterized in that In step S5, after the collected information is processed, the processed information is analyzed and mined to obtain the data in the information.
7. The information collection method based on big data and artificial intelligence according to claim 1, It is characterized in that In the step S6, various collected information is integrated based on the data obtained after the information is analyzed, and information of the same category is integrated together.
8. According to claim 1, the information collection method based on big data and artificial intelligence, It is characterized in that In step S7, the information that has completed the integration process is stored in an information database.
