A big data analysis method with a screening function

Abstract

The present invention discloses a big data analysis method with a screening function, including the following steps. S1: Data acquisition stage: The acquisition module obtains relevant information of customers through the customer information library and the mobile terminal module, and the obtained information will be collected and integrated into the target database through the integration module; S2: Data screening stage: The big data server accesses the Internet big data, and then the analysis module and the screening module conduct preliminary screening on the relevant data; S21: The data that does not meet the screening requirements will be automatically deleted by the deletion module; S22: The data that meets the requirements will be displayed through the first display module and stored in the data area module. For this big data analysis method with a screening function, the acquisition module can obtain customer data from multiple sources through the customer information library and the mobile terminal module. At the same time, the integration module can centrally integrate the obtained data, which is convenient for enriching the comparison samples and ensuring the comprehensiveness of resource acquisition.

Description

Technical Field

[0001] The present invention relates to the technical field of big data analysis, and specifically to a big data analysis method with a screening function. Background Art

[0002] Big data analysis refers to analyzing a large amount of collected data using appropriate statistical analysis methods, summarizing, understanding, and digesting them to maximize the development of the data's functions and play the role of the data. Data analysis is a process of detailed research and summary of data to extract useful information and form conclusions. After retrieval, it is found that a typical big data analysis method in the prior art is, for example, a big data analysis method and device with the publication number CN114092224A, which is applied to the field of big data technology. The method includes: receiving data to be analyzed, resampling the number of sub-samples, calculating a set of statistics, model evaluation indicators, resampling the number of sub-samples, calculating a set of statistics, model evaluation indicators, a result integration method, and a preset result accuracy; circularly determining the optimal sample data, and each cycle period performs the following operations: resampling the data to be analyzed to obtain a periodic sub-sample; obtaining the calculation result of the periodic sub-sample according to the set of calculated statistics; obtaining the accuracy of the calculation result of the periodic sub-sample according to the model evaluation indicators; integrating the accuracy of the calculation results of the sub-samples of each period according to the result integration method to obtain the accuracy of the periodic integration result; when the accuracy of the periodic integration result reaches the preset result accuracy, obtaining the optimal sample data. The present invention can perform big data analysis efficiently and accurately to obtain high-precision sample data, enabling efficient and accurate modeling under limited hardware conditions.

[0003] In order to improve the analysis efficiency and correlation when analyzing data, most existing big data analysis methods obtain data by comparing sampling samples. However, using the method of comparing samples to analyze big data makes the obtained resources incomplete. On the other hand, the existing screening and comparison methods are single in procedure, resulting in certain data omission problems during the screening process. In view of the above problems in the prior art, it is necessary to improve the existing equipment. Summary of the Invention

[0004] The purpose of the present invention is to provide a big data analysis method with a screening function to solve the problem that most existing big data analysis methods obtain data by comparing sampling samples in order to improve the analysis efficiency and correlation when analyzing data, but using the method of comparing samples to analyze big data makes the obtained resources incomplete as mentioned in the above background art.

[0005] To achieve the above purpose, the present invention provides the following technical solution: A big data analysis method with a screening function, including the following steps, characterized in that:

[0006] S1: Data acquisition stage:

[0007] The acquisition module obtains relevant information of customers through the customer information database and the mobile module, and the obtained information will be aggregated and integrated into the target database through the integration module;

[0008] S2: Data screening stage:

[0009] The big data server is connected to the Internet big data, and then the analysis module and the screening module are used to preliminarily screen the relevant data;

[0010] S21: The data that does not meet the screening requirements will be automatically deleted by the deletion module;

[0011] S22: The data that meets the requirements will be displayed through the first display module and stored in the data area module.

[0012] S3: Data comparison stage:

[0013] The comparison module is used to compare the data in the data area module with the data in the existing target database;

[0014] S31: The recheck module is used to recheck the resources after comparison;

[0015] S4: Use the correlation ranking module to rank the resources;

[0016] S41: Use the encryption module to encrypt the data to be transmitted.

[0017] Preferably, the big data server is connected to the Internet big data, the analysis module analyzes and processes the accessed big data, and then the screening module preliminarily screens the relevant data after analysis.

[0018] Preferably, the deletion module will automatically delete the data that does not meet the screening requirements, and the data that meets the requirements will be displayed through the first display module and stored in the data area module.

[0019] Preferably, the comparison module compares the data in the data area module with the data in the existing target database. When the data in the data area module does not meet any of the preference data in the target database, the ignore module will ignore the corresponding resources, and the data that meets the preference requirements will be displayed through the second display module.

[0020] Preferably, the recheck module will recheck the displayed resources, and the data that meets the recheck requirements will be displayed through the third display module. The data that does not meet the requirements will be ignored again if it fails the comparison analysis of the comparison module three times.

[0021] Preferably, the content displayed by the third display module is selected to be arranged according to data correlation by the correlation arrangement module, and is also arranged according to the data release time before and after by the time arrangement module.

[0022] Preferably, the third display module includes a correlation arrangement module and a time arrangement module, and both the correlation arrangement module and the time arrangement module are connected to the classification module.

[0023] Preferably, the classification module will store the classified data inside the second storage module.

[0024] Preferably, the encryption module encrypts the data in the second storage module and then sends it to the customer through the sending module. At the same time, the digital encryption module is selected for digital encryption according to requirements, or the text encryption module is used for text encryption.

[0025] Compared with the prior art, the beneficial effects of the present invention are as follows: This big data analysis method with a screening function

[0026] (1) By the combined use of the customer information library, the mobile terminal module, the acquisition module, and the integration module, the present invention can effectively solve the problem that in order to improve the analysis efficiency and correlation degree when analyzing data in the existing big data analysis methods, most of them obtain data by comparing sampling samples, and analyzing big data by using the method of comparing samples makes the obtained resources not comprehensive enough. The acquisition module can obtain the data of customers from multiple sources through the customer information library and the mobile terminal module. At the same time, the integration module can centrally integrate the obtained data, thereby facilitating the enrichment of the comparison samples and ensuring the comprehensiveness of resource acquisition.

[0027] (2) By the combined use of the screening module and the comparison module, the present invention can effectively solve the problem that big data is prone to low analysis efficiency due to complex and repeated access resources. The screening module can screen the information obtained by accessing the big data server, and the comparison module can avoid repeated and useless data from repeatedly comparing with the data in the target database, thereby facilitating the improvement of the analysis efficiency. At the same time, the repeated and useless data will be deleted and ignored by the deletion module and the ignoring module during screening and comparison, so as to facilitate the reduction of the displayed resources and more intuitively view and obtain the resources through the relevant display module.

[0028] (3) By using the comparison module, re-inspection module, and third display module in combination, the present invention can effectively solve the problem that the existing screening and comparison methods have a single program, resulting in certain data omissions. The re-inspection module can re-inspect the data used for display by the third display module, that is, the data that has been screened and compared. The data that meets the re-inspection requirements will be displayed through the third display module, and the data that does not meet the re-inspection requirements will be output to the comparison module again for re-comparison. If the data that does not meet the re-inspection requirements fails to pass the comparative analysis of the comparison module three times, it will be ignored by the ignoring module again, thereby avoiding the problem of data omission to a certain extent.

[0029] (4) By using the third display module, correlation degree arrangement module, time arrangement module, and classification module in combination, the present invention can effectively solve the problem that the display of the final data obtained through analysis, that is, the viewing method is single and not convenient to meet the diverse viewing needs of users. The correlation degree arrangement module in the third display module can arrange the final data in order from front to back according to the correlation degree with the sample, or it can be arranged from new to old according to the data release date through the time arrangement module, thereby facilitating the quick and accurate search for relevant data. Brief Description of the Drawings

[0030] Figure 1 It is a schematic work flow diagram of the screening module of the present invention;

[0031] Figure 2 It is a schematic work flow diagram of the generation of the target database of the present invention;

[0032] Figure 3 It is a schematic work flow diagram of the control relationship among the data area module, comparison module, target database, and third display module of the present invention;

[0033] Figure 4 It is a schematic work flow diagram of the re-inspection module of the present invention;

[0034] Figure 5 It is a schematic work flow diagram of the output relationship between the third display module and the second storage module of the present invention;

[0035] Figure 6 It is a schematic work flow diagram of the composition and input-output relationship of the encryption module of the present invention.

[0036] In the figure: 1. Customer information database; 2. Mobile terminal module; 3. Acquisition module; 4. Integration module; 5. Target database; 6. Big data server; 7. Analysis module; 8. Screening module; 9. First display module; 10. Deletion module; 11. Data area module; 12. Comparison module; 13. Second display module; 14. Ignoring module; 15. Re-inspection module; 16. Third display module; 17. Relevance ranking module; 18. Time ranking module; 19. Classification module; 20. Second storage module; 21. Encryption module; 22. Digital encryption module; 23. Text encryption module; 24. Sending module. Specific implementation manners

[0037] Next, the technical solutions in the embodiments of the present invention will be clearly and completely described in conjunction with the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all the embodiments. All other embodiments obtained by those of ordinary skill in the art based on the embodiments of the present invention without creative efforts shall fall within the protection scope of the present invention.

[0038] Please refer to Figure 1-6 , the present invention provides a technical solution: a big data analysis method with a screening function, including the following steps. S1: Data acquisition stage:

[0039] The acquisition module 3 acquires relevant information of customers through the customer information database 1 and the mobile terminal module 2, and the acquired information will be collected and integrated into the target database 5 through the integration module 4.

[0040] S2: Data screening stage:

[0041] The big data server 6 is used to access the Internet big data, and then the analysis module 7 and the screening module 8 are used to preliminarily screen the relevant data.

[0042] S21: The data that does not meet the screening requirements will be automatically deleted by the deletion module 10.

[0043] S22: The data that meets the requirements will be displayed through the first display module 9 and stored in the data area module 11.

[0044] S3: Data comparison stage:

[0045] The comparison module 12 is used to compare the data in the data area module 11 with the data in the existing target database 5.

[0046] S31: The re-inspection module 15 is used to re-inspect the resources after comparison.

[0047] S4: Use the relevance ranking module 17 to rank the resources.

[0048] S41: Use the encryption module 21 to encrypt the data to be transmitted.

[0049] Use the big data server 6 to access the Internet big data. The analysis module 7 will analyze and process the accessed big data, and then the screening module 8 will initially screen the relevant data after analysis.

[0050] The deletion module 10 will automatically delete the data that does not meet the screening requirements. The data that meets the requirements will be displayed through the first display module 9 and stored in the data area module 11.

[0051] The comparison module 12 compares the data in the data area module 11 with the data in the existing target database 5. When the data in the data area module 11 does not meet any of the preference data in the target database 5, the ignore module 14 will ignore the corresponding resources. The data that meets the preference requirements will be displayed through the second display module 13.

[0052] The re-inspection module 15 will re-inspect the displayed resources. The data that meets the re-inspection requirements will be displayed through the third display module 16. The data that does not meet the requirements will be ignored again if it fails the comparison and analysis of the comparison module 12 three times.

[0053] The content displayed by the third display module 16 is sorted by the correlation degree arrangement module 17 according to the data correlation degree, and also sorted by the time arrangement module 18 according to the data release time before and after.

[0054] The third display module 16 includes the correlation degree arrangement module 17 and the time arrangement module 18, and both the correlation degree arrangement module 17 and the time arrangement module 18 are connected to the classification module 19.

[0055] The classification module 19 will store the classified data in the second storage module 20.

[0056] The encryption module 21 encrypts the data in the second storage module 20 and then sends it to the customer through the sending module 24. At the same time, according to the requirements, the digital encryption module 22 is selected for digital encryption, or the text encryption module 23 is used for text encryption.

[0057] Such as Figure 1 、 Figure 2 、 Figure 3 、 Figure 4 、 Figure 5 and Figure 6As shown, during the actual working process, the acquisition module 3 can obtain relevant information of customers before through the customer information database 1 and the mobile terminal module 2. The obtained information will be collected and integrated into the target database 5 through the integration module 4. The big data server 6 is used to access network big data. The analysis module 7 will analyze and process the accessed big data. Then, the screening module 8 will conduct a preliminary screening on the relevant data after analysis. The data that does not meet the screening requirements will be input into the deletion module 10 and automatically deleted by the deletion module 10. The data that meets the requirements will be displayed through the first display module 9 and stored in the data area module 11. The comparison module 12 can compare the data in the data area module 11 with the data in the existing target database 5. When the data in the data area module 11 does not conform to any preference data in the target database 5, the ignoring module 14 will ignore the corresponding resources. The data that meets the preference requirements will be displayed through the second display module 13. Immediately afterwards, the re-inspection module 15 will conduct a re-inspection on the displayed resources. The data that meets the re-inspection requirements will be displayed through the third display module 16. The data that does not meet the requirements will be ignored again if it fails the comparison and analysis of the comparison module 12 three times. The content displayed by the third display module 16 can be arranged by the correlation degree arrangement module 17 according to the high and low data correlation degree, or can be arranged by the time arrangement module 18 according to the sequence of data release time. The arranged data will be stored inside the second storage module 20 after being classified by the classification module 19. Then, the encryption module 21 can encrypt the data in the second storage module 20 and send it to the customer through the sending module 24. The customer can view the received data after decryption.

[0058] Although the present invention has been described in detail with reference to the foregoing embodiments, for those skilled in the art, they can still modify the technical solutions recorded in the foregoing embodiments, or perform equivalent replacements on some of the technical features. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

Claims

1. A big data analysis method with a screening function, comprising the following steps, characterized in that: S1: Data acquisition stage: The acquisition module (3) obtains relevant information of customers through the customer information database (1) and the mobile terminal module (2), and the obtained information will be collected and integrated into the target database (5) through the integration module (4); S2: Data screening stage: The big data server (6) accesses the Internet big data, and then the analysis module (7) and the screening module (8) conduct preliminary screening on the relevant data; S21: The data that does not meet the screening requirements will be automatically deleted by the deletion module (10); S22: The data that meets the requirements will be displayed through the first display module (9) and stored in the data area module (11); S3: Data comparison stage: The comparison module (12) compares the data in the data area module (11) with the data in the existing target database (5); When the data in the data area module (11) does not conform to any of the preference data in the target database (5), the ignore module (14) will ignore the corresponding resources, and the data that meets the preference requirements will be displayed through the second display module (13); S31: The re-inspection module (15) re-inspects the resources after comparison; The data that meets the re-inspection requirements will be displayed through the third display module 16, and the data that does not meet the requirements will be ignored again if it fails the comparison analysis of the comparison module 12 three times; S4: The resources can be arranged according to the data correlation degree by the correlation degree arrangement module (17) or arranged according to the data release time before and after by the time arrangement module 18; The encryption module (21) encrypts the data in the second storage module (20) and then sends it to the customer through the sending module (24), and at the same time, the digital encryption module (22) is selected for digital encryption according to the requirements, or the text encryption module (23) is used for text encryption.

Images