A data processing system based on a data resource catalog
By automatically extracting and analyzing resource object information in the data resource directory construction system and generating abbreviated content information, the problem of difficulty for users to locate data in the directory is solved, and the efficiency and accuracy of data access are improved.
Patent Information
- Application Number
- CN202210672567.8
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2022-06-14
- Publication Date
- 2024-11-26
- Estimated Expiration
- 2042-06-14
AI Technical Summary
After the data resource directory is built, it is difficult for users to quickly locate and call the required data, resulting in frequent call errors.
Design a data processing system based on data resource catalogs. By storing resource object information and its catalogs in the database, and transferring object catalogs to the display unit using self-acquisition unit and synchronization unit. Users can click on the required catalogs, and the system will automatically Extract and analyze resource object information to generate abbreviated content information to facilitate users to confirm the required content.
It simplifies the user's content positioning and selection in the data resource directory, reduces the occurrence of call errors, and improves the efficiency and accuracy of data access.
Smart Images

Figure CN114969052B_ABST
Abstract
Description
Technical Field
[0001] The invention belongs to the field of data processing, and in particular is a data processing system constructed based on a data resource directory. Background Art
[0002] The patent with publication number CN111159180A discloses a data processing method and system based on data resource directory construction, the method includes: classifying the power information big data collected from each data resource directory construction and storing it according to data type; selecting corresponding data from the data stored according to data type according to business needs and each data resource directory construction, centrally storing and processing to form business data; providing the business data to users; wherein, the data resource directory construction is stored based on database type; the data type includes model data, operation data and statistical analysis data; and improving the data performance of different businesses in multiple aspects of control big data.
[0003] However, when building a data resource directory, after the directory is built, it is usually to facilitate users to quickly access the data in the corresponding directory for easy reading or calling. However, usually, if the user is not familiar with the directory content, calling errors often occur when calling. In order to locate the content that the user wants more quickly, we need a solution. Summary of the invention
[0004] The object of the present invention is to provide a data processing system constructed based on a data resource directory.
[0005] The purpose of the present invention can be achieved through the following technical solutions:
[0006] A data processing system based on a data resource directory, comprising:
[0007] The database stores a number of resource object information, which is data information that needs to be analyzed or read, and also contains a directory of the resource object information, which is marked as an object directory. The object directory contains an address link, and when the link is clicked, it will automatically go to the corresponding resource object information; the self-acquisition unit is used to obtain the object directory in the database and transmit it to the display unit;
[0008] The synchronization unit is used to synchronize the resource object information and its corresponding object directory in the database, and transmit it to the data self-extraction unit, which is used to extract and analyze the resource object information and the object directory, and determine the abbreviated content information of all resource object information according to the specific situation of the extraction and analysis;
[0009] The data self-extraction unit is used to transmit all object directories and their corresponding thumbnail content information to the point selection library, and the point selection library receives the object directories and their corresponding thumbnail content information transmitted by the data self-extraction unit and stores them in real time;
[0010] The self-acquisition unit is used to acquire all object directories in the database and transmit them to the display unit for real-time display;
[0011] The selection module is used for users to select the required object directory according to the object directory displayed by the display unit. After clicking, the selected object directory is transmitted to the processor. The processor automatically retrieves the thumbnail content information corresponding to the object directory in the corresponding selection library from the selection library and transmits it to the display unit for real-time display.
[0012] Furthermore, the specific method of extraction and analysis is as follows:
[0013] Step 1: Get all object directories and select any object directory;
[0014] Step 2: Then obtain the resource object information of the corresponding object directory;
[0015] Step 3: Extract the importance of resource object information to obtain all intercepted segments Pi, and their corresponding number of copies Fi, total stay time TI and number of characters Zi;
[0016] Step 4: Use the formula to calculate the selected value Qi. The specific calculation formula is:
[0017] Qi=0.58*Ti+0.42*Fi;
[0018] In the formula, 0.58 and 0.42 are the preset weights;
[0019] Step 5: Then, according to the order of Qi from large to small, the intercepted segments Pi are selected in sequence to obtain all the combinations of segments to be selected;
[0020] Step 6: Select and represent the candidate segment combination, specifically, obtain the special selected values Qi of the candidate segments in all the candidate segment combinations, add them up, and mark the combination with the smallest value as the target candidate segment combination;
[0021] Step 7: Merge the target candidate segment combination and the selected intercepted segment, and mark the obtained content as abbreviated content information;
[0022] Step 8: Select the next object directory, obtain the corresponding resource object information, repeat steps 2 to 8, and obtain the abbreviated content information of all resource object information.
[0023] Furthermore, the specific method of extracting importance in step three is:
[0024] S1: Get the specific content in the resource object information and perform data interception;
[0025] S2: Get all the copied segments, mark them as intercepted segments, and get all the intercepted segments;
[0026] S3: Obtain the number of times all the intercepted segments are copied. When a certain intercepted segment has been copied but also exists in another longer intercepted segment, it will also be counted as a copy. The number of times all the intercepted segments are copied is obtained, and the intercepted segments are marked as Pi, i=1...n, and the corresponding number of copies is marked as Fi, i=1...n;
[0027] S4: Then, the residence time of the corresponding intercepted segment in the past six months is obtained, where the residence time is the time the intercepted segment stays on the screen, and all the residence times are added together to obtain the total residence time; the total residence time of all the intercepted segments is obtained and marked as Ti, where i=1...n;
[0028] S5: synchronously obtain the number of characters of all intercepted segments and mark them as Zi, i=1...n;
[0029] S6: Obtain all the intercepted segments Pi, and their corresponding number of copies Fi, total stay time TI and number of characters Zi.
[0030] Furthermore, in step 5, the specific method of sequentially selecting the intercepted segments Pi is:
[0031] Get the set character threshold Y, where Y is the value preset by the administrator;
[0032] According to the order of Qi from large to small, the intercepted segments Pi are selected in turn, and the number of characters Zi corresponding to the selected intercepted segments Pi are added, and the sum value is marked as the selected sum value. When the selected sum value is greater than or equal to Y / 2, a patchwork signal is generated; the character threshold Y is subtracted from the selected sum value, and the obtained value is marked as the remaining sum value;
[0033] When a patchwork signal is generated, the unselected intercepted segments Pi are automatically marked as the segments to be selected;
[0034] Combine all the candidate segments, and select all the combinations that satisfy the sum of the number of characters in the intercepted segment Pi, which is equal to the residual sum or infinitely close to the residual sum. Infinitely close to the residual sum means that the sum of the residual sum minus the sum of the number of characters in the intercepted segment after the combination is less than or equal to Y1, where Y1 = 0.05*Y;
[0035] Get all the candidate fragment combinations.
[0036] Furthermore, the specific method of extraction and analysis is as follows:
[0037] S01: Obtain all object directories, select an object directory, and obtain resource object information of the corresponding object directory;
[0038] S02: extract the importance of resource object information to obtain each resource word and its corresponding word occurrence times;
[0039] S03: Then, data is screened, specifically by obtaining a number of common words;
[0040] S04: remove common words from resource segmentation words, and then sort the remaining resource segmentation words according to the number of occurrences of the corresponding segmentation words;
[0041] S05: Obtain the complete content of the current segment where the remaining resource segmentation words are located. The complete content refers to the complete content between the two delimiters that appear before and after the resource segmentation word, mark it as a complete segment, and obtain all the complete segments;
[0042] S06: After obtaining the set character threshold Y, the resource segmentations contained in the complete segment are sorted in order according to the top segmentation, and the corresponding complete segments are selected in sequence until the number of characters of all complete segments is greater than or equal to Y, then the selection of complete segments is stopped, the selected complete segments are merged, and the obtained content is marked as abbreviated content information;
[0043] S07: Select the next object directory, obtain the corresponding resource object information, repeat steps S01 to S07, and obtain the abbreviated content information of all resource object information.
[0044] Furthermore, the specific method of extracting the importance in step S02 is:
[0045] S021: Acquire specific content in the resource object information, perform word segmentation processing, and obtain a number of resource word segments;
[0046] S022: Obtain the number of occurrences of each resource segmentation word in the corresponding resource object information, and mark it as the segmentation word occurrence times.
[0047] Furthermore, the extraction analysis is performed on text content data.
[0048] Furthermore, the common word determination method in step S03 is:
[0049] After obtaining all resource object information and completing the word segmentation process, if there is any resource word, the number of times it appears in the resource object information is marked as the common occurrence times. Here, if it appears once or more in any resource object information, the common occurrence times is increased by one;
[0050] Divide the general occurrence times by the number of resource object information, and mark the resulting value as the occurrence ratio;
[0051] When the occurrence ratio exceeds B1, it is marked as a common word.
[0052] Beneficial effects of the present invention:
[0053] The present invention stores a plurality of resource object information in a database, and includes a directory of the resource object information, marking it as an object directory. The object directory includes an address link, and when the link is clicked, it will automatically go to the corresponding resource object information. Then, a self-acquisition unit is used to acquire the object directory in the database and transmit it to a display unit, so that the user can select different directories.
[0054] Afterwards, the resource object information and its corresponding object directory in the database are synchronized with the help of the synchronization unit, and transmitted to the data self-extraction unit, and the resource object information and the object directory are extracted and analyzed with the help of the data self-extraction unit, and the thumbnail content information of all the resource object information is determined according to the specific circumstances of the extraction and analysis; subsequently, when the user first clicks on the corresponding directory, the thumbnail information will be displayed, so as to facilitate confirmation whether the corresponding directory is the content required by the user. The present invention is simple, effective, and easy to use. BRIEF DESCRIPTION OF THE DRAWINGS
[0055] In order to facilitate understanding by those skilled in the art, the present invention is further described below with reference to the accompanying drawings.
[0056] Figure 1 It is a system block diagram of the present invention. DETAILED DESCRIPTION
[0057] like Figure 1 As shown, a data processing system built based on a data resource directory includes a data self-extraction unit, a synchronization unit, a database, a point selection library, a processor, a point selection module, a self-acquisition unit and a display unit;
[0058] The database stores a number of resource object information, which is data information that needs to be analyzed or read, and also contains a directory of the resource object information, which is marked as an object directory. The object directory contains an address link, and when the link is clicked, it will automatically go to the corresponding resource object information; the self-acquisition unit is used to obtain the object directory in the database and transmit it to the display unit for real-time display;
[0059] The synchronization unit is used to synchronize the resource object information and its corresponding object directory in the database, and transmit it to the data self-extraction unit. The data self-extraction unit is used to extract and analyze the resource object information and the object directory. The specific method of extraction and analysis is:
[0060] Step 1: Get all object directories and select any object directory;
[0061] Step 2: Then obtain the resource object information of the corresponding object directory;
[0062] Step 3: Extract the importance of resource object information. The specific method of extracting importance is as follows:
[0063] S1: Get the specific content in the resource object information and perform data interception;
[0064] S2: Get all the copied segments, mark them as intercepted segments, and get all the intercepted segments;
[0065] S3: Obtain the number of times all the intercepted segments are copied. When a certain intercepted segment has been copied but also exists in another longer intercepted segment, it will also be counted as a copy. The number of times all the intercepted segments are copied is obtained, and the intercepted segments are marked as Pi, i=1...n, and the corresponding number of copies is marked as Fi, i=1...n;
[0066] S4: Then, the residence time of the corresponding intercepted segment in the past six months is obtained, where the residence time is the time the intercepted segment stays on the screen, and all the residence times are added together to obtain the total residence time; the total residence time of all the intercepted segments is obtained and marked as Ti, where i=1...n;
[0067] S5: synchronously obtain the number of characters of all intercepted segments and mark them as Zi, i=1...n;
[0068] S6: Get all the intercepted segments Pi, and their corresponding number of copies Fi, total stay time TI and number of characters Zi;
[0069] Step 4: Use the formula to calculate the selected value Qi. The specific calculation formula is:
[0070] Qi=0.58*Ti+0.42*Fi;
[0071] In the formula, 0.58 and 0.42 are preset weights used to highlight the different importance of different factors;
[0072] Step 5: Then select the intercepted segments Pi in order from large to small according to Qi, and the selection method is:
[0073] Get the set character threshold Y, where Y is the value preset by the administrator;
[0074] According to the order of Qi from large to small, the intercepted segments Pi are selected in turn, and the number of characters Zi corresponding to the selected intercepted segments Pi are added, and the sum value is marked as the selected sum value. When the selected sum value is greater than or equal to Y / 2, a patchwork signal is generated; the character threshold Y is subtracted from the selected sum value, and the obtained value is marked as the remaining sum value;
[0075] When a patchwork signal is generated, the unselected intercepted segments Pi are automatically marked as the segments to be selected;
[0076] Combine all the candidate segments, and select all the combinations that satisfy the sum of the number of characters in the intercepted segment Pi, which is equal to the residual sum or infinitely close to the residual sum. Infinitely close to the residual sum means that the sum of the residual sum minus the sum of the number of characters in the intercepted segment after the combination is less than or equal to Y1, where Y1 = 0.05*Y;
[0077] Get all the candidate fragment combinations;
[0078] Step 6: Select and represent the candidate segment combination, specifically, obtain the special selected values Qi of the candidate segments in all the candidate segment combinations, add them up, and mark the combination with the smallest value as the target candidate segment combination;
[0079] Step 7: Merge the target candidate segment combination and the selected intercepted segment, and mark the obtained content as abbreviated content information;
[0080] Step 8: Select the next object directory, obtain the corresponding resource object information, repeat steps 2 to 8, and obtain the abbreviated content information of all resource object information;
[0081] The data self-extraction unit is used to transmit all object directories and their corresponding thumbnail content information to the point selection library, and the point selection library receives the object directories and their corresponding thumbnail content information transmitted by the data self-extraction unit and stores them in real time;
[0082] The self-acquisition unit is used to acquire all object directories in the database and transmit them to the display unit for real-time display for user selection;
[0083] The selection module is used for users to select the required object directory according to the object directory displayed by the display unit. After the selection, the selected object directory is transmitted to the processor. The processor automatically retrieves the thumbnail content information corresponding to the object directory in the corresponding selection library from the selection library and transmits it to the display unit for real-time display, so as to facilitate the user to make accurate selections;
[0084] As another embodiment of the present invention, the difference between the present invention and the first embodiment is that the data self-extraction unit is used to extract and analyze the resource object information and the object directory in a different manner. The extraction and analysis provided in this embodiment is specifically for the text content, and the specific method is:
[0085] S01: Obtain all object directories, select an object directory, and obtain resource object information of the corresponding object directory;
[0086] S02: Extract the importance of resource object information. The specific method of extracting the importance is as follows:
[0087] S021: Acquire specific content in the resource object information and perform word segmentation processing, wherein the word segmentation processing is implemented according to the existing technology to obtain a number of resource word segments;
[0088] S022: Obtain the number of occurrences of each resource segmentation word in the corresponding resource object information, and mark it as the segmentation word occurrence times;
[0089] S03: Then, data is screened out. The specific method is to obtain a number of common words. The common words are determined as follows:
[0090] After obtaining all resource object information and completing the word segmentation process, if there is any resource word, the number of times it appears in the resource object information is marked as the common occurrence times. Here, if it appears once or more in any resource object information, the common occurrence times is increased by one;
[0091] Divide the general occurrence times by the number of resource object information, and mark the resulting value as the occurrence ratio;
[0092] When the occurrence ratio exceeds B1, it is marked as a common word; B1 is generally set to 0.7, and other values set by the administrator can also be used;
[0093] S04: remove common words from resource segmentation words, and then sort the remaining resource segmentation words according to the number of occurrences of the corresponding segmentation words;
[0094] S05: Obtain the complete content of the current segment where the remaining resource segmentation words are located. The complete content refers to the complete content between the two delimiters that appear before and after the resource segmentation word, mark it as a complete segment, and obtain all the complete segments;
[0095] S06: After obtaining the set character threshold Y, the resource segmentations contained in the complete segment are sorted in order according to the top segmentation, and the corresponding complete segments are selected in sequence until the number of characters of all complete segments is greater than or equal to Y, then the selection of complete segments is stopped, the selected complete segments are merged, and the obtained content is marked as abbreviated content information;
[0096] S07: Select the next object directory, obtain the corresponding resource object information, repeat steps S01 to S07, and obtain the abbreviated content information of all resource object information.
[0097] The above contents are merely examples and explanations of the structure of the present invention. The technicians in this technical field may make various modifications or additions to the specific embodiments described or replace them in a similar manner. As long as they do not deviate from the structure of the invention or exceed the scope defined by the claims, they should all fall within the protection scope of the present invention.
Claims
1. A data processing system based on a data resource directory, It is characterized in that include: The database stores a number of resource object information, which is the data information that needs to be analyzed or read. It also contains a directory of resource object information, which is marked as an object directory. The object directory contains address links, and when you click the link, you will automatically be redirected to the corresponding resource object information; A self-acquisition unit, used to acquire the object directory in the database and transmit it to the display unit; A synchronization unit is used to synchronize the resource object information and the corresponding object directory in the database and transmit it to the data self-extraction unit. The data self-extraction unit is used to extract and analyze the resource object information and the object directory, and determine the abbreviated content information of all resource object information according to the specific situation of the extraction and analysis; The data self-extraction unit is used to transmit all object directories and their corresponding thumbnail content information to the point selection library, and the point selection library receives the object directories and their corresponding thumbnail content information transmitted by the data self-extraction unit and stores them in real time; The self-acquisition unit is used to acquire all object directories in the database and transmit them to the display unit for real-time display; The selection module is used for users to select the required object directory according to the object directory displayed by the display unit. After clicking, the selected object directory is transmitted to the processor. The processor automatically retrieves the thumbnail content information corresponding to the object directory in the corresponding selection library from the selection library and transmits it to the display unit for real-time display.
2. A data processing system based on a data resource directory according to claim 1, It is characterized in that The specific method of extraction and analysis is: Step 1: Get all object directories and select any object directory; Step 2: Then obtain the resource object information of the corresponding object directory; Step 3: Extract the importance of resource object information to obtain all intercepted fragments Pi, and their corresponding number of copies Fi, total stay time Ti and number of characters Zi; Step 4: Use the formula to calculate the selected value Qi. The specific calculation formula is: Qi=0.58*Ti+0.42*Fi; In the formula, 0.58 and 0.42 are the preset weights; Step 5: Then, according to the order of Qi from large to small, the intercepted segments Pi are selected in sequence to obtain all the combinations of segments to be selected; Step 6: Select and represent the candidate segment combination, specifically, obtain the special selected values Qi of the candidate segments in all the candidate segment combinations, add them up, and mark the combination with the smallest value as the target candidate segment combination; Step 7: Merge the target candidate segment combination and the selected intercepted segment, and mark the obtained content as abbreviated content information; Step 8: Select the next object directory, obtain the corresponding resource object information, repeat steps 2 to 8, and obtain the abbreviated content information of all resource object information.
3. A data processing system based on a data resource directory according to claim 2, It is characterized in that The specific method of extracting importance in step three is: S1: Get the specific content in the resource object information and perform data interception; S2: Get all the copied segments, mark them as intercepted segments, and get all the intercepted segments; S3: Obtain the number of times all the intercepted segments are copied. When a certain intercepted segment has been copied but also exists in another longer intercepted segment, it will also be counted as a copy. The number of times all the intercepted segments are copied is obtained, and the intercepted segments are marked as Pi, i=1...n, and the corresponding number of copies is marked as Fi, i=1...n; S4: Then, the residence time of the corresponding intercepted segment in the past six months is obtained, where the residence time is the time the intercepted segment stays on the screen, and all the residence times are added together to obtain the total residence time; the total residence time of all the intercepted segments is obtained and marked as Ti, where i=1...n; S5: synchronously obtain the number of characters of all intercepted segments and mark them as Zi, i=1...n; S6: Obtain all intercepted segments Pi, and their corresponding number of copies Fi, total stay time Ti and number of characters Zi.
4. A data processing system based on a data resource directory according to claim 2, It is characterized in that The specific method of selecting the intercepted segments Pi in sequence in step 5 is: Get the set character threshold Y, where Y is the value preset by the administrator; According to the order of Qi from large to small, the intercepted segments Pi are selected in turn, and the number of characters Zi corresponding to the selected intercepted segments Pi are added, and the sum value is marked as the selected sum value. When the selected sum value is greater than or equal to Y / 2, a patchwork signal is generated; the character threshold Y is subtracted from the selected sum value, and the obtained value is marked as the remaining sum value; When a patchwork signal is generated, the unselected intercepted segments Pi are automatically marked as the segments to be selected; Combine all the candidate segments, and select all the combinations that satisfy the sum of the number of characters in the intercepted segment Pi, which is equal to the residual sum or infinitely close to the residual sum. Infinitely close to the residual sum means that the sum of the residual sum minus the sum of the number of characters in the intercepted segment after the combination is less than or equal to Y1, where Y1 = 0.05*Y; Get all the candidate fragment combinations.
5. A data processing system based on a data resource directory according to claim 1, It is characterized in that The specific method of extraction and analysis is: S01: Obtain all object directories, select an object directory, and obtain resource object information of the corresponding object directory; S02: extract the importance of resource object information to obtain each resource word and its corresponding word occurrence times; S03: Then, data is screened, specifically by obtaining a number of common words; S04: remove common words from resource segmentation words, and then sort the remaining resource segmentation words according to the number of occurrences of the corresponding segmentation words; S05: Obtain the complete content of the current segment where the remaining resource segmentation words are located. The complete content refers to the complete content between the two delimiters that appear before and after the resource segmentation word, mark it as a complete segment, and obtain all the complete segments; S06: After obtaining the set character threshold Y, the resource segmentations contained in the complete segment are sorted in order according to the top segmentation, and the corresponding complete segments are selected in sequence until the number of characters of all complete segments is greater than or equal to Y, then the selection of complete segments is stopped, the selected complete segments are merged, and the obtained content is marked as abbreviated content information; S07: Select the next object directory, obtain the corresponding resource object information, repeat steps S01 to S07, and obtain the abbreviated content information of all resource object information.
6. A data processing system based on a data resource directory according to claim 5, It is characterized in that The specific method of extracting importance in step S02 is: S021: Acquire specific content in the resource object information, perform word segmentation processing, and obtain a number of resource word segments; S022: Obtain the number of occurrences of each resource segmentation word in the corresponding resource object information, and mark it as the segmentation word occurrence times.
7. A data processing system based on a data resource directory according to claim 5, It is characterized in that Extraction analysis targets text content data.
8. A data processing system based on a data resource directory according to claim 5, It is characterized in that The common word determination method in step S03 is: After obtaining all resource object information and completing the word segmentation process, if there is any resource word, the number of times it appears in the resource object information is marked as the common occurrence times. Here, if it appears once or more in any resource object information, the common occurrence times is increased by one; Divide the general occurrence times by the number of resource object information, and mark the resulting value as the occurrence ratio; When the occurrence ratio exceeds B1, it is marked as a common word.
Citation Information
Patent Citations
Data processing method and system based on data resource directory construction
CN111159180A
Data encryption system based on data analysis
CN114282241A
Data standardization management method and sharing system
CN114519085A
Method for monitoring database directory, storage medium, terminal device and apparatus
WO2019242123A1
