Data authority control method and system for data segmentation

By constructing a core keyword collection, segmenting and permission configuration of data texts is solved, the high cost of data permission settings in the existing technology is solved, efficient data permission management is realized, and the cost and complexity of data permission configuration is reduced.

CN119089475APending Publication Date: 2024-12-06JIAXING HAOHAN SOFTWARE TECH CO LTD
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202411258172.9
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-09-09
Publication Date
2024-12-06

AI Technical Summary

Technical Problem

In the prior art, data permission setting requires a large number of early data classification operations, resulting in high cost of identification and classification storage, and it is difficult to effectively manage different content permissions in big data texts.

Method used

By building a collection of core keywords, segmenting the data text, finding the location and frequency information of core keywords, performing category segmentation, and setting data permissions based on different types of core keywords, directly segmenting on the original data Identify and configure to avoid complex classification storage.

Benefits of technology

It reduces the preprocessing cost of data permissions, improves the efficiency of data permission configuration, significantly reduces the cost of configuration of different data permissions, and realizes collaborative processing operations under different data permissions.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119089475A_ABST
    Figure CN119089475A_ABST
Patent Text Reader

Abstract

The invention provides a data authority management method and system for data segmentation, and the method comprises the steps: obtaining original data, carrying out the data cleaning of the original data, obtaining clean data, sorting the clean data according to an obtained time sequence, and obtaining sequence data; core keyword sets used for segmentation are configured in advance, each core keyword set represents a segmented data type, and each corresponding core keyword in the sequence data is searched and matched according to the core keyword sets; obtaining the matched position and number of each core keyword in the sequence data, and screening and determining an initial position keyword and a tail position keyword according to the position and number of each core keyword; and segmenting the sequence data according to the positions of the sequence data where the initial position keyword and the tail position keyword are located, and configuring a corresponding data authority for the data sequence of each segment.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the technical field of data authority processing, and in particular to a data authority management method and system for data segmentation. Background Art

[0002] With the development of the Internet, a large amount of data is generated every moment, so we are faced with the problem of managing huge amounts of data. For a large data text, it may include different types of data content, and different data contents may require people with different permissions to view or process them. Especially for large data texts involving confidential file contents, the access to different contents even requires setting permissions. The setting of data permissions for data texts in the existing technical solutions is implemented on the basis of classified storage, such as pre-identifying the data content type through different interfaces, and storing different types of data content in different storage spaces. The data content in different storage spaces is configured with different data permissions according to its own attributes. The data permission control in the above-mentioned prior art requires a large number of preliminary data classification operations, wherein the data classification operation involves including but not limited to manual classification, and the identification and classification of natural language model-related technical solutions. Therefore, the data permission setting in the prior art has a high identification and classification storage cost. Summary of the invention

[0003] One of the inventive purposes of the present invention is to provide a data permission management method and system for data segmentation, wherein the method and system segments data text by constructing a core keyword set for different segments, searches for position information and frequency information of corresponding core keywords in the data text through the core keywords, and classifies the data text according to the position information and frequency information of the core keywords, and assigns different data permissions to the class-segmented data texts. Therefore, the present invention does not need to perform manual or related model text classification on big data, and can directly perform direct segmentation recognition based on the original data, and there is no need to classify and store the original data, which effectively reduces the preprocessing cost of data permissions.

[0004] Another inventive object of the present invention is to provide a data permission management method and system for data segmentation, wherein the method and system need to determine the initial position and the end position of the preliminary screening of the core keywords corresponding to the data text, and perform character frequency calculation on the core keywords between the initial position and the end position of the preliminary screening, and set a character frequency threshold, and locate the preliminary screening core keywords and the end core keywords in the data text on the basis of satisfying the character frequency threshold, and segment the data text based on the located initial position core keywords and the end position keywords, so as to obtain segmented data texts of different types of core keywords, and give corresponding data permissions to each segmented data text according to its own core keyword type.

[0005] Another inventive object of the present invention is to provide a data permission management method and system for data segmentation, wherein the method and system only perform data text segmentation in the original data, without the need for complex classification of the original text, thereby effectively improving the efficiency of data permission configuration and significantly reducing the cost of different data permission configurations, and can realize collaborative processing operations under different data permissions.

[0006] In order to achieve at least one of the above-mentioned invention objects, the present invention further provides a data rights management method for data segments, the method comprising: Acquire raw data, clean the raw data to obtain clean data, and sort the clean data in the order of acquisition time to obtain sequence data; Preconfigure a core keyword set for segmentation, wherein each core keyword set represents a segmentation data type, and search for each corresponding core keyword in the sequence data according to the core keyword set; Obtaining the position and quantity of each core keyword in the sequence data that is matched in the sequence data, and screening and determining the initial position keyword and the final position keyword according to the position and quantity of each core keyword; The sequence data is segmented according to the sequence data positions where the initial position keyword and the end position keyword are located, and corresponding data permissions are configured for each segmented data sequence.

[0007] According to one of the preferred embodiments of the present invention, the method for constructing the core keyword set includes: selecting keywords in advance according to user needs, analyzing keyword attribute characteristics, calculating the average cosine similarity value of all keyword attributes, setting a similarity difference threshold, calculating the similarity value between each keyword attribute and all other keywords, and taking the average value of the similarity values. If the difference between the average value of the similarity of the corresponding keyword attribute and the average cosine similarity value of all keyword attributes is greater than the preset similarity difference threshold, the keyword is eliminated, otherwise it is retained as a core keyword to construct the core keyword set.

[0008] According to another preferred embodiment of the present invention, the raw data cleaning method includes: converting the raw data into text data in a unified format, and sorting the raw data in the chronological order of acquisition of the raw data to obtain an initial serial number of each character in the raw data, and performing data cleaning in a manner including removing stop words, removing spaces, and deleting repeated characters, and sorting the cleaned raw data again in the chronological order of acquisition of the raw data to obtain sequence data and corresponding sequence values ​​of each raw data character.

[0009] According to another preferred embodiment of the present invention, the method for screening and determining the keywords of the initial position keywords and the last position keywords includes: taking the sequence value of each core keyword in the core keyword set in the sequence data, and obtaining the core keywords with the highest sequence value and the lowest sequence value in the core keyword set, and further based on the core keywords with the highest sequence value and the lowest sequence value, taking the core keyword with the highest sequence value and the core keyword with the lowest sequence value as the basis for segmenting the sequence data, taking the starting character of the core keyword with the lowest core keyword sequence value as the starting segmentation cutting point of the sequence data, and taking the core keyword with the highest sequence value and the last character as the segmentation points of the last segment.

[0010] According to another preferred embodiment of the present invention, the method for determining the segmentation points of the sequence data includes: calculating the position where each core keyword in the core keyword set appears in the sequence data, and calculating the average value of all character sequences in which each core keyword appears in the sequence data, and using the average value of all character sequences as the annotation data of the corresponding core keyword, numerically sorting all the annotation data to obtain the maximum and minimum values ​​of the annotation data, and searching for the sequence value of the initial character and the sequence value of the last character of the corresponding core keyword according to the maximum and minimum values ​​of the annotation data.

[0011] According to another preferred embodiment of the present invention, the method for determining the segmentation points of the sequence data includes: according to the annotation data of each core keyword, starting from the two ends of the maximum and minimum extreme values ​​of the core keyword sequence value, respectively screening out n adjacent reference core keyword groups, and calculating the number of annotation data between the two reference core keyword groups located at the two ends of the extreme values ​​from the outside to the inside, and calculating the number of characters of the core keyword corresponding to each annotation data, and calculating the total number of first characters of all core keywords corresponding to the annotation data; calculating the total number of second characters of all sequence data between the two reference core keyword groups, dividing the first total number of characters and the second total number of characters to obtain the character frequency value of the core keyword, and screening and determining the initial core keyword and the ending core keyword according to the core keyword character frequency value.

[0012] According to another preferred embodiment of the present invention, a character frequency value threshold is set in advance, and the character frequency values ​​of n reference core keyword groups are calculated from the outside to the inside in sequence. When the character frequency value of the core keyword of one group of reference core keyword groups is greater than the character frequency value threshold, the group of reference core keyword groups is the target core keyword group, and further calculation of the character frequency values ​​of the n reference core keyword groups is stopped, and the sequence data is segmented according to the position of the target core keyword group.

[0013] According to another preferred embodiment of the present invention, the segmentation method of the target core keyword group includes: obtaining the highest sequence value and the lowest sequence value of the characters in the target core keyword group, and segmenting according to the highest sequence value and the lowest sequence value of the characters in the target core keyword group, and configuring corresponding data permissions for each segmented sequence data to the corresponding user.

[0014] In order to achieve at least one of the above-mentioned purposes of the invention, the present invention further provides a data permission management system for data segments, and the system executes the above-mentioned data permission management method for data segments.

[0015] The present invention further provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and the computer program is executed by a processor to implement the above-mentioned data segmentation data permission management method. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] Figure 1 Shown is a flow chart of a data rights management method for data segments of the present invention. DETAILED DESCRIPTION

[0017] The following description is used to disclose the present invention so that those skilled in the art can implement the present invention. The preferred embodiments described below are only examples, and those skilled in the art can think of other obvious variations. The basic principles of the present invention defined in the following description can be applied to other embodiments, variations, improvements, equivalents, and other technical solutions that do not deviate from the spirit and scope of the present invention.

[0018] It is to be understood that the term "one" should be understood as "at least one" or "one or more", that is, in one embodiment, the number of an element may be one, while in another embodiment, the number of the element may be multiple, and the term "one" should not be understood as a limitation on the quantity.

[0019] Please refer to Figure 1 The present invention discloses a data permission management method and system for data segmentation, and the method mainly includes the following steps: first, it is necessary to obtain original data, wherein the original data may be original data from different channels, and a corresponding timestamp is configured for each original data; after obtaining the timestamp of the original data, the original data is cleaned and sorted according to the timestamp to obtain sequence data. After completing the data cleaning of the original data, the present invention further performs a core keyword identification and positioning operation including the original data. The core keywords in the present invention need to be pre-configured, and each data type has a different core keyword type. There may be multiple core keywords that need to be segmented, so it is necessary to screen the core keywords for segmentation, so as to obtain a core keyword set, and use the core keyword set to search for sequence data, so as to find sequence data segments with more core keywords in the sequence data. Further perform the segmentation and permission configuration of the corresponding sequence data segment. In the complete sequence data in the present invention, the sequence data segment at the same position can be set to an exclusive or exclusive permission configuration. For example, in sequence data segment A, it can be configured as permission a1 and permission a2 at the same time, when the permission of the sequence segment A is only exclusively selected between permission a1 and permission a2. When the sequence data segment A has and can only configure the permission a1, the sequence data segment is an exclusive permission configuration. The permission configuration method corresponding to the sequence data segment can be configured according to actual needs, and the present invention will not be described in detail.

[0020] Furthermore, the present invention needs to configure a core keyword set, which is used to determine the user's demand for data types, thereby being used to filter out sequence segments from complete sequence data. The method for constructing the core keyword set includes: selecting keywords in advance according to user needs, analyzing keyword attribute characteristics, calculating the average cosine similarity value of all keyword attributes, setting a similarity difference threshold, calculating the similarity value between each keyword attribute and all other keywords, and taking the average of the similarity values. If the difference between the average value of the similarity of the corresponding keyword attribute and the average cosine similarity value of all keyword attributes is greater than the preset similarity difference threshold, the keyword is eliminated, otherwise it is retained as a core keyword to construct a core keyword set.

[0021] In order to improve the accuracy of searching for character data such as core keywords in the original data, the present invention further performs a cleaning operation on the original data. The original data cleaning method includes: converting the original data into text data in a unified format, and sorting the original data in the chronological order of acquisition of the original data to obtain an initial serial number of each character in the original data, and performing data cleaning in a manner including removing stop words, removing spaces, and deleting repeated characters, and sorting the cleaned original data again in the chronological order of acquisition of the original data to obtain sequence data of each original data character and a corresponding sequence value.

[0022] It should be noted that the initial position keywords and the final position keywords are the core keyword groups for data segment classification. Since some core keywords may be scattered in other sequence data due to non-core descriptions, the initial position keywords and the final position keywords need to be identified and located in the present invention.

[0023] In one of the preferred embodiments of the present invention, if the clustering of the initial position keywords and the last position keywords is not considered, the keyword screening and determination method of the initial position keywords and the last position keywords includes: taking the sequence value of each core keyword in the core keyword set in the sequence data, and obtaining the core keywords with the highest sequence value and the lowest sequence value in the core keyword set, and further based on the core keywords with the highest sequence value and the lowest sequence value, taking the core keyword with the highest sequence value and the core keyword with the lowest sequence value as the basis for segmenting the sequence data, taking the starting character of the core keyword with the lowest core keyword sequence value as the starting segmentation cutting point of the sequence data, and taking the core keyword with the highest sequence value and the last character as the segmentation point of the last segment.

[0024] The method for determining the segmentation points of the sequence data includes: calculating the position of each core keyword in the core keyword set in the sequence data, and calculating the average value of all character sequences in which each core keyword appears in the sequence data, and using the average value of all character sequences as the annotation data of the corresponding core keyword, numerically sorting all the annotation data to obtain the maximum and minimum values ​​of the annotation data, and searching for the sequence value of the initial character and the sequence value of the last character of the corresponding core keyword according to the maximum and minimum values ​​of the annotation data.

[0025] In another preferred embodiment of the present invention, if the semantic aggregation of core keywords is taken into account, the method for determining the segmentation points of the sequence data includes: according to the annotated data of each core keyword, n adjacent reference core keyword groups are respectively screened out from the positions at both ends of the maximum and minimum extreme values ​​of the core keyword sequence value, and the number of annotated data between the two reference core keyword groups located at both ends of the extreme value is calculated from the outside to the inside, and the number of characters of the core keyword corresponding to each annotated data is calculated, and the total number of first characters of all core keywords corresponding to the annotated data is calculated; the total number of second characters of all sequence data between the two reference core keyword groups is calculated, and the total number of the first characters and the total number of the second characters are divided to obtain the character frequency value of the core keyword, and the initial core keyword and the end core keyword are screened and determined according to the character frequency value of the core keyword. For example: the order of core keywords can be: A1, A2, A3...B3, B2, B1, at this time A1 and B1 are the minimum and maximum values ​​of the core keyword annotated data corresponding to the sequence value data. Therefore, it is necessary to start searching from the reference core keyword group [A1, B1] from the outside to the inside. Further reference core keyword groups may be [A1, B2], [A2, B1] or [A2, B2], etc. Each search requires calculating the character frequency value of the corresponding reference core keyword group, wherein the character frequency value includes all the core keyword characters of the corresponding core keyword group that have appeared in the sequence data.

[0026] A character frequency value threshold is set in advance, and the character frequency values ​​of n reference core keyword groups are calculated from the outside to the inside in sequence. When the character frequency value of the core keyword of one group of reference core keyword groups is greater than the character frequency value threshold, the group of reference core keyword groups is the target core keyword group, and further calculation of the character frequency values ​​of the n reference core keyword groups is stopped, and the sequence data is segmented according to the position of the target core keyword group.

[0027] The segmentation method of the target core keyword group includes: obtaining the highest sequence value and the lowest sequence value of the characters in the target core keyword group, and segmenting according to the highest sequence value and the lowest sequence value of the characters in the target core keyword group, and configuring corresponding data permissions for each segmented sequence data to the corresponding user.

[0028] The embodiments disclosed in the present invention, the process described above with reference to the flowchart can be implemented as a computer software program. The embodiments disclosed in the present invention include a computer program product, which includes a computer program carried on a computer-readable medium, and the computer program contains a program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from the network through the communication part, and / or installed from a removable medium. When the computer program is executed by the central processing unit (CPU), the above-mentioned functions defined in the method of the present application are executed. It should be noted that the above-mentioned computer-readable medium of the present application can be a computer-readable signal medium or a computer-readable storage medium or any combination of the above two. The computer-readable storage medium can be, for example, but not limited to, a system, device or device of an electrical, magnetic, optical, electromagnetic, infrared segment, or semiconductor, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to, an electrical connection with one or more wire segments, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In the present application, a computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, apparatus, or device. In the present application, a computer-readable signal medium may include a data signal propagated in baseband or as part of a carrier wave, which carries a computer-readable program code. Such propagated data signals may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. A computer-readable signal medium may also be any computer-readable medium other than a computer-readable storage medium, which may send, propagate, or transmit a program for use by or in conjunction with an instruction execution system, apparatus, or device. The program code embodied on the computer readable medium may be transmitted using any appropriate medium, including but not limited to wireless, electrical wire, optical cable, RF, etc., or any suitable combination of the foregoing.

[0029] The flow chart and block diagram in the accompanying drawings illustrate the possible architecture, function and operation of the system, method and computer program product according to various embodiments of the present invention. In this regard, each square box in the flow chart or block diagram can represent a module, a program segment or a part of a code, and the module, the program segment or a part of the code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the square box can also occur in a sequence different from that marked in the accompanying drawings. For example, two square boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each square box in the block diagram and / or flow chart, and the combination of the square boxes in the block diagram and / or flow chart can be implemented with a dedicated hardware-based system that performs the specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0030] Those skilled in the art should understand that the embodiments of the present invention described above and shown in the accompanying drawings are only examples and do not limit the present invention. The objectives of the present invention have been fully and effectively achieved. The functional and structural principles of the present invention have been demonstrated and explained in the embodiments. Without departing from the principles, the implementation methods of the present invention may be deformed or modified in any way.

Claims

1. A data rights management method for data segmentation, characterized in that: The method comprises: Acquire raw data, clean the raw data to obtain clean data, and sort the clean data in the order of acquisition time to obtain sequence data; Preconfigure a core keyword set for segmentation, wherein each core keyword set represents a segmentation data type, and search for each corresponding core keyword in the sequence data according to the core keyword set; Obtaining the position and quantity of each core keyword in the sequence data that is matched in the sequence data, and screening and determining the initial position keyword and the final position keyword according to the position and quantity of each core keyword; The sequence data is segmented according to the sequence data positions where the initial position keyword and the end position keyword are located, and corresponding data permissions are configured for each segmented data sequence.

2. A data rights management method for data segmentation according to claim 1, characterized in that: The method for constructing the core keyword set includes: selecting keywords in advance according to user needs, analyzing keyword attribute characteristics, calculating the average cosine similarity value of all keyword attributes, setting a similarity difference threshold, calculating the similarity value between each keyword attribute and all other keywords, and taking the average value of the similarity values. If the difference between the average value of the similarity of the corresponding keyword attribute and the average cosine similarity value of all keyword attributes is greater than the preset similarity difference threshold, the keyword is eliminated, otherwise it is retained as a core keyword to construct the core keyword set.

3. A data rights management method for data segmentation according to claim 1, characterized in that: The raw data cleaning method comprises: converting the raw data into text data in a unified format, and sorting the raw data in the time sequence of obtaining the raw data to obtain the initial sequence number of each character in the raw data, and performing data cleaning in a manner including removing stop words, removing spaces and deleting repeated words, and sorting the cleaned raw data again in the time sequence of obtaining the raw data to obtain sequence data of each raw data character and a corresponding sequence value.

4. The data rights management method for data segmentation according to claim 1 is characterized in that: The method for screening and determining the keywords of the initial position keywords and the last position keywords includes: taking the sequence value of each core keyword in the core keyword set in the sequence data, and obtaining the core keywords with the highest sequence value and the lowest sequence value in the core keyword set, and further based on the core keywords with the highest sequence value and the lowest sequence value, taking the core keyword with the highest sequence value and the core keyword with the lowest sequence value as the basis for segmenting the sequence data, taking the starting character of the core keyword with the lowest core keyword sequence value as the starting segmentation cutting point of the sequence data, and taking the core keyword with the highest sequence value and the last character as the segmentation points of the last segment.

5. The data rights management method for data segmentation according to claim 1 is characterized in that: The method for determining the segmentation points of the sequence data includes: calculating the position of each core keyword in the core keyword set in the sequence data, and calculating the average value of all character sequences in which each core keyword appears in the sequence data, and using the average value of all character sequences as the annotation data of the corresponding core keyword, numerically sorting all the annotation data to obtain the maximum and minimum values ​​of the annotation data, and searching for the sequence value of the initial character and the sequence value of the last character of the corresponding core keyword according to the maximum and minimum values ​​of the annotation data.

6. A data rights management method for data segmentation according to claim 1, characterized in that: The method for determining the segmentation points of the sequence data includes: according to the annotated data of each core keyword, n adjacent reference core keyword groups are respectively selected from the two extreme positions of the maximum and minimum core keyword sequence values, and the number of annotated data between the two reference core keyword groups located at the two extreme positions is calculated from the outside to the inside, and the number of characters of the core keyword corresponding to each annotated data is calculated, and the total number of first characters of all core keywords corresponding to the annotated data is calculated; the total number of second characters of all sequence data between the two reference core keyword groups is calculated, and the total number of the first characters and the second characters are divided to obtain the character frequency value of the core keyword, and the initial core keyword and the ending core keyword are selected and determined according to the character frequency value of the core keyword.

7. A data rights management method for data segmentation according to claim 1, characterized in that: A character frequency value threshold is set in advance, and the character frequency values ​​of n reference core keyword groups are calculated from the outside to the inside in sequence. When the character frequency value of the core keyword of one group of reference core keyword groups is greater than the character frequency value threshold, the group of reference core keyword groups is the target core keyword group, and further calculation of the character frequency values ​​of the n reference core keyword groups is stopped, and the sequence data is segmented according to the position of the target core keyword group.

8. A data rights management method for data segmentation according to claim 7, characterized in that: The segmentation method of the target core keyword group includes: obtaining the highest sequence value and the lowest sequence value of the characters in the target core keyword group, and segmenting according to the highest sequence value and the lowest sequence value of the characters in the target core keyword group, and configuring corresponding data permissions for each segmented sequence data to the corresponding user.

9. A data rights management system for data segmentation, characterized in that: The system executes a data permission management method for data segments as described in any one of claims 1-8 above.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and the computer program is executed by a processor to implement a data permission management method for data segments as described in any one of claims 1-8.