A personnel file digitization automatic classification method and system based on vocabulary statistics

Abstract

The application relates to the technical field of file classification, and provides a personnel file digitization automatic classification method and system based on vocabulary statistics, which separates the title, the main text and the attribute area of a personnel file through identification, independently counts vocabularies to generate partition word frequency data, combines cross-file vocabulary weight layering and weighting fusion to form an initial vector set, and triggers an adaptive clustering mechanism of the target set to compare the similarity between new vectors and existing vectors, to trigger boundary expansion and competitive feedback driven vector updating controlled by an alarm parameter; if the similarity is continuously too low, a new category and a clustering vector are established, and finally, the classification result is composed of the updated clustering vector and the category. The application can automatically and adaptively classify personnel files based on vocabulary statistics, and through dynamic clustering boundary adjustment and a new category establishment mechanism, the problem of insufficient incremental processing efficiency in the prior art is solved, and efficient processing of incremental files and continuous optimization of a classification system are realized.

Description

TECHNICAL FIELD

[0001] The application relates to the technical field of file classification, and in particular to a personnel file digital automatic classification method and system based on vocabulary statistics. BACKGROUND

[0002] With the refinement of personnel management of an organization and the acceleration of the process of file digitization, how to efficiently and accurately automatically classify and archive massive, various types and continuously growing personnel files (such as employee resumes, assessment forms, history forms, etc.) becomes a key requirement for improving the efficiency of human resource management and the ability of knowledge mining. Specifically, there is an urgent need for a digital classification technical solution that can automatically understand the content of the files, adapt to the dynamic inflow of new data, and maintain the stability of the classification system.

[0003] Currently, for personnel file text classification, there is an existing solution based on keyword feature extraction and incremental clustering. The solution first extracts keywords from the entire file document using methods such as TF-IDF, assigns weights to the keywords to reflect their discriminability in the global file library, and forms a document feature vector. Then, when a new file is added, the system converts it into a feature vector and calculates the similarity with the existing archive clustering center (usually generated by early batch clustering), and decides whether to assign it to an existing category or trigger a small-scale re-clustering update according to the preset similarity threshold.

[0004] However, such existing solutions still have obvious defects in actual application: first, the internal structure information of the file is ignored, and the title, attribute value and other key identification areas are treated the same as ordinary text, which fails to effectively utilize the key indication of high-weight keywords in the title area and structured fields in the attribute area, weakening the accuracy of classification. Second, the incremental processing efficiency is insufficient, especially when facing a large number of new files, there is a lack of priority processing mechanism for high-dimensional or core feature vectors, which is prone to form a processing bottleneck during peak periods. Third, the dynamic adaptability of the clustering boundary is poor, relying on a fixed similarity threshold (alert parameter) to determine the classification and new category, which is difficult to effectively cope with the natural "concept drift" that may occur within the file category with incremental data, leading to lag or classification deviation of the model for new forms and new meaning files. SUMMARY

[0005] The application provides a personnel file digital automatic classification method and system based on vocabulary statistics to solve the problem of insufficient incremental processing efficiency in the prior art.

[0006] In a first aspect, the application provides a personnel file digital automatic classification method based on vocabulary statistics, comprising:

[0007] Obtaining original archive data of personnel archives, identifying a structured area boundary in the original archive data, separating a title area, a text area, and an attribute label area, and independently performing multi-dimensional word counting on each type of area to generate partition word frequency data;

[0008] Performing hierarchical weighted fusion on the partition word frequency data and cross-archive word weight quantization data reflecting the contribution of the distinctness of specific words in the entire archive set to form an initial archive vector set;

[0009] Monitoring the peak state of the newly added archive data flow, when the number of continuously added archives exceeds the preset parallel processing threshold, controlling the high-dimensional archive vectors in the initial archive vector set to enter the clustering process preferentially to generate a target archive vector set;

[0010] Inputting the target archive vector set into an adaptive clustering mechanism, obtaining the similarity relationship between the newly added archive vector and the existing clustering vector by comparing them, triggering a clustering boundary expansion operation controlled by an alarm parameter based on the similarity relationship, and synchronously driving the iterative update of the existing clustering vector based on a competitive feedback mechanism;

[0011] When the similarity relationship continuously falls below the threshold set by the alarm parameter, classifying the newly added archive vector into a new archive category and establishing a new clustering vector, the new clustering vector is added to the existing clustering vector set, and the digital classification result of the personnel archives is constructed through the existing clustering vector set and the corresponding archive categories.

[0012] Optionally, triggering a clustering boundary expansion operation controlled by an alarm parameter based on the similarity relationship, and synchronously driving the iterative update of the existing clustering vector based on a competitive feedback mechanism, includes:

[0013] Determining whether the similarity relationship is greater than or equal to a preset alarm parameter value, when the similarity relationship is greater than or equal to, triggering a clustering boundary expansion operation, adding the newly added archive vector to the existing archive vector, and replacing the original clustering vector with the average value of the clustering vector corresponding to the newly added archive vector and the existing clustering vector as a new clustering vector;

[0014] When the similarity relationship is less than, triggering a clustering boundary contraction operation, calculating the competitive weight based on the corresponding positions of the elements of the newly added archive vector and the existing archive vector, and replacing the original clustering vector with the weight proportion average value of the clustering vector corresponding to the newly added archive vector and the existing clustering vector as a new clustering vector;

[0015] The expansion or contraction operation of the clustering boundary drives the recalculation of all existing clustering vectors to complete the iterative update.

[0016] Optionally, inputting the target profile vector set into an adaptive clustering mechanism, obtaining similarity relations between the new profile vector and existing cluster vectors by comparing them, including:

[0017] Calculating the sum of absolute values of element corresponding position differences between the new profile vector and existing cluster vectors corresponding to each of the existing profile vectors as a preliminary difference value;

[0018] Based on the preliminary difference value, the number of elements of the new profile vector and the existing cluster vector, generating a normalized distance value according to a pre-designed calculation rule;

[0019] Converting the normalized distance value into a corresponding similarity relation, wherein when the new profile vector is input for the first time, the similarity relation is set to 0.

[0020] Optionally, when the similarity relation continuously falls below the threshold value set by the alert parameter, classifying the new profile vector into a new profile category and establishing a new cluster vector, the new cluster vector being added to the existing cluster vector set, including:

[0021] Setting a counter with an initial value of 0 for each similarity relation obtained by comparison, and increasing the counter value each time the similarity relation is less than the alert parameter;

[0022] When the counter value reaches a preset value within a preset time window and an expansion operation is not triggered, it is determined that the similarity relation continuously falls below the threshold value;

[0023] When the similarity relation continuously falls below the threshold value set by the alert parameter, assigning a new profile category number to the new profile vector, and using the new profile vector as a new cluster vector;

[0024] Adding the new cluster vector to the existing cluster vector set and updating the profile category record.

[0025] Optionally, hierarchically weighting and fusing the partition word frequency data and cross-profile word weight quantization data reflecting the contribution of the distinctness of specific words in the overall profile set to form an initial profile vector set, including:

[0026] Based on the partition word frequency data of the overall profile set, calculating the frequency difference value of each word as a distinctness value;

[0027] Generating region-specific cross-profile word weight data for the title area, the text area, and the attribute annotation area, the weight value being determined by the distinctness value;

[0028] The title area word distribution, the text area word distribution and the attribute label area word distribution of each archive are sequentially multiplied by the region-specific cross-archive word weight data;

[0029] The title area result, the text area result and the attribute label area result after the multiplication operation are added to form an initial archive vector of each archive, and the initial archive vectors of all archives are combined to form an initial archive vector set.

[0030] Optionally, the peak state of the newly added archive data flow is monitored, and when the number of continuously added archives exceeds a preset parallel processing threshold, the high-dimensional archive vector in the initial archive vector set is controlled to enter the clustering process first to generate a target archive vector set, comprising:

[0031] The number of newly added archive data arrivals and the interval time are recorded, and when the number of continuously added archives reaches the preset parallel processing threshold, the priority judgment logic is triggered;

[0032] The dimension number of each vector in the initial archive vector is calculated by the priority judgment logic, and the archive vector with the largest dimension number is marked as a high-dimensional archive vector;

[0033] The high-dimensional archive vector is controlled to enter the clustering process first to perform subsequent operations, and the non-high-dimensional archive vector waits for processing;

[0034] The clustering process outputs the high-dimensional archive vector and other archive vectors, which are combined into the target archive vector set.

[0035] Optionally, the structured region boundaries in the original archive data are identified, and the title area, the text area and the attribute label area are separated, and multi-dimensional word statistical processing is independently performed on each type of area to generate partition word frequency data, comprising:

[0036] The features including the starting position of each text line, the interval distance from other lines and the text concentration in the original archive data are detected;

[0037] The features are compared with the preset boundary judgment rule to identify and separate the title area boundary, the attribute label area boundary and the text area boundary;

[0038] The text in the title area, the text area and the attribute label area is sequentially subjected to word splitting operation, and the occurrence number and the character length of each word are independently calculated;

[0039] The occurrence number and the character length of each word are combined to form the title area word distribution, the text area word distribution and the attribute label area word distribution as partition word frequency data.

[0040] In a second aspect, the present application provides a personnel file digitalization automatic classification system based on vocabulary statistics, comprising:

[0041] A generation module is configured to acquire original file data of personnel files, identify boundaries of structured areas in the original file data, separate title areas, text areas and attribute label areas, and perform multi-dimensional vocabulary statistical processing on each type of area to generate partition word frequency data;

[0042] A fusion module is configured to perform hierarchical weighted fusion of the partition word frequency data and cross-file vocabulary weight quantitative data reflecting the contribution of the distinctness of specific vocabulary in the whole file set to form an initial file vector set;

[0043] A monitoring module is configured to monitor the peak state of newly added file data flow, and when the number of continuously added files exceeds a preset parallel processing threshold, control high-dimensional file vectors in the initial file vector set to enter a clustering process preferentially to generate a target file vector set;

[0044] A driving module is configured to input the target file vector set into an adaptive clustering mechanism, compare the similarity relationship between a newly added file vector and an existing clustering vector, trigger a clustering boundary expansion and contraction operation controlled by an alert parameter based on the similarity relationship, and synchronously drive iterative updating of the existing clustering vector based on a competitive feedback mechanism;

[0045] A classification module is configured to, when the similarity relationship continuously falls below a threshold set by the alert parameter, classify the newly added file vector into a new file category and establish a new clustering vector, add the new clustering vector to the existing clustering vector set, and construct a digital classification result of personnel files through the existing clustering vector set and corresponding file categories.

[0046] In a third aspect, the present application provides a computing device comprising a processing component and a storage component; the storage component stores one or more computer instructions; the one or more computer instructions are used to be called and executed by the processing component to implement a personnel file digitalization automatic classification method based on vocabulary statistics as described in the first aspect above.

[0047] In a fourth aspect, the present application provides a computer storage medium storing a computer program, wherein the computer program is executed by a computer to implement a personnel file digitalization automatic classification method based on vocabulary statistics as described in the first aspect.

[0048] The application can accurately utilize the difference contribution of different structure regions, significantly improve the feature extraction pertinence and accuracy, and solve the problem of ignoring structure information by identifying and separating the title, body and attribute annotation area in the personnel file original data, and independently performing multi-dimensional word statistics to generate partition word frequency data for each type of area. Then, the initial file vector set is formed by hierarchically weighting and fusing the partition word frequency data and the cross-file word weight reflecting the overall word discrimination contribution, which can comprehensively integrate local and global features of the file to generate a high-discrimination feature vector. Further, by monitoring the peak value of the newly added file flow and preferentially processing high-dimensional file vectors to generate a target vector set when the threshold is exceeded, the resource allocation during the flood peak period can be optimized, the system throughput efficiency can be improved, and the incremental processing bottleneck can be solved. Then, by inputting the target vector set into the adaptive clustering mechanism, the clustering boundary expansion or contraction operation is triggered based on the similarity between the new and existing clustering vectors, and the clustering vector iterative update based on the competitive feedback mechanism is simultaneously driven, which can dynamically evaluate the matching degree and adaptively adjust the boundary and center representation, realize model self-learning and dynamic evolution. Finally, by establishing a new file category and a new clustering vector when the similarity is continuously below the warning threshold, the existing set is constituted to form the result, which can automatically identify new type files, dynamically expand the classification system, and effectively cope with the evolution of file content and concept drift.

[0049] Further, when the similarity between the new vector and the existing clustering vector is greater than or equal to the warning parameter value, the clustering boundary expansion operation is performed (the new vector is added to the existing category, and the average value of all vectors of the category including the new one is updated as the new clustering vector); when the similarity is less than the warning parameter value, the clustering boundary contraction operation is performed (the competitive weight of the new vector and the existing category vector is calculated, and the weight proportion average value of the new and old clustering vectors is calculated as the new clustering vector to replace the original clustering vector). Regardless of the expansion or contraction operation, the recalculation of all existing clustering vectors is triggered to complete the iterative update. Through the specific mechanism of the above boundary expansion and contraction (expansion and contraction) and the clustering vector update rule based on the competitive weight, the differential and accurate dynamic adjustment of the clustering boundary (intra-class expansion or inter-class isolation) and the optimization of the clustering center expression can be performed based on the similarity judgment, especially the competitive weight mechanism in the contraction operation can more finely depict the influence or discrimination of the new sample on the original category, avoiding rough division, so that the adaptive learning and update of the classification model can be more accurately and smoothly realized while maintaining the overall stability of the classification system, and finally the dynamic adaptation ability and robustness of the classification model to new data and potential category changes are greatly enhanced.

[0050] These aspects or other aspects of the application will be more apparent in the following description of the embodiments. BRIEF DESCRIPTION OF DRAWINGS

[0051] In order to make the technical scheme of the present application or the prior art clearer, the accompanying drawings needed in the description of the embodiments or the prior art will be briefly introduced. Obviously, the accompanying drawings in the following description are some embodiments of the present application, and other drawings can be obtained by those skilled in the art without any creative effort.

[0052] Figure 1 A flow chart of a personnel file digitization automatic classification method based on vocabulary statistics provided by the present application is shown;

[0053] Figure 2 A scene schematic diagram of a personnel file digitization automatic classification method based on vocabulary statistics provided by the present application is shown;

[0054] Figure 3 A structure schematic diagram of a personnel file digitization automatic classification system based on vocabulary statistics provided by the present application is shown;

[0055] Figure 4 A structure schematic diagram of a computing device provided by the present application is shown. DETAILED DESCRIPTION

[0056] In order to make the technical scheme of the present application or the prior art clearer, the accompanying drawings needed in the description of the embodiments or the prior art will be briefly introduced. Obviously, the accompanying drawings in the following description are some embodiments of the present application, and other drawings can be obtained by those skilled in the art without any creative effort.

[0057] In some of the processes described in the specification and claims of the present application and the above-mentioned accompanying drawings, a plurality of operations appearing in a specific order are included, but it should be clearly understood that these operations can be executed or in parallel without the order appearing in the text, and the serial numbers of the operations, such as 101, 102, etc., are only used to distinguish different operations, and the serial numbers themselves do not represent any execution order. In addition, these processes can include more or fewer operations, and these operations can be executed in sequence or in parallel. It should be noted that the "first", "second", etc. in the text are used to distinguish different messages, devices, modules, etc., and do not represent the order, nor do "first" and "second" represent different types.

[0058] In the field of digitized automatic classification of personnel files, the existing scheme based on keyword feature extraction and incremental clustering still faces significant challenges: first, the utilization of structural information is missing, as the high-weight indicative words in the file title area, the structured fields in the attribute area, and the ordinary text are treated together, which dilutes the key classification features and significantly affects the classification accuracy; second, the incremental processing efficiency is insufficient, especially when a large number of new files suddenly flow in, the lack of priority scheduling mechanism for core high-dimensional feature vectors easily forms a processing bottleneck, reducing the system response speed and overall throughput capacity; third, the clustering dynamic adaptability is poor, relying on fixed similarity threshold (alert parameter) for hard division, which is difficult to effectively perceive and adapt to the "concept drift" that may occur in the connotation of new data, leading to lag or misjudgment in identifying new or semantically evolving files, limiting the generalization ability and timeliness of the classification model.

[0059] To solve the above problems, the present application proposes a method for digitized automatic classification of personnel files based on word statistics and fusion of structure perception and dynamic adaptive mechanism. The core of this scheme is: first, innovatively identify and separate the title area, text area and attribute annotation area in the original data of the file, perform multi-dimensional word statistics on each area to generate partition word frequency data, fully exploit the differential value of different structural areas for classification decision, and solve the problem of missing structural information utilization to improve accuracy. Second, by weighting and fusing the partition word frequency data and the cross-file weight reflecting the global discriminability of words at different levels, an initial file vector set with high representation is constructed. Further, the new data flow is monitored, and when the peak value exceeds the threshold, the high-dimensional file vector is processed preferentially to optimize resource allocation and break through the efficiency bottleneck. Key is to introduce an adaptive clustering mechanism based on alert parameter control: by calculating the similarity relationship between the new vector and the existing clustering vector, dynamically trigger the clustering boundary stretching operation (expansion or contraction) and drive the clustering vector iteration update based on competitive feedback; when the similarity is continuously lower than the alert threshold, a new file category and clustering vector are automatically established. This mechanism discards the traditional fixed threshold judgment mode, through the boundary elastic adjustment of the alert parameter and the vector self-update driven by competitive feedback, responds to the changes in the characteristics of new data and smoothly adapts to the evolution of category concepts (i.e. "concept drift"). This method not only ensures the structured perception accuracy and peak processing efficiency of the classification system, but also gives the model strong online self-learning ability and continuous dynamic optimization characteristics, fundamentally solving the three defects of existing technology in structure utilization, incremental efficiency and dynamic adaptability, providing an automatic classification solution with high precision, high efficiency and continuous evolution ability for personnel file management.

[0060] With reference to the accompanying drawings, the technical solutions in the embodiments of the present application will be clearly and completely described below, obviously, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative work are within the scope of protection of the present application.

[0061] Figure 1 A flow chart of a personnel file digitization automatic classification method based on vocabulary statistics is provided for the embodiments of the present application, as shown in the figure, the method comprises: Figure 1

[0062] 101, original file data of personnel files, identifying the structured area boundary in the original file data, separating the title area, the text area and the attribute annotation area, and independently performing multi-dimensional vocabulary statistics processing on each type of area to generate partition word frequency data;

[0063] Optionally, step 101 can specifically include the following steps:

[0064] 1011, detecting the features of the starting position of each text line, the spacing distance from other lines and the text concentration in the original file data;

[0065] 1012, comparing the features with the preset boundary judgment rules to identify and separate the title area boundary, the attribute annotation area boundary and the text area boundary;

[0066] 1013, performing vocabulary splitting operation on the text in the title area, the text area and the attribute annotation area in turn, and independently calculating the occurrence frequency and character length of each vocabulary;

[0067] 1014, combining the occurrence frequency and character length of each vocabulary to form the title area vocabulary distribution, the text area vocabulary distribution and the attribute annotation area vocabulary distribution as partition word frequency data.

[0068] In the above scheme, the structured area boundary refers to the starting and ending positions of different content type areas in the personnel file document. For example, the title area at the top of the document, the attribute annotation area containing labels such as name and department, and the text area of the main content. The partition word frequency data refers to three independent statistical data obtained by respectively counting the occurrence frequency and average character length of all words in the title area, the text area and the attribute annotation area. The starting position of the text line refers to the coordinate position of the beginning of each line of text in the document. The spacing distance from other lines refers to the size of the blank gap between adjacent two lines of text in the vertical direction. The text concentration refers to the closeness of the arrangement of text lines in a small range of the document, for example, a small window composed of several lines. The area with more and denser lines of text has high concentration, and the area with fewer and sparser lines of text has low concentration.​

[0069] Firstly, the embodiment of the application scans the whole original file by 1011: using document layout analysis technology. Analyzing each text line in the document, three core features are extracted: the first is the starting position of the text line, i.e. the coordinate position of the top-left corner of the line in the document, which is used to determine its absolute position in the page; the second is the interval distance from other lines, i.e. the blank gap size in the vertical direction between the line and the immediately adjacent upper text line, which is used to judge the degree of regional isolation; the third is the text concentration, i.e. the total number of other text lines existing within a fixed height region centered on the line, which is used to measure the density of the text in the region. For example, in a personnel file of organization A, it is found by analysis that the position of the first line of text, the personnel D resume, is very close to the top of the page, i.e. the starting position Y value is very small, and there is no other line above it or the interval above it is huge, i.e. the interval distance value is very high, and there are almost no other text lines within the region set around the line, i.e. the concentration value is very low, which strongly suggests that it may be a title.

[0070] Secondly, the feature values calculated by step 1011 for each line, including the starting position Y value, the interval distance value, and the concentration value, are compared one by one with the system's pre-set multiple boundary judgment rules. The rules predefine that the title area usually has a very low starting position Y value, close to the top of the page, a large average interval distance value, with obvious large blank between lines, and a very low concentration value, with few lines around; the attribute annotation area usually has a stable starting position X value, such as left alignment, a medium and uniform interval distance value, with stable line spacing, and a medium concentration value; the main text area usually has a higher starting position Y value, located in the middle or lower part of the page, a smaller interval distance value, with close line spacing, and a higher concentration value, with dense text lines. The system automatically marks the continuous line block that meets the title rule as the title area, marks the continuous line block that meets the attribute rule as the attribute annotation area, and marks the remaining line block that meets the main text rule as the main text area, thereby accurately identifying the boundaries of the three areas. For example, in the file of organization A, the content of the 3rd to 10th lines, such as identifier E value F, shows that their starting position X value is very stable, all within a certain fixed X range on the left, the interval distance value is small and does not change much, and the concentration value is medium, with several lines of content clustered, which are highly matched with the pre-set attribute annotation area rule. The system marks this part as the attribute annotation area. Similarly, the 11th line and subsequent parts show that the position is in the main body of the page, the distance is small and consistent, and the concentration is high, so it is determined as the main text area. The 1st line has been determined as the title area.

[0071] Next, by 1013: the three independent region title region attribute annotation region text region identified in step 1012 are processed in turn. In each region, use a word segmentation tool such as Chinese jieba or English morphological analyzer to split all continuous text content in the region into a list of independent meaningful words to complete the word splitting operation. After splitting, each unique word in the region is traversed to perform two statistics: first, calculate the total number of occurrences of the word in the current region, called the occurrence frequency, which reflects the importance of the word in the region; Second, calculate the number of characters contained in the word string itself, called the character length. This statistics is independent for each region. For example, in the title region of the agency A file, only contains the text Personnel D resume, after splitting, two words Personnel D and resume are obtained. The occurrence frequency of word Personnel D in the title region is 1 and the character length is 4, assuming that Personnel D consists of two Chinese characters and one letter; The occurrence frequency of word resume is 1 and the character length is 3. In the attribute annotation region such as identifier G value H row, the word obtained after splitting may be value H depending on the word segmentation rules. The occurrence frequency of the word in the attribute region is 1 and the character length is 5, assuming.

[0072] Finally, by 1014: collect all the statistical results of the words in each independent region completed in step 1013, that is, the occurrence frequency and character length of each word in the region. Then create a dedicated data set for the title region containing all the words appearing in the region and their corresponding occurrence frequency and length information. This set is called the title region word distribution. In the same way, create an attribute annotation region word distribution for the attribute annotation region and a text region word distribution for the text region. These three distribution data are independent of each other and are combined together to form the output partition word frequency data of step 101. For example, in the agency A file processing, the title region word data Personnel D with occurrence frequency 1 and character length 4, and resume with occurrence frequency 1 and character length 3 are combined into the title region word distribution. The attribute annotation region data value H with occurrence frequency 1 and character length 5 and other attribute region words are combined into the attribute annotation region word distribution. A large number of word statistical data in the text region are combined into the text region word distribution. These three distributions, i.e. the partition word frequency data of the file, clearly show the use characteristics of words in different regions.

[0073] In practical application, an electronic personnel resume of an employee of an organization A is analyzed. Firstly, in step 1011, the system analyzes each line of information in the document and finds that the first line of text of the resume of the employee D is near the top of the page, but there is a large blank space between the first line and the second line, and there are few lines of text near the first line. The third to tenth lines, such as identifier E: value F, identifier G: value H, and identifier I: value J, have approximately aligned starting positions on the left side, and the line spacing is small and stable. The eleventh line and subsequent lines of text are in the main body area of the page, and the line spacing is small and uniform, and the lines of text are dense. Secondly, in step 1012, according to the rules, it is determined that the first line is in the title area, the third to tenth lines are in the attribute annotation area, and the eleventh line and subsequent lines are in the text area. Thirdly, in step 1013, the content of the title area is split to obtain the words “employee D” and “resume”, and the frequency and length of the characters in the title area are counted. The content of the attribute area, such as “identifier G: value H”, is split, and the value H in the attribute area appears once with a length of 5. The content of the text area, such as “responsible for K project L stage content management”, is split, and the frequency and length of each word in the text area are counted. Finally, in step 1014, the data in the title area, the attribute area, and the text area are combined to form the word frequency data in the title area, the word frequency data in the attribute annotation area, and the word frequency data in the text area, which together constitute the partitioned word frequency data.

[0074] The present scheme can effectively distinguish the roles and importance of words in different types of areas, such as title keywords, attribute values, and text descriptions, by accurately identifying the boundaries of the title, attribute, and text areas in the document and independently counting the frequency and average length of words in each area. This fine-grained statistical method provides more rich and focused basic data for subsequent steps, significantly improves the ability of the system to identify subtle features of the document content, avoids missing or confusing key information, and lays a solid foundation for accurate classification.

[0075] 102. hierarchically weighting and fusing the partitioned word frequency data with cross-document word weight quantitative data reflecting the contribution of the distinctness of specific words in the overall document collection to form an initial document vector set;

[0076] Optionally, step 102 can specifically include the following steps:

[0077] 1021. calculating the frequency difference value of each word as the distinctness value based on the partitioned word frequency data of the overall document collection;

[0078] 1022. generating region-specific cross-document word weight data for the title area, the text area, and the attribute annotation area, and the weight value is determined by the distinctness value;

[0079] 1023. For each file, the word distribution in the title area, the word distribution in the body area, and the word distribution in the attribute annotation area are sequentially multiplied by the specific cross-file word weight data of the respective area.

[0080] 1024. Add the title area result, body area result, and attribute annotation area result after the dot product operation to form the initial file vector of each file, and merge the initial file vectors of all files to form the initial file vector set.

[0081] In the above scheme, the partitioned word frequency data refers to the word distribution in the title region, the word distribution in the body region, and the word distribution in the attribute annotation region output in step 101. The word distribution in the title region includes all words in that region, their frequency of occurrence, and their character length. Cross-file word weight quantification data refers to a pre-calculated dataset. It reflects the contribution of specific words in the entire set of personnel files across the organization to distinguishing different types of files. This contribution, called discriminative power, is converted into a numerical weight. Words with high discriminative power, such as professional titles or senior engineers, are assigned higher weights. Hierarchical weighted fusion refers to the operation being completed in two steps. The first step, hierarchical fusion, involves independently calculating the weights for the title region, body region, and attribute region using their respective weights. The second step, fusion, combines the calculation results from the three regions. The initial file vector set refers to the set of numerical vectors representing the final result of all files. Each file is converted into an independent numerical string called the initial file vector; the set of all vectors is the initial file vector set.

[0082] In this embodiment, firstly, through 1021: the system calculates based on the partitioned word frequency data generated in step 101 for all personnel files, namely, the title and body attribute word frequency data for all files. For each word in the vocabulary, the system calculates the degree of difference in the number of times the word appears in the title area of ​​all files, the degree of difference in the number of times it appears in the body area, and the degree of difference in the number of times it appears in the attribute annotation area. The numerical value that measures this difference is called the discrimination value. The higher the discrimination value, the greater the difference in the appearance of the word in different categories of files and the stronger the discrimination ability. For example, the word "engineer" appears frequently in technical job category files but rarely in administrative job category files, so its discrimination value is high. The word "employee" appears generally frequently in all categories, so its discrimination value is very low.

[0083] Second, by 1022: the system generates three sets of region-specific cross-profile lexical weight data for the title region, the body region, and the attribute label region respectively, independently, using the lexical discriminativeness value of each word calculated in step 1021. For each word in each region, the system uses its discriminativeness value in that region as the weight value directly or after adjustment by a mapping function. This means that the same word can have different weight values in the title region, the body region, and the attribute label region. For example, the word "senior engineer" can have a weight of 0.8 in the title region because its title is indicative, a weight of 0.7 in the attribute label region because its title is indicative, and a weight of 0.5 in the body region because its title is descriptive. The word "name" can have a weight of 0.3 in the attribute label region but a weight close to 0 in the title or body region.

[0084] Next, by 1023: for each specific personnel profile, such as profile X, the system takes the title region lexical distribution, the body region lexical distribution, and the attribute label region lexical distribution of the profile, i.e., the sub-part of the profile's zoned word frequency data. Then the system operates on each region independently. Taking the title region as an example, the system performs a dot product operation between the title region lexical distribution of profile X, i.e., a vector containing the local word frequency of each word, and the region-specific weight vector generated for the title region in step 1022. The dot product operation is to multiply the two values at the corresponding positions and then add all the products to get a scalar result representing the weighted value of the region. The same operation is performed for the body region and the attribute label region. For example, the title region lexical distribution of profile X shows that the word "senior engineer" appears once and its weight in the title region is 0.8 in the weight data. The dot product contribution is 1 times 0.8, which equals 0.8. The system adds up all the dot product results to get the dot product result for the title region.

[0085] Finally, by 1024: the system adds up the three region dot product results calculated for profile X in step 1023, i.e., the title region result, the body region result, and the attribute label region result, to get the initial profile vector VX representing profile X. The vector is usually a scalar or a low-dimensional vector. The system repeats steps 1023 and 1024 for all the profiles in the organization to collect the corresponding vectors of each profile to form an initial profile vector set. For example, the dot product results of profile X are 0.8 for the title region, 0.5 for the body region, and 0.7 for the attribute label region. Adding them up, VX = 0.8 + 0.5 + 0.7 = 2.0. The dot product results of profile Y are 1.5. The final initial profile vector set contains profile X 2.0, profile Y 1.5, etc.

[0086] In practical applications, in the personnel file digitization system of institution B. 1021: analyze all file word frequencies to calculate the difference in the number of occurrences of word identifier C in the attribute area, and discover that department M files rarely appear and department N files frequently appear, so the discrimination value of identifier C in the attribute area is high. 1022: generate area weight attribute area identifier C weight 0.9 common word name weight 0.1 title area responsible person D weight 0.7 common word document weight 0.2 text area responsible project E weight 0.6. 1023: process file F, and the attribute area identifier C appears once, and the point multiplication attribute weight is 1 multiplied by 0.9, which is 0.9. The title area responsible person D appears once, and the point multiplication attribute weight is 1 multiplied by 0.7, which is 0.7. The text area point multiplication is 0.5. 1024: add the results of the three areas, 0.9 plus 0.7 plus 0.5, to get the initial file vector VF of file F equal to 2.1. The final vector set contains file F 2.1 and file G 1.8.

[0087] The scheme adopts a hierarchical weighted fusion strategy to calculate the global discrimination weight of the title and text attribute area respectively, and then merge the results. This method fully respects the contribution difference of different areas of words to classification, such as professional words appearing in the title or attribute area being more decisive than the same words in the text. The generated file vector not only contains the characteristics of the file itself, but also integrates the global word discrimination information, significantly enhancing the vector representation ability and semantic discrimination, laying a solid foundation for subsequent accurate classification.

[0088] 103、Monitor the peak state of the newly added file data flow, and when the number of continuously added files exceeds the preset parallel processing threshold, control the high-dimensional file vector in the initial file vector set to enter the clustering process first to generate a target file vector set;

[0089] Optionally, step 103 can specifically include the following steps:

[0090] 1031, record the number of newly added file data arrivals and the arrival interval time, and trigger the priority judgment logic when the number of continuously added files reaches the preset parallel processing threshold;

[0091] 1032, calculate the dimension number of each vector in the initial file vector through the priority judgment logic, and mark the file vector with the largest dimension number as a high-dimensional file vector;

[0092] 1033, control the high-dimensional file vector to enter the clustering process first to perform subsequent operations, and the non-high-dimensional file vector waits for processing;

[0093] 1034, output the high-dimensional file vector and other file vectors after the clustering process, and combine them into the target file vector set.

[0094] In the above scheme, the newly added archive data traffic peak state refers to the sudden significant increase of the influx of newly added archives per unit time. The parallel processing threshold is a quantity threshold preset by the system, which is used to determine whether to enter the traffic peak. The high-dimensional archive vector refers to the vector with the most number of features in the initial archive vector set, and the number of dimensions is obtained by calculating the number of effective feature items contained in the vector. The clustering process is a calculation process of automatically classifying similar archives. The target archive vector set is the final vector set after all archives are processed.

[0095] In the embodiment of the present application, first, the system continuously monitors the arrival of newly added archives through 1031, records the arrival time and cumulative number of each newly added archive. When the number of continuously uninterrupted newly added archives reaches a preset value, for example, 500, the priority judgment module is triggered. For example, the preset threshold in the system of institution C is 500, and when the 501st newly added archive is continuously added, the scheduling logic is triggered. Second, the priority module scans the initial archive vector set to be processed through 1032, and calculates the dimension number of each vector, that is, the number of effective data items such as keyword weights contained in the vector. All vectors are sorted by dimension number, and the top several vectors with dimension number greater than 100 are marked as high-dimensional archive vectors. For example, a vector containing 120 features is marked as high-dimensional, and a vector containing only 80 features is not marked. Then, the system schedules resources to make the marked high-dimensional archive vectors enter the clustering calculation process preferentially through 1033. Other unmarked vectors are suspended in the waiting queue, and then the non-high-dimensional vectors are processed in turn after the high-dimensional vectors are processed. For example, 10 high-dimensional research and development vectors are preferentially processed, and 50 regular-dimensional administrative position vectors are temporarily stored in the waiting queue. Finally, the clustering calculation of all high-dimensional vectors to be preferentially processed is completed, and the system automatically restarts the queuing mechanism to process the remaining non-high-dimensional vectors through 1034. Finally, all processed high-dimensional vectors and non-high-dimensional vectors are merged to form a target archive vector set. For example, the 120-dimensional research and development vectors processed preferentially are merged with the 80-dimensional administrative vectors processed non-preferentially to output.

[0096] In practical application, the archive system of institution C continuously receives 500 newly added archives in the afternoon, reaches the preset threshold to trigger scheduling. The system calculates the vectors to be processed: the employee D resume vector contains 500 job title / project keyword dimensions and is marked as high-dimensional with the highest dimension, and the employee E resume contains only 80 ordinary fields. The high-dimensional employee D resume is preferentially processed in the classification process, and the employee E resume is waiting. After processing, all output vectors are merged to form the classification result data set of the day.

[0097] The present scheme monitors the traffic flood peak and preferentially schedules the processing of high-dimensional archive vectors. This method can automatically focus on processing the longest core archives during periods of tight system resources, significantly shorten the overall processing period, effectively solve the calculation bottleneck problem when the incremental data set is flooded, and ensure the stability of the system processing efficiency and response speed.

[0098] 104. Input the target file vector set into the adaptive clustering mechanism, obtain the similarity relationship between the newly added file vector and the existing cluster vector by comparing the new file vector and the existing cluster vector, trigger the cluster boundary scaling operation controlled by the warning parameter based on the similarity relationship, and synchronously drive the iterative update of the existing cluster vector based on the competitive feedback mechanism.

[0099] Optionally, step 104 may specifically include the following steps:

[0100] 1041. Calculate the sum of the absolute values ​​of the differences between the corresponding positions of the elements of the newly added file vector and the existing cluster vector corresponding to each existing file vector as the preliminary difference value;

[0101] 1042. Based on the preliminary difference value, the number of elements in the newly added file vector and the existing cluster vector, a normalized distance value is generated according to a preset calculation rule;

[0102] 1043. Convert the normalized distance value into the corresponding similarity relationship, wherein when the newly added file vector is input for the first time, the similarity relationship is set to 0.

[0103] 1044. Determine whether the similarity relationship is greater than or equal to a preset warning parameter value. When the similarity relationship is greater than or equal to, trigger the cluster boundary expansion operation, add the newly added file vector to the current existing file vector, and use the average value of the cluster vector corresponding to the newly added file vector and the existing cluster vector as the new cluster vector to replace the original cluster vector.

[0104] 1045. When the similarity relationship is less than 1045, a cluster boundary shrinkage operation is triggered. The competitive feedback weight is calculated based on the corresponding positions of the elements of the newly added file vector and the existing file vector. The average weight ratio of the existing cluster vector and the cluster vector corresponding to the newly added file vector is used as the new cluster vector to replace the original cluster vector.

[0105] 1046. The expansion or contraction operation of the cluster boundary drives the recalculation of all existing cluster vectors to complete the iterative update.

[0106] In the above scheme, the adaptive clustering mechanism refers to a dynamic classification algorithm that can automatically adjust the classification rules. Existing cluster vectors are the center representative vectors of already classified file groups. The similarity relationship is the numerical value representing the degree of matching between new files and existing categories. The warning parameter is a preset similarity threshold that controls the inclusiveness of the categories. Cluster boundary scaling refers to the dynamic expansion of the existing category range or the shrinking to isolate new categories. The competitive feedback weight is an adjustment coefficient generated based on the degree of difference when calculating the fusion ratio of old and new vectors. Iterative update refers to recalculating the center vectors of all categories to ensure the model's timeliness.

[0107] The embodiment of the application first calculates the absolute value sum of the element corresponding position difference between the new archive vector X and each existing cluster vector C, to obtain a preliminary difference value Δ, wherein the calculation formula is: For example, the R&D post cluster vector C = [0.9, 0.2], and the new vector X = [0.7, 0.3], then Δ = |0.7-0.9| + |0.3-0.2| = 0.3. Secondly, based on the preliminary difference value Δ and the vector dimension number d (d = 2 in this example), the normalized distance value D is calculated according to the pre-designed calculation rule, wherein the calculation formula is: Then, the distance value is converted into a similarity relationship S (formula: ). The similarity relationship S is forced to be 0 when the new archive is first input. In this example, S = 1-0.14 = 0.86. Then, when the similarity relationship is less than, the cluster boundary contraction operation is triggered, and the relationship between S and the alert parameter ρ is judged. If S ≥ ρ (for example, 0.86 ≥ 0.6), the expansion operation is triggered: X is added to the R&D post category, and the new cluster vector is updated to If S < ρ (for example, S = 0.5), the contraction operation is triggered: the competitive counterweight W = 1 / (1+|X-C|) is calculated, and in this example, if |X-C| = 0.5, then W = 0.67, and the new cluster vector is Finally, through 1046, whether the expansion or contraction operation, all existing cluster vectors in the system, such as the R&D post / administrative post category center, are recalculated to complete the model iteration update.

[0108] In actual application, in the agency D system, the new technical expert archive vector X = [0.75, 0.4]. 1041 calculates the Δ = |0.75-0.8| + |0.4-0.3| = 0.15 between X and the R&D post cluster vector C1 = [0.8, 0.3]. 1042 calculates D = 0.15 / (1.15+1.1) ≈ 0.07. 1043 obtains S = 0.93. Assuming that the alert parameter ρ = 0.6, S > ρ triggers the 1044 expansion operation: X is incorporated into the R&D post, and the new cluster vector is (X+C1) / 2 = [0.775, 0.35]. The system synchronously updates all category center vectors to complete iteration.

[0109] ​​The scheme dynamically judges the similarity and links with the alert parameter. The step can intelligently identify whether the new file should be classified into the existing category or a new category. The boundary stretching operation gives the model the ability to include reasonable variants or isolate essential differences. The competitive feedback mechanism ensures smooth and stable updating process. Synchronous iteration of all clustering vectors enables the model to continuously track data evolution trends, fundamentally solving the classification rigidity problem caused by fixed thresholds in traditional methods, and significantly improving the adaptability of the system to incremental files.

[0110] 105. When the similarity relationship continuously falls below the threshold value set by the alert parameter, the new file vector is classified into a new file category, and a new clustering vector is established. The new clustering vector is added to the existing clustering vector set, and the digital classification result of personnel files is formed by the existing clustering vector set and the corresponding file category.

[0111] Optionally, step 105 can specifically include the following steps:

[0112] 1051. Set a counter with an initial value of zero for each similarity relationship obtained. Increase the counter value every time the similarity relationship is less than the alert parameter;

[0113] 1052. When the counter value reaches the preset value within the preset time window and the expansion operation is not triggered, it is determined that the similarity relationship continuously falls below the threshold value;

[0114] 1053. When the similarity relationship continuously falls below the threshold value set by the alert parameter, a new file category number is assigned to the new file vector, and the new file vector is used as a new clustering vector;

[0115] 1054. Add the new clustering vector to the existing clustering vector set, and update the file category record.

[0116] In the above scheme, the similarity relationship refers to the matching degree numerical interval 0-1 between the new file vector and the existing clustering vector. The alert parameter is the preset matching degree threshold value, for example, 0.6. The continuous fall below the threshold value means that the similarity of the new file cannot reach the alert value in continuous multiple comparisons. The new file category is a new classification number created by the system for files that cannot be classified into existing categories. The new clustering vector is a center vector representing the core features of the new category. The existing clustering vector set is a summary of all existing category center vectors. The digital classification result is the final structured classification system.

[0117] The embodiment of the present application first sets an independent counter with initial value 0 for the similarity relationship between each newly added file vector and each existing category. When the similarity relationship is less than the warning parameter in a comparison, the counter value increases by 1. For example, the similarity between the newly added marketing plan file and the research and development category is 0.5, which is less than the threshold value 0.6, and the corresponding counter of the research and development category is incremented by 1. Secondly, through 1052, a time window such as 30 minutes and a count threshold such as 10 times are set. If a counter reaches 10 times within the time window and the newly added file has never triggered the expansion operation of 1044, it is determined that the similarity relationship is continuously lower than the threshold value. For example, the file fails to match the research and development category for 10 times within 30 minutes and does not trigger the new category determination by being integrated into this category. Then, through 1053, when it is determined that the similarity relationship is continuously lower than the threshold value, the system automatically allocates a new category number such as category ID 999 and uses the newly added file vector as the new cluster vector to initialize the new category. Finally, through 1054, the generated new cluster vector is added to the existing cluster vector set and the file category record is updated, such as adding the new innovative marketing position category. The system outputs all cluster vectors and corresponding categories as the final classification result.

[0118] In actual application, a cross-border marketing plan file is added in the system of the organization E. In the step 1051, the similarity between the file and the existing market category research and development category is continuously lower than 0.6 for 8 times, and the corresponding counter is increased to 8. In the step 1052, the file fails to match for the 10th time within 20 minutes, the counter reaches 10 times, and the merging operation has not been triggered. In the step 1053, the system creates a new category ID 999 marketing innovative position, and uses the file vector as the new cluster vector. In the step 1054, the new vector is added to the category library, and the system outputs the complete classification system including the new category.

[0119] The present scheme dynamically monitors the files that cannot be classified by using the counter mechanism, and can reliably identify the truly new type files to avoid misjudgment caused by accidental fluctuations. The automatic creation of the new category and the cluster vector enables the classification system to have dynamic expansion capability, and ensures that the system can continuously track the evolution of industry positions, fundamentally solves the defect that the traditional method cannot automatically discover new categories, and enhances the completeness and timeliness of the classification system.

[0120] Figure 2 A scene diagram of a personnel file digitization automatic classification method based on word statistics is provided for the embodiment of the present application, as shown in Figure 2 For a complete embodiment of steps 101-105, it includes:

[0121] In the human resources system of Company A, when processing the blockchain technology position profile of Employee E of Division B, step 101 is first performed to identify the profile title area through document layout analysis. The first line of text "Employee E - Senior Blockchain Engineer" is identified as the title area, which is near the top of the page and meets the definition of a title. The second to fifth lines of the attribute area, "Division B" and "Tech Stack Solidity Rust", are aligned to the left and have consistent line spacing. The body area is the remaining item description text. The title area is independently split and the word distribution is output as "Senior Blockchain Engineer" appears once with a length of 7, "Solidity" appears once with a length of 8, and "Rust" appears once with a length of 4. The body area output distribution is "distributed" appears three times with a length of 3, and so on, forming partition word frequency data.

[0122] Then step 102 is performed to call precomputed global weight data: title area blockchain weight 0.92, engineering weight 0.75, attribute area Solidity weight 0.88, and body area distributed weight 0.61. The hierarchical calculation point multiplication for Employee E's profile is performed as follows: title area "Senior Blockchain Engineer" word frequency 1 x global title weight 0.92 = 0.92, attribute area "Solidity" word frequency 1 x 0.88 + "Rust" word frequency 1 x 0.72 = 1.6, and body area "distributed" word frequency 3 x 0.61 + the cumulative word frequency of the remaining words = 2.35. The sum of the three is the initial profile vector value 4.87.

[0123] When the system detects that the number of new profiles added in a single day exceeds the preset threshold of 500, step 103 is triggered to scan the set of vectors to be processed and detect that the Employee E vector contains 35 professional feature dimensions such as Solidity / Rust, which is determined to be a high-dimensional profile and is prioritized for scheduling into the clustering process. The target vector set is output.

[0124] In step 104, the Employee E vector 4.87 is compared with the existing research and development cluster vector 4.5 to calculate the initial difference value 0.37, which is normalized to a distance value 0.07 and converted to a similarity value 0.93. When this value is higher than the alert parameter 0.6, the boundary expansion operation is triggered, and Employee E is integrated into the research and development category to update the cluster vector to 4.685, and all category vectors are recalculated.

[0125] In the continuous processing process, it is found that a new cross-border marketing profile repeatedly matches the existing category through step 104, and the similarity value is consistently below 0.6. When it fails to match 10 times within 30 minutes and does not trigger expansion, it is determined that the threshold is consistently below the threshold. The system automatically creates a new profile category, "Marketing Innovation Position", initializes a new cluster vector with the profile vector, and integrates it into the existing cluster set. Finally, the complete digital classification system containing the new category is output.

[0126] The scheme significantly improves the accuracy of file classification by identifying structured areas and fusing different weights. In the case of a blockchain engineer, it accurately captures key features such as professional titles in the title area and technology stacks in the attribute area. The high-dimensional vector dynamic scheduling mechanism effectively deals with the new file flood, avoiding system processing congestion. The adaptive clustering model uses the alert parameter as a boundary to intelligently expand the matching samples to the R&D category while maintaining the core features. Continuous low similarity monitoring reliably identifies new job types and automatically builds marketing innovation categories. The entire process realizes a self-optimizing closed loop of the classification system, accurately capturing the subtle differences in professional features of existing categories and dynamically expanding to adapt to organizational structure evolution, ultimately forming a highly robust digital classification capability.

[0127] Figure 3 A structure diagram of a personnel file digital automatic classification system based on word statistics is provided for the embodiments of the present application, as shown in Figure 3 The system comprises:

[0128] The generation module 31 is configured to obtain original file data of personnel files, identify the boundaries of structured areas in the original file data, separate the title area, the text area, and the attribute annotation area, and perform multi-dimensional word statistics processing on each type of area to generate partition word frequency data.

[0129] The fusion module 32 is configured to perform hierarchical weighted fusion of the partition word frequency data and cross-file word weight quantitative data reflecting the contribution of specific words to the differentiation degree in the overall file set, to form an initial file vector set.

[0130] The monitoring module 33 is configured to monitor the peak state of the new file data flow, and when the number of continuously added files exceeds a preset parallel processing threshold, the high-dimensional file vector in the initial file vector set is preferentially put into the clustering process to generate a target file vector set.

[0131] The driving module 34 is configured to input the target file vector set into an adaptive clustering mechanism, compare the similarity relationship between the new file vector and the existing clustering vector, trigger the clustering boundary stretching operation controlled by the alert parameter based on the similarity relationship, and simultaneously drive the iterative update of the existing clustering vector based on the competitive feedback mechanism.

[0132] The classification module 35 is configured to classify the new file vector into a new file category and establish a new clustering vector when the similarity relationship continuously falls below the threshold set by the alert parameter, and the new clustering vector is added to the existing clustering vector set, and the existing clustering vector set and the corresponding file category are used to construct the digital classification result of personnel files.

[0133] Figure 3The one kind of personnel file digitization automatic classification system based on vocabulary statistics can execute Figure 1 The one kind of personnel file digitization automatic classification method based on vocabulary statistics of the embodiment shown can not be described again for the implementation principle and technical effect. For the specific manner of the operation of each module, unit of the one kind of personnel file digitization automatic classification system based on vocabulary statistics in the above embodiment has been described in detail in the embodiment related to the method, which will not be described in detail here.

[0134] In one possible design, Figure 3 The one kind of personnel file digitization automatic classification system based on vocabulary statistics of the embodiment shown can be implemented as a computing device, such as Figure 4 As shown, the computing device can include a storage component 41 and a processing component 42.

[0135] The storage component 41 stores one or more computer instructions, wherein the one or more computer instructions are called for execution by the processing component 42.

[0136] The processing component 42 is used for the above Figure 1 The one kind of personnel file digitization automatic classification method based on vocabulary statistics of the embodiment.

[0137] Among them, the processing component 42 can include one or more processors to execute computer instructions to complete all or part of the steps in the above method. Of course, the processing component can also be one or more application specific integrated circuits (ASIC), digital signal processors (DSP), digital signal processing devices (DSPD), programmable logic devices (PLD), field programmable gate arrays (FPGA), controllers, microcontrollers, microprocessors or other electronic elements, for executing the above method.

[0138] The storage component 41 is configured to store various types of data to support the operation of the terminal. The storage component can be realized by any type of volatile or non-volatile storage device or their combination, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic storage, flash memory, magnetic disk or optical disk.

[0139] Of course, the computing device must also include other components, such as input / output interface, display component, communication component, etc.

[0140] The input / output interface provides an interface between the processing component and the peripheral interface module, which can be an output device, an input device, etc.

[0141] The communication component is configured to facilitate wired or wireless communication between the computing device and other devices, etc.

[0142] The computing device can be a physical device or an elastic computing host provided by a cloud computing platform, and the computing device can be a cloud server. The processing component, the storage component, etc. can be basic server resources rented or purchased from the cloud computing platform.

[0143] The embodiments of the present application further provide a computer storage medium storing a computer program, and the computer program can implement the above-mentioned Figure 1 The embodiment shown in the figure is a personnel file digital automatic classification method based on vocabulary statistics.

[0144] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the above-mentioned system, device and unit can refer to the corresponding processes in the foregoing method embodiments, which will not be described here.

[0145] The device embodiments described above are only schematic, and the units described as separate components can or can not be physically separated, and the components shown as units can or can not be physical units, that is, they can be located in one place, or distributed on multiple network units. Part or all of the modules can be selected to achieve the purpose of the embodiment scheme according to actual needs. Those skilled in the art can understand and implement without creative labor.

[0146] Through the description of the foregoing embodiments, those skilled in the art can clearly understand that each embodiment can be realized by means of software and necessary general hardware platform, and of course, it can also be realized by hardware. Based on such understanding, the above technical solutions can be embodied in the form of software product, which can be stored in a computer readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes a plurality of instructions to make a computer device (which can be a personal computer, server, etc.) execute the method described in each embodiment or some part of the embodiment.

[0147] Finally, it should be noted that: the above embodiments are only used to illustrate the technical solutions of the present application, and not to limit them; although the present application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that: it can still modify the technical solutions recorded in the foregoing embodiments, or make equivalent replacement for some technical features; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present application.

Claims

1. A method for automatic classification of personnel files based on lexical statistics, characterized in that, The method comprises the following steps: Obtaining original file data of personnel files, identifying the boundaries of structured areas in the original file data, separating the title area, the text area and the attribute label area, and independently performing multi-dimensional word count processing on each type of area to generate partition word frequency data; Performing hierarchical weighted fusion on the partition word frequency data and cross-file word weight quantitative data reflecting the contribution of the distinctness of specific words in the overall file set to form an initial file vector set; Monitoring the peak state of the newly added file data flow, and when the number of continuously added files exceeds the preset parallel processing threshold, controlling the high-dimensional file vectors in the initial file vector set to enter the clustering process preferentially to generate a target file vector set; Inputting the target file vector set into an adaptive clustering mechanism, comparing the newly added file vector with the existing clustering vector to obtain their similarity relationship, triggering the clustering boundary expansion and contraction operation controlled by the alert parameter based on the similarity relationship, and synchronously driving the iterative update of the existing clustering vector based on the competitive feedback mechanism; When the similarity relationship continuously falls below the threshold set by the alert parameter, classifying the newly added file vector into a new file category and establishing a new clustering vector, the new clustering vector is added to the existing clustering vector set, and the existing clustering vector set and the corresponding file category are used to construct the digital classification result of the personnel file.

2. The method of claim 1, wherein, Triggering the clustering boundary expansion and contraction operation controlled by the alert parameter based on the similarity relationship, and synchronously driving the iterative update of the existing clustering vector based on the competitive feedback mechanism, comprising: Judging whether the similarity relationship is greater than or equal to the preset alert parameter value, when the similarity relationship is greater than or equal to, triggering the clustering boundary expansion operation, adding the newly added file vector to the existing file vector, and replacing the original clustering vector with the average value of the clustering vector corresponding to the newly added file vector and the existing clustering vector as a new clustering vector; When the similarity relationship is less than, triggering the clustering boundary contraction operation, calculating the competitive weight based on the corresponding position of the elements of the newly added file vector and the existing file vector, and replacing the original clustering vector with the weight proportion average value of the clustering vector corresponding to the newly added file vector and the existing clustering vector as a new clustering vector; The expansion or contraction operation of the clustering boundary drives the recalculation of all existing clustering vectors to complete the iterative update.

3. The method of claim 1, wherein, Inputting the target file vector set into an adaptive clustering mechanism, comparing the newly added file vector with the existing clustering vector to obtain their similarity relationship, comprising: Calculating the sum of the absolute values of the difference values of the corresponding positions of the elements of the newly added file vector and the existing clustering vector corresponding to each existing file vector as a preliminary difference value; Based on the preliminary difference value, the number of elements of the newly added file vector and the existing clustering vector, a normalized distance value is calculated according to the pre-designed calculation rule; The normalized distance value is converted into a corresponding similarity relationship, wherein when the newly added file vector is input for the first time, the similarity relationship is set to 0.

4. The method of claim 1, wherein, When the similarity relationship continues to be lower than the threshold value set by the alert parameter, a new archive vector is classified into a new archive category according to the new archive vector, and a new clustering vector is established, and the new clustering vector is added to the existing clustering vector set, including: A counter with an initial value of zero is set for each similarity relationship obtained by comparison, and a counter value is increased each time the similarity relationship is less than the alert parameter; When the counter value reaches a preset value within a preset time window and the expansion operation is not triggered, it is judged that the similarity relationship continues to be lower than the threshold value; When the similarity relationship continues to be lower than the threshold value set by the alert parameter, a new archive category number is assigned to the new archive vector, and the new archive vector is used as a new clustering vector; The new clustering vector is added to the existing clustering vector set, and the archive category record is updated.

5. The method of claim 1, wherein, The partition word frequency data is hierarchically weighted and fused with the cross-archive word weight quantization data reflecting the contribution of the distinctness of a specific word in the overall archive set to form an initial archive vector set, including: Based on the partition word frequency data of the overall archive set, the frequency difference value of each word is calculated as a distinctness value; Region-specific cross-archive word weight data is generated for the title area, the text area, and the attribute annotation area, and the weight value is determined by the distinctness value; The title area word distribution, the text area word distribution, and the attribute annotation area word distribution of each archive are sequentially multiplied with the region-specific cross-archive word weight data; The title area result, the text area result, and the attribute annotation area result after the multiplication operation are added to form the initial archive vector of each archive, and the initial archive vectors of all archives are combined to form the initial archive vector set.

6. The method of claim 1, wherein, The initial archive vector set is monitored for a new archive data flow peak state, and when the number of continuously added archives exceeds a preset parallel processing threshold, the high-dimensional archive vector in the initial archive vector set is controlled to enter the clustering process first to generate a target archive vector set, including: The number of arriving new archive data and the interval time are recorded, and when the number of continuously added archives reaches a preset parallel processing threshold, a priority judgment logic is triggered; The dimension number of each vector in the initial archive vector is calculated by the priority judgment logic, and the archive vector with the largest dimension number is marked as a high-dimensional archive vector; The high-dimensional archive vector is controlled to enter the clustering process first for subsequent operation, and the non-high-dimensional archive vector waits for processing; The clustering process outputs the high-dimensional archive vector and other archive vectors to form the target archive vector set.

7. The method of claim 1, wherein, The structured region boundaries in the original archive data are identified, and the title area, the text area, and the attribute annotation area are separated, and multi-dimensional word statistical processing is independently performed on each type of area to generate partition word frequency data, including: The features of the starting position of each text line, the interval distance from other lines, and the text concentration in the original archive data are detected; Based on the comparison of the features and the preset boundary judgment rule, the title area boundary, the attribute annotation area boundary, and the text area boundary are identified and separated; The text in the title area, the text area and the attribute mark area is sequentially subjected to vocabulary splitting operation, and the occurrence frequency and the character length of each vocabulary are independently calculated; The occurrence frequency and the character length of each vocabulary are combined to form title area vocabulary distribution, text area vocabulary distribution and attribute mark area vocabulary distribution as partition word frequency data.

8. A personnel file digitization automatic classification system based on lexical statistics, characterized by, It comprises: Obtaining original archive data of personnel archives, identifying the boundary of structured area in the original archive data, separating the title area, the text area and the attribute mark area, and independently performing multi-dimensional vocabulary statistical processing on each type of area to generate partition word frequency data; The partition word frequency data and the cross-archive vocabulary weight quantization data reflecting the contribution of the specific vocabulary to the differentiation degree in the whole archive set are hierarchically weighted and fused to form an initial archive vector set; Monitoring the peak state of the newly added archive data flow, when the number of continuously added archives exceeds the preset parallel processing threshold, the high-dimensional archive vector in the initial archive vector set is preferentially put into the clustering process to generate a target archive vector set; The target archive vector set is input into the adaptive clustering mechanism, the similarity relationship between the newly added archive vector and the existing clustering vector is obtained by comparison, and the clustering boundary stretching operation controlled by the alert parameter is triggered based on the similarity relationship, and the iterative update of the existing clustering vector based on the competitive feedback mechanism is synchronously driven; When the similarity relationship continuously falls below the threshold set by the alert parameter, the newly added archive vector is classified into a new archive category and a new clustering vector is established, the new clustering vector is added to the existing clustering vector set, and the existing clustering vector set and the corresponding archive category are used to construct the digital classification result of personnel archives.

9. A computing device, comprising: It comprises a processing component and a storage component; the storage component stores one or more computer instructions; the one or more computer instructions are used to be called and executed by the processing component to realize the personnel archive digital automatic classification method based on vocabulary statistics according to any one of claims 1 to 7.

10. A computer storage medium, characterized in that, The computer program is stored in the computer, and when the computer program is executed by the computer, the personnel archive digital automatic classification method based on vocabulary statistics according to any one of claims 1 to 7 is realized.

Images