Multimodal fusion resume processing methods, equipment and media
By employing a multimodal fusion resume processing method, text, image, and layout features of various resume formats are extracted and processed. Pre-trained models and knowledge bases are used to improve the accuracy and robustness of resume recognition, thus solving the problem of low recognition accuracy in existing technologies.
Patent Information
- Application Number
- CN202611360545.2
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2026-09-03
- Publication Date
- 2026-11-17
AI Technical Summary
In existing technologies, OCR recognition technology has low accuracy in recognizing handwritten resumes, blurry scans, and resumes with complex layouts. It is prone to missing characters and misaligned fields, which affects subsequent resume processing.
A multimodal fusion resume processing method is adopted to extract text features, image features and layout features of resume files, and to perform resume recognition using a pre-trained target recognition model and a large language model, combined with a pre-set knowledge base for data extraction and processing.
It improves the robustness of recognition of complex resume formats and the accuracy of field extraction, reduces character omissions and field misalignments, and supports efficient processing of various resume formats.
Smart Images

Figure CN123118050A_ABST
Abstract
Description
Technical Field
[0001] This application relates to the field of digital human resource management technology, and more specifically, to a multimodal resume processing method, device, and medium. Background Technology
[0002] In the current process of digital human resource management, corporate recruitment often involves processing massive amounts of resume data in various formats, including scanned copies of paper resumes, image resumes, PDF resumes, Word resumes, and more. Current technology uses OCR to extract data from these resumes and processes the resumes based on the extracted results. However, traditional OCR technology is primarily optimized for single formats (such as standard printed PDFs), and its accuracy is low for handwritten resumes, blurry scans, and resumes with complex layouts (such as multi-column layouts and mixed text and images). This can easily lead to problems such as missed character recognition and misaligned fields, affecting subsequent resume processing. Summary of the Invention
[0003] The purpose of this application is to address the shortcomings of existing technologies by providing a multimodal fusion resume processing method, device, and medium. This solution addresses the problems of existing resume data extraction methods that only optimize for a single format, making it difficult to effectively identify other formats, resulting in low recognition accuracy, easy character omissions, and field misalignments, which affect subsequent resume processing.
[0004] To achieve the above objectives, this application adopts the following technical solution: This application provides a multimodal fusion resume processing method, the method comprising: Extract multimodal features from resume files, and obtain resume recognition results based on the multimodal features and a pre-trained target recognition model. The multimodal features include text features, image features, and layout features. The target recognition model is trained based on the labeled resume and the corresponding multimodal features. Resume data is extracted from the resume recognition results based on a preset knowledge base and a large language model. The preset knowledge base is built based on multidimensional knowledge, which includes field names, synonyms, data formats, and related fields. The resume data is processed, and the resume processing includes at least one of generating reports, data statistics, and generating polishing suggestions.
[0005] In one possible implementation, the extraction of multimodal features from the resume file includes: Obtain the resume file, call the extraction model to extract the multimodal features in parallel, and use different extraction models for different multimodal features.
[0006] In one possible implementation, the target recognition model includes a teacher model and a student model, and the acquisition of the target recognition model includes: Annotate historical resumes, including character-level text annotations, layout area-level annotations, field entity-level annotations, and visual attribute-level annotations; A multimodal backbone network is trained using labeled historical resumes to obtain a teacher model. The loss function of the teacher model is generated based on the loss of the text recognition branch, the loss of the layout detection branch, the loss of the entity extraction branch, and the cross-modal loss in the multimodal backbone network. The teacher model is distilled to obtain the student model.
[0007] In one possible implementation, the distillation teacher model includes: Obtain the soft label probability distribution and intermediate layer feature distribution corresponding to the teacher model; The soft label distribution and corresponding hard label are used to constrain the output layer of the student model, and feature alignment constraints are applied to the multimodal fusion feature layers of the teacher model and the student model based on the intermediate layer feature distribution.
[0008] In one possible implementation, obtaining the resume recognition result based on the multimodal features and the pre-trained target recognition model includes: The resume file and the multimodal features are input into the student model to obtain the output results of the student model. The output results include the recognition results of each region in the resume file and the confidence scores of each region. The resume recognition results are obtained based on the confidence level and layout features of each region.
[0009] In one possible implementation, obtaining the resume recognition result based on the confidence level and layout features of each region includes: It is determined that the confidence level is less than a preset threshold and the layout feature is a core field; A local multimodal feature package is generated based on the local image, text features, and layout coordinates of the region corresponding to the confidence level. The region recognition result corresponding to the local multimodal feature package is obtained using the teacher model, and a resume recognition result is generated based on the region recognition result and the output result of the student model.
[0010] In one possible implementation, the step of extracting resume data from the resume recognition results based on a preset knowledge base and a large language model includes: The resume recognition results are semantically understood and processed using a large language model to obtain text fragments. The semantic understanding and processing includes item boundary segmentation, semantic role determination, and association completion. The text fragments are processed by entity linking and standardized expression based on the preset knowledge base to obtain the resume data. Verify the resume data and perform ETL processing on the resume data based on the verification result.
[0011] In one possible implementation, the ETL processing of the resume data based on the verification result includes: Based on the verification results, obtain the analytical confidence level and the recognition confidence level output by the target recognition model; The resume data is processed using ETL based on the resolution confidence and the identification confidence.
[0012] In another aspect, this application provides an electronic device including a processor and a memory, wherein the processor is communicatively connected to the memory, the memory stores program data, and the program data is executed by the processor to implement the method described in any of the above embodiments.
[0013] In another aspect, this application provides a computer-readable storage medium storing a computer program that is executed by a processor to implement the methods described in any of the above embodiments.
[0014] The beneficial effects of the technical solutions provided in this application are: The multimodal fusion resume processing method provided in this application includes: extracting multimodal features from resume files; obtaining resume recognition results based on the multimodal features and a pre-trained target recognition model; the multimodal features include text features, image features, and layout features; the target recognition model is trained based on the labeled resume and the corresponding multimodal features; extracting resume data from the resume recognition results based on a preset knowledge base and a large language model; the preset knowledge base is built based on multidimensional knowledge, including field names, synonyms, data formats, and related fields; and processing the resume data, including at least one of generating reports, data statistics, and generating polishing suggestions. This application's embodiment integrates multimodal features such as text, images, and layout, combines a pre-trained target recognition model for resume recognition, and utilizes a preset knowledge base and a large language model for resume data extraction. This effectively improves the robustness of recognition and field extraction accuracy for resumes with complex formats (such as handwritten text, blurry scans, and multi-column mixed-format PDFs), reduces the occurrence of character omissions and field misalignments, and facilitates subsequent resume processing. Attached Figure Description
[0015] To more clearly illustrate the technical solutions in the embodiments of this application, the accompanying drawings used in the description of the embodiments of this application will be briefly introduced below.
[0016] Figure 1 A flowchart of a multimodal fusion resume processing method provided in an embodiment of this application; Figure 2 A flowchart generated for the polishing suggestions provided in the embodiments of this application; Figure 3 A structural diagram of the electronic device provided in the embodiments of this application; Figure 4 This is a structural diagram of a computer-readable storage medium provided in an embodiment of this application. Detailed Implementation
[0017] The embodiments of this application are described below with reference to the accompanying drawings. It should be understood that the embodiments described below with reference to the accompanying drawings are exemplary descriptions for explaining the technical solutions of the embodiments of this application, and do not constitute a limitation on the technical solutions of the embodiments of this application.
[0018] Those skilled in the art will understand that, unless specifically stated otherwise, the singular forms “a,” “an,” “the,” and “the” used herein may also include the plural forms. It should be further understood that the terms “comprising” and “including” as used in embodiments of this application mean that the corresponding feature can be implemented as the presented feature, information, data, step, operation, element, and / or component, but do not exclude implementation as other features, information, data, step, operation, element, component, and / or combinations thereof supported by this art. It should be understood that when we say that an element is “connected” or “coupled” to another element, the one element can be directly connected or coupled to the other element, or it can mean that the one element and the other element establish a connection relationship through an intermediate element. Furthermore, “connected” or “coupled” as used herein can include wireless connection or wireless coupling. The term “and / or” as used herein indicates at least one of the items defined by the term; for example, “A and / or B” indicates implementation as “A,” or implementation as “A,” or implementation as “A and B.”
[0019] To make the objectives, technical solutions, and advantages of this application clearer, the embodiments of this application will be described in further detail below with reference to the accompanying drawings.
[0020] The technical solutions of this application and their effects are described below through several exemplary embodiments. It should be noted that the following embodiments can be referenced, borrowed from, or combined with each other. Identical terms, similar features, and similar implementation steps in different embodiments will not be repeated.
[0021] The multimodal fusion resume processing method, device and medium provided in this application are intended to solve at least one technical problem existing in the prior art.
[0022] Optionally, such as Figure 1 , Figure 2 As shown, the multimodal fusion resume processing method of this application can be used on computers, mobile phones, servers, cloud platforms, and other terminals that can be used to extract resume data and process resumes.
[0023] Optionally, the multimodal fusion resume processing method of this application includes: S101: Extract multimodal features from resume files, and obtain resume recognition results based on multimodal features and a pre-trained target recognition model.
[0024] Optionally, the multimodal features include text features, image features, and layout features. The target recognition model is trained based on the labeled resume and the corresponding multimodal features. By combining the target recognition model with the multimodal features, high-precision recognition of resumes with complex formats can be achieved.
[0025] Optionally, the resume file can be in various formats such as PDF (text, scanned image), image (JPG, PNG), Word (text, embedded image), excel (csv, xls), ppt, video, etc.
[0026] Optionally, before extracting multimodal features, the resume file can be converted into text or an image, and multimodal feature extraction can be performed based on this text or image. Different extraction methods are used for different resume file formats. Specifically, for image-based resumes, OCR can be used to recognize and convert each element into high-precision text, with skew correction and noise removal of meaningless characters performed on the corresponding character coordinates. For video resumes, frame extraction can be performed to convert the extracted video frames into image content. For text-based content in resumes, the text can be extracted and concatenated into a long text, which is then combined with the OCR recognition results for deduplication and fusion to obtain the full text of the resume.
[0027] Optionally, extracting multimodal features from the resume file includes: obtaining the resume file, calling an extraction model to extract multimodal features in parallel, with different extraction models corresponding to different multimodal features.
[0028] Optionally, text features may include low-level visual texture features and high-level semantic features. The low-level visual texture features may include grayscale value features and edge features. The extraction model corresponding to the text features may be a model trained using a convolutional neural network.
[0029] In one embodiment, when extracting grayscale features, text blocks related to the resume document can be converted into single-channel grayscale images. Then, the convolutional kernels of a lightweight convolutional neural network in the extraction model are used to learn the pixel grayscale distribution, stroke density, and thickness variations of the character regions, capturing the stroke shapes of the characters. When extracting edge features, gradient convolutional kernels in the extraction model are used to extract the contour edges, stroke endpoints, and corner information of the characters, characterizing their structural outlines and thus distinguishing the font levels of titles and body text. After extracting grayscale and edge features, the extraction model outputs a fixed-dimensional visual feature vector (including edge and grayscale features), corresponding to the visual attributes of each character and text block.
[0030] In one embodiment, when extracting high-level semantic features, the text character sequence of a resume document (which can be obtained through OCR recognition) can be acquired. A pre-trained extraction model (trained using character sequence samples labeled with high-level semantic features) is then used to perform word embedding encoding on this character sequence to obtain character-level and word-level semantic feature vectors. This captures the semantics, part-of-speech, and contextual relationships of the text, achieving semantic understanding of the fields. The extraction models corresponding to the low-level visual texture features and the high-level semantic content features can be different.
[0031] Optionally, the image feature extraction model can be trained using object detection algorithms (such as R-CNN, Fast R-CNN, Faster R-CNN, YOLO, SSD, and other types of algorithms). This extraction model distinguishes between text and image regions in a resume file, avoiding interference from images in resume processing.
[0032] Alternatively, image feature extraction can be performed by first detecting image-text region separation, and then extracting image region features based on the detection results.
[0033] In one embodiment, during image-text region separation detection, the original raster image of the entire resume file is acquired. This original raster image is input into the extraction model, which performs category detection on all elements in the image and outputs the bounding box and category label for each region. Preset categories include: plain text blocks, avatars, company logos, decorative patterns, charts, QR codes, separator lines, etc. Based on these bounding boxes and category labels, non-text regions are masked from the text recognition process, preventing decorative images, avatar backgrounds, and table textures from being misidentified as text, thus significantly reducing the OCR false recognition rate.
[0034] When extracting image region features, the image features of image blocks in the resume file are extracted based on the bounding box and category label. These image features include normalized position coordinates, size, page percentage, and category.
[0035] Optionally, layout features can be extracted using image features. The image features can be input into the extraction model corresponding to the layout features, and the global feature vector (layout features) of the image can be obtained through the extraction model. The layout features can be used to assist in determining the attributes of the blocks.
[0036] Optionally, the extraction model corresponding to the layout features can use the Transformer architecture and leverage a lightweight CNN to extract global feature vectors. When training this extraction model, the positional and attribute codes of each element in historical resume files can be obtained based on historical image features, and the extraction model is trained based on these positional and attribute codes. After training on a large amount of resume data, the model will automatically summarize typical resume layout patterns, such as "large font text centered at the top of the page usually indicates the name," "a timeline on the left and body text on the right usually indicates work experience," and "horizontally arranged tags at the bottom usually indicate skills."
[0037] In one embodiment, after acquiring image features, a layout feature library is established based on these image features. The specific content of establishing the layout feature library includes: ① Element coordinate normalization and attribute serialization: Based on text and image features, the bounding box coordinates (x1, y1, x2, y2) of all elements are obtained. These coordinates are then normalized to the width and height of the entire resume, mapping to the 0-1 range to eliminate the influence of different resolutions and paper sizes. Each element is supplemented with attribute tags: element type (e.g., title, body text, table, image), font size level, bolding, number of rows and columns, etc.
[0038] ②Spatial location coding: Generate a unique layout embedding vector for each element, containing three types of information: Absolute position encoding: the top, bottom, left, and right positions of an element on the page; Relative position encoding: the spatial distance and alignment relationship between two elements; Size hierarchy encoding: The hierarchy corresponding to the width, height, and font size of an element.
[0039] The positional and attribute codes of all elements are input into the Transformer encoder of the extraction model. The model learns the spatial dependencies between elements through a self-attention mechanism, thereby outputting the layout features (such as layout patterns) corresponding to the document. The layout features can be based on the normalized element bounding box coordinates, and absolute and relative position embeddings are generated through sine position encoding. The element size and font level attribute codes are fused to fully represent the spatial topological relationships and hierarchical structure of the page elements.
[0040] Optionally, the target recognition model includes a teacher model and a student model. The acquisition of the target recognition model includes: labeling historical resumes, including character-level text labeling, layout region-level labeling, field entity-level labeling, and visual attribute-level labeling; training a multimodal backbone network using the labeled historical resumes to obtain the teacher model, the loss function of which is generated based on the loss of the text recognition branch, the layout detection branch, the entity extraction branch, and the cross-modal loss in the multimodal backbone network; and distilling the teacher model to obtain the student model.
[0041] In one embodiment, marking historical resumes includes: Character-level text annotation: Annotate text line bounding boxes, character-by-character content and character-level coordinates; some high-precision datasets are supplemented with annotations of stroke key points; Page area labeling: Based on image features, semantic segmentation and labeling are performed on the entire page of historical resumes, and the detection boxes and category labels of each area are output. The core categories include personal information section, work experience block, education experience table, skills label area, avatar, decorative image, separator line, etc. Field entity level annotation: Using the BIOS sequence annotation format, the text content is annotated with field entity types, and the association and pairing relationships of the experienced items are also annotated, which directly correspond to the training target of structured extraction; Visual attribute level annotation: supplement the annotation of text font level, bold status, font size level, table cell row and column affiliation, etc.
[0042] Optionally, the teacher model can be based on a full-parameter-scale multimodal backbone network, aiming to achieve optimal accuracy in resume scenarios, serving as the source of knowledge distillation. Specifically, the teacher model can employ a Base / Large-level multimodal backbone network, retaining all attention heads and hidden layer dimensions, possessing the strongest cross-modal feature fusion and semantic reasoning capabilities. During training, multi-task joint end-to-end training can be performed based on labeled data.
[0043] Optionally, in the loss function of the teacher model, the text recognition branch provides CTC (Connectionist Temporal Classification) loss to constrain the sequence accuracy of character recognition, which can be obtained through the extraction model corresponding to the text features; the layout detection branch provides CIoU (Complete IoU) loss to constrain the localization accuracy of the layout region, which can be obtained through the extraction model corresponding to the image features; the entity extraction branch provides cross-entropy loss and CRF loss to constrain the boundary and classification accuracy of field extraction; and the cross-modal alignment loss is used to constrain the spatial semantic alignment consistency of the three types of features. After the teacher model is trained, the teacher model weights after product convergence, as well as the soft label probability distribution and intermediate layer feature distribution of the corresponding samples, are obtained.
[0044] Optionally, transfer learning can be applied to the teacher model. This involves introducing prior knowledge of the location of core resume fields (name, phone number, email, years of work experience, etc.) to enable the model to prioritize high-value field regions, thus improving recognition efficiency. During transfer learning, a parameter update mechanism of freezing the bottom layer and fine-tuning the top layer can be used. This freezes the CNN visual feature extraction layer and the first N Transformer encoding layers of the teacher model, preserving general feature extraction capabilities, while updating parameters only for the top-level cross-modal attention layer and the task output header.
[0045] In one embodiment, for the teacher model, a spatial attention bias matrix can be constructed based on statistical patterns in resume layout. This matrix can then be used to apply weighted constraints to the attention scores of the Transformer's self-attention layer, strengthening the model's attention allocation to key areas on the top and left side of the timeline, thus prioritizing high-value fields. Furthermore, a dedicated character set and terminology dictionary can be built based on the resume corpus, expanding the character classification dimensions of the model's output layer. Combined with a word constraint decoding mechanism, uncommon characters and industry-specific job terms can be included in the decoding constraints, reducing the character misrecognition rate. Simultaneously, a dedicated dictionary can be built for common uncommon characters and professional terms in resumes (such as "UI / UX design" and "cross-border e-commerce operations"), optimizing the character recognition dictionary and further reducing the misrecognition rate.
[0046] Optionally, the teacher model is distilled, including: obtaining the soft label probability distribution and intermediate layer feature distribution corresponding to the teacher model; using the soft label distribution and the corresponding hard labels to constrain the output layer of the student model, and performing feature alignment constraints on the multimodal fusion feature layer of the teacher model and the student model based on the intermediate layer feature distribution.
[0047] Optionally, the multimodal backbone network corresponding to the teacher model can be obtained based on the teacher model weights. The size of this backbone network can then be compressed using pruning and lightweight design to obtain the multimodal backbone network corresponding to the student model. Specifically, a lightweight backbone network can be constructed by reducing the number of Transformer layers, reducing the hidden layer dimension, and replacing depthwise separable convolutions.
[0048] In one embodiment, a two-layer distillation mechanism is used to distill the teacher model, specifically as follows: 1. Output layer soft label distillation: For the resume samples used to train the student model, the class probability distribution output by the teacher model is used as a soft supervision signal. Together with the hard labels (the true labels of the samples) of the real annotation, the output layer of the student model is constrained, so that the student model learns the classification confidence and fuzzy judgment logic of the teacher model, rather than just memorizing the standard answer. 2. Intermediate Feature Layer Distillation: Feature alignment constraints are applied to the multimodal fusion feature layers of the teacher and student models. The intermediate feature distribution of the student model is then made to approximate that of the teacher model using the MES loss, indirectly learning its multimodal feature fusion capability. The MES loss can be expressed as: Total distillation loss = α Hard label loss + β Soft label loss + γ The feature alignment loss uses α, β, and γ as weighting coefficients to balance the contributions of the three types of losses.
[0049] Optionally, obtaining resume recognition results based on multimodal features and a pre-trained target recognition model includes: inputting the resume file and multimodal features into a student model, obtaining the output results of the student model, the output results including the recognition results of each region in the resume file and the confidence level of each region; and obtaining the resume recognition results based on the confidence level and layout features of each region.
[0050] Optionally, after inputting the target recognition model, the attention mechanism in the model is used to perform weighted fusion of the three features: text, image, and layout. The confidence level of the recognition result is output based on the weighted fusion result. For regions with a confidence level below a threshold, a secondary recognition mechanism is initiated.
[0051] The processing steps of the target recognition model can be as follows: 1. First, map text features, image features, and layout features to the same-dimensional feature space through independent embedding layers to ensure that the three can interact and be computed. 2. Using text features as the query vector and image features and layout features as key-value vectors, the cross-modal attention module is used to complete feature interaction; 3. The cross-modal attention module calculates the contribution weight of layout features and image features to the current recognition task in each text region of the resume file: for example, in the large-font title region at the top, the weight of layout features will automatically increase to assist in judging field attributes; in the region of rare characters in the blurry scan, the weight of image edge features will automatically increase to assist in character shape judgment; in the region of professional terms with ambiguity, the weight of text semantic features is higher.
[0052] 4. Finally, the fused unified feature vector is output, and based on the attention weight distribution and classification probability concentration, the confidence score of the region recognition result is output. The higher the feature consistency and the more concentrated the classification probability, the higher the confidence score.
[0053] Optionally, the student model includes a cross-modal attention module and a confidence output head, which outputs the confidence level corresponding to the recognition result. Feature fusion, result output, and confidence level calculation can be completed simultaneously in a single forward inference. After the student model outputs the resume recognition result and the confidence level corresponding to each region, the inference scheduling engine determines the flow based on a threshold (such as a preset value of 95%): core fields trigger teacher model refinement, and non-core fields trigger edge feature completion.
[0054] Optionally, resume recognition results are obtained based on the confidence level and layout features of each region, including: determining that the confidence level is less than a preset threshold and the layout features are the core fields; generating a local multimodal feature package based on the local image, text features, and layout coordinates of the region corresponding to the confidence level; using the teacher model to obtain the region recognition results corresponding to the local multimodal feature package; and generating resume recognition results based on the region recognition results and the output results of the student model.
[0055] In one embodiment, for the region corresponding to a core field with a confidence level below a threshold, the following operation is performed: 1. Extract the local image of the region, its corresponding text features, and layout coordinates, and encapsulate them into a local multimodal feature package; 2. Input the local multimodal feature package into the teacher model, and use the teacher model's deeper network layers, stronger cross-modal fusion capability and scene prior knowledge to perform refined reasoning on the region, and output higher accuracy recognition results and feature representations; 3. Use the high-precision results output by the teacher model to directly replace the recognition results of the corresponding regions in the student model to complete the local correction.
[0056] For low-confidence regions in non-core fields, the character edge contour features (stroke endpoints, corners, connected component structure) of the corresponding text in that region can be extracted. These features are then used to validate and complete the top N candidate resume recognition results with the highest confidence output by the student model. The completion and correction occur during the candidate set decision stage of character recognition, where the candidate results output by the student model are reordered and corrected, resulting in a corrected resume recognition result.
[0057] S102: Extract resume data from resume recognition results based on a pre-set knowledge base and a large language model.
[0058] Optionally, the default knowledge base is built based on multidimensional knowledge, which includes field names, synonyms, data formats, and related fields. Specifically, it can organize resume field standards and industry-standard expressions in the human resources field to construct a four-dimensional knowledge base containing "field name - synonym - data format - related fields".
[0059] Optionally, the large language model, based on the knowledge base and resume recognition results, uses semantic understanding and entity linking technology to achieve contextual association recognition of fields; for non-standard expressions, it matches standard fields through semantic similarity calculation.
[0060] Optionally, resume data is extracted from the resume recognition results based on a preset knowledge base and a large language model, including: using the large language model to perform semantic understanding processing on the resume recognition results to obtain text fragments, the semantic understanding processing including item boundary segmentation, semantic role determination, and association completion; performing entity linking processing and standardized expression processing on the text fragments according to the preset knowledge base to obtain resume data; verifying the resume data, and performing ETL processing on the resume data according to the verification results.
[0061] In one embodiment, semantic understanding processing includes: ① Item boundary segmentation: The large language model first performs semantic hierarchical parsing on the resume recognition results, and combines layout features (line breaks, bullet points, timelines, heading levels) with semantic logic to break down independent business entries. For example, multiple work experiences are split into independent single experiences, distinguishing different modules such as education experience and project experience, thus avoiding field confusion across entries and modules from the root.
[0062] ②Preliminary semantic role assessment: Based on the context, a preliminary determination is made of the field attributes of the text fragment corresponding to each business item. For example, the phrase "responsible for user growth and channel operations" cannot be clearly identified as belonging to any field when viewed in isolation. However, combined with the job information of "senior operations specialist" mentioned above, semantic understanding can determine that it belongs to the "job responsibilities" field, rather than an independent job or skill description.
[0063] ③Implicit relation completion: The model identifies referential and ellipsis expressions in text analysis and completes the logical relationships between fields. For example, in a resume, if the phrase "led the XX project during the same period" appears, the model can determine through context that it belongs to the previous work experience and automatically fill in the corresponding company and time information, thus achieving implicit field association.
[0064] Optionally, entity linking is used to establish a one-to-one mapping between entity fragments in the text and standard fields in the knowledge base, and field pairing is completed based on the association rules of the knowledge base. Specific processing methods include: ① Entity matching and disambiguation: The text segments split out by semantic understanding are matched with "field name + synonymous expression" in the four-dimensional knowledge base, and ambiguity is eliminated in combination with context semantics. For example, the term "project manager" in the text is linked to the standard field "Position Held" when appearing in the work experience module, and linked to "Project Role" when appearing in the project experience module. Entity linking will complete semantic disambiguation according to the context of the affiliated module to ensure accurate field mapping.
[0065] ② Associated field binding: Using the prior relationship of "associated fields" in the knowledge base, multiple fields under the same entry are structurally bound. For example, under the same work experience, the identified "start date, end date, employing company, position, job responsibility" will be bound into a complete work experience entity through the preset association relationship of the knowledge base, rather than scattered independent fields.
[0066] Optionally, after performing semantic understanding processing and entity linking processing on the resume recognition result, the processed resume recognition result is subjected to standardization processing. In the standardization process, synonymous expressions in the knowledge base can be accurately matched based on semantic similarity. For synonymous expressions, industry common names, and abbreviations already included in the knowledge base, semantic vector similarity calculation is used to complete the matching, so as to correspond to the "synonymous expression" dimension of the knowledge base, and the standardized expression corresponding to the resume recognition result is determined based on this dimension (for example, the synonymous expression in the knowledge base is used as the standardized expression corresponding to the recognition result).
[0067] It is also possible to convert the standard names, synonymous expressions, and industry terms of all fields in the preset knowledge base into semantic vectors of a unified dimension before matching, to construct a vector retrieval index; generate semantic vectors synchronously for the non-standard text segments in the resume recognition result, and match the standard field with the closest semantics in the knowledge base through cosine similarity calculation; perform secondary verification in combination with the module to which the field belongs and the data format rules. For example, "full-time undergraduate", "bachelor's degree", and "undergraduate graduation" will all be matched to the standard field "Education Background - Bachelor", and "UI design" and "user interface design" will both be matched to "Skill - UI Design".
[0068] For unknown content such as emerging terms, industry jargon, and personalized colloquial expressions not covered by the knowledge base in the resume recognition result, zero-shot recognition is performed relying on the pre-trained semantic knowledge of the large model. Based on the general semantic common sense, industry cognition and context understanding ability learned in the pre-training stage, the large model performs semantic inference and field classification on unfamiliar expressions. For example, the colloquial expression "writing code" will be semantically determined as "Skill - Programming and Development", and the emerging position "growth hacker" will be matched to "Position - User Growth Operation", which can be recognized without pre-entry into the knowledge base, and then the recognition result is determined as the corresponding standardized expression.
[0069] Optionally, when acquiring resume data, constraints can be enhanced by combining the "data format" dimension of the preset knowledge base. For example, the standard field "phone number" corresponds to an 11-digit number format, and "email" corresponds to an email format containing @. Even if the context is not standard, the field type can be determined by the format features, reducing the misjudgment rate.
[0070] Optionally, a rule engine can be introduced to perform secondary validation on the parsing results of the large language model (the standard expressions output by the large language model). Specifically, for fields with fixed formats such as ID card numbers, mobile phone numbers, and email addresses, regular expressions are used to verify the correctness of the format; for logically related fields such as years of work experience and education level, the logical rationality is verified to ensure the accuracy of the parsing results: the rule engine verifies the format validity, performs logical validation based on the four-dimensional knowledge graph, and corrects based on user feedback. Fields with parsing errors are automatically added to the "synonymous expressions" dimension of the four-dimensional knowledge graph, automatically adapting for subsequent batches of parsing, achieving iterative optimization of the model. The four-dimensional knowledge graph uses the field entities of the four-dimensional knowledge base as basic nodes, supplementing them with hierarchical relationships, logical constraints, and business rules between nodes, upgrading "static attributes" into a "reasonable relational network."
[0071] In one embodiment, the content for logical verification based on a four-dimensional knowledge graph can be: ① Timeline logic verification: Verify the chronological order and causal plausibility: Educational experience dimension: verify that the enrollment time is earlier than the graduation time, the duration of a single degree is in line with the regular school system (such as 4 years for undergraduate and 2-3 years for master's), and the time of multiple degrees does not overlap and the hierarchical order is reasonable (such as not having the reverse logic of "going to high school after graduating from undergraduate").
[0072] Work experience dimension: Verify that there is no serious overlap in the time of multiple work experiences and that the start date is earlier than the end date; at the same time, automatically calculate the cumulative years of work experience based on the start and end dates of all work experiences and compare it with the "Years of Work Experience" field separately marked in the resume. If the deviation exceeds the threshold, it is marked as abnormal.
[0073] Cross-dimensional verification: Combine graduation time with current age to deduce a reasonable range of years of work experience. For example, undergraduate graduates are generally around 22 years old. If the parsed years of work experience significantly exceed the reasonable range of "current age - graduation age", it is judged as a logical anomaly.
[0074] ② Hierarchical and attribute logic validation: Based on the hierarchical relationships of entities in the knowledge graph, verify the reasonableness of the matching of field attributes: Educational level: Verify that the order of educational level fields conforms to common sense (Doctor > Master > Bachelor > Associate's) to avoid level contradictions.
[0075] Job title and grade: Combine industry knowledge to verify the match between job title and grade. For example, the "intern" grade will not correspond to a senior management position such as "technical director".
[0076] ③ Field association consistency check: Verify whether the semantics of strongly related fields match, such as the attribute matching of "graduating institution" and "education level", the semantic relevance of "employer" and "industry", and the matching degree of "position" and "core skills". If the semantic deviation of the related fields is too large, mark it as a low confidence result.
[0077] Optionally, the ETL processing of resume data based on the verification results includes: obtaining the analytical confidence score and the recognition confidence score output by the target recognition model based on the verification results; and performing ETL processing on the resume data based on the analytical confidence score and the recognition confidence score. The ETL processing can employ a "smart cleaning-dynamic transformation-incremental loading" mechanism. Intelligent data cleaning: Based on the verification feedback of the parsing results, it automatically processes duplicate data, invalid data, and abnormal data; and converts them into a unified data format.
[0078] Dynamic transformation: The data structure of resume data is dynamically adjusted according to the schema of the target database and subsequent statistical needs, and related indexes of core fields are established; parallel transformation of batch data is supported to improve processing efficiency.
[0079] Incremental loading: An incremental loading mechanism is adopted, which loads only the newly added or modified resume data into the target database to avoid the waste of resources caused by full loading; during the loading process, data integrity is monitored in real time, and automatic rollback and alarm are triggered when an anomaly occurs to ensure data consistency.
[0080] In one embodiment, the schema of the target database refers to the structural specifications of the target business tables where the resume structured data is implemented. Specifically, it includes: field names, data types, length constraints, primary / foreign key definitions, inter-table relationships, enumeration value specifications, etc. For example, the schema of the main resume table in the talent pool would explicitly define the name field as a string type, the mobile phone number as an 11-digit fixed-length string, and the work experience sub-table as associated with the main table through the resume ID.
[0081] Statistical requirements include: ①Talent profiling: such as the distribution of educational background, years of work experience, proportion of different levels of educational institutions, geographical distribution, and coverage of skill tags, to support the overall structure analysis of the talent pool and the formulation of recruitment strategies; ② Business Screening and Retrieval: Supports the recruitment system to quickly screen resumes by skills, industry experience, job level, desired salary, and other dimensions; ③ Correlation analysis needs: such as correlation analysis of skills and job matching, education and onboarding pass rate, and work experience and salary level, for talent quality assessment and recruitment effectiveness review; ④ System operation and maintenance statistics requirements: such as statistics on the field completeness rate and parsing accuracy rate of resumes from different channels, used for monitoring module performance and iterative optimization.
[0082] The data structure of resume data can be dynamically adjusted through a metadata mapping mechanism. The specific steps are as follows: ① Dynamic configuration of field mapping The system establishes a mapping configuration between the standard resume fields output by parsing and the schema fields in the target database, supporting field renaming, deletion, and merging. For example, if the target table in system A defines "graduation institution" as `school` and system B defines it as `edu_org`, only the mapping configuration needs to be modified to complete the adaptation, without changing the parsing and conversion logic. When adding custom fields, the mapping relationship can also be directly supplemented through configuration without refactoring the entire process.
[0083] ② Automatic data type adaptation Based on the type constraints of the target schema, the system automatically performs format normalization and type conversion. For example, it automatically converts string-formatted dates to the DATE type in the target database, converts text-based work years to numeric types, and performs fallback formatting for non-standard values to ensure that the data fully conforms to the type constraints of the target database.
[0084] Optionally, the core field's association indexes include logical association indexes and physical association indexes. Logical association indexes include: primary key association indexes and physical association indexes. Primary key association indexes: Using the unique ID of each resume as the core primary key, foreign key relationships are established for sub-data such as work experience, education experience, project experience, and skill tags, forming a one-to-one correspondence index relationship between the "main table" and "sub-tables." This ensures that all fields of the same resume can be quickly associated through the ID, avoiding misalignment of multiple experiences. Business association indexes: Based on the field association rules of a four-dimensional knowledge graph, logical mapping indexes are constructed between fields, such as the association mapping between "skill tags" and "job categories" or "industry classifications" and "employers." These indexes can be directly called when performing association statistics later. For physical association indexes, indexes can be created for the name and phone number fields extracted from the resume.
[0085] Optionally, the resolution confidence level and the identification confidence level can be used as the basis for ETL scheduling. The core confidence level threshold is manually determined based on business error tolerance and industry-standard practices: for example, a 95% high identification confidence level corresponds to the business standard of "direct entry into the database without manual review," while an 80% low confidence level threshold corresponds to the risk threshold of "manual verification must be triggered." The threshold can be flexibly adjusted according to data quality requirements. The threshold values are calibrated using historical batch data: optimization is performed based on the identification error distribution and resolution error rate of real resumes to ensure that the error rate in the high confidence range after grading is controlled within an acceptable business range, avoiding threshold settings that are out of touch with reality.
[0086] In one embodiment, the recognition confidence score is the text recognition confidence score output by the target recognition model. A single-field confidence score ≥95% is considered high confidence, 80%-95% is medium confidence, and <80% is low confidence. This recognition confidence score is obtained by aggregating the classification probabilities layer by layer, corresponding to the final recognition confidence score after fusing text, image, and layout modalities. This recognition confidence score includes character-level confidence and field-level confidence. Specifically, when the model classifies each character, it outputs the probability distribution of all candidate characters corresponding to that character. The probability value of the highest category is the character-level confidence score (value 0-100%). This probability integrates visual features such as character grayscale and edge texture with contextual semantic features and is the smallest computational unit for recognition confidence. After determining the character-level confidence score, all character-level confidence scores contained in a single field are aggregated using a weighted average and weakest link correction method to obtain the field-level recognition confidence score corresponding to that single field. The calculation formula for this field-level confidence score can be: Field-level confidence level = 0.7 The mean confidence level for each character within the field is +0.3. Lowest character-level confidence level within the field Optionally, the parsing confidence can be calculated based on field matching similarity and logical verification pass rate. ≥90% is considered high confidence, 70%-90% medium confidence, and <70% low confidence. The field matching similarity is calculated by converting the text fragment to be parsed and all synonyms of the standard fields in the four-dimensional knowledge base into semantic vectors of a unified dimension, transforming the text semantics into quantifiable numerical vectors. Using a cosine similarity algorithm, the cosine of the spatial angle between the text vector to be parsed and the candidate standard field vectors is calculated. This cosine value is determined as the field matching similarity, which ranges from 0 to 100%, with higher values indicating closer semantic similarity. The logical verification pass rate corresponds to the rule engine of the parsing result verification module and the logical verification system of the four-dimensional knowledge graph. It is calculated by, for a single field or a single experience entry, the percentage of valid logical verification rules that pass. Logical validation pass rate = (Number of rules that passed validation / Total number of validation rules associated with this field) 100% The confidence score can be calculated using a weighted linear summation model, with field matching similarity as the core criterion, logical validation pass rate as a constraint correction, and format validation as a strong constraint coefficient. The calculation formula is as follows: Single-field parsing confidence = (field matching similarity) W1+ logic verification pass rate W2) Format verification coefficient Weight coefficients W1 and W2 are configurable. In typical business scenarios, they are configured as W1=0.6 and W2=0.4, meaning semantic similarity is the core criterion, with logical validation constraining reasonableness. For scenarios with high requirements for logical consistency, the value of W2 can be increased. Format validation coefficient: For fields with fixed formats such as phone numbers, email addresses, and ID card numbers, the coefficient is 1.0 when the format is compliant and 0.5 when the format is non-compliant; for fields without fixed formats such as names and company names, the coefficient is 1.0.
[0087] Optionally, an ETL processing strategy can be automatically matched based on the "dual confidence" combination: For high recognition confidence and high resolution confidence: a "fast track" is used. Deduplication, format standardization, and incremental loading are performed automatically without manual intervention, and this process has the highest priority. For the identification confidence level and the parsing confidence level: the "standard channel" is used. A "field cross-validation" step is added to automatically correct minor formatting errors; if no correction is possible, the error is marked and processing continues. Use regular expressions related to field semantics for matching, and call custom rule function code for processing, for example: The mobile phone number field is identified as a 12-digit number (medium confidence, one more digit is identified). Combining the format rule of "11-digit mobile phone number" and referring to the semantic label of the field as "contact number", redundant interference digits are removed and corrected to the standard 11-digit format. The email field identifies missing @ symbols (medium confidence level) and, combining standard email format rules and field semantics, completes the symbols.
[0088] If the work experience field has a medium confidence level (e.g., identified as 8 years), but the start and end dates of each work experience are all high confidence levels, and the cumulative actual work experience is 5 years, then the cumulative value is used to correct the work experience field. When the ID number is identified as having high confidence, the birth date and age fields are extracted and cross-validated to correct the recognition error of the birth date; If the verification finds that the graduation time is later than the start date of the first job, and one of the fields has a medium confidence level, the time deviation will be automatically corrected by combining rules such as the length of schooling and the typical graduation age. If it cannot be determined, it will be marked as abnormal.
[0089] For low identification confidence and low resolution confidence: a "manual intervention channel" is used. The processing of this part of the data is paused, and the data is pushed to the manual review interface in real time, showing the specific reasons for the low confidence. The data is then manually corrected before being reintegrated into the ETL process, avoiding batch rework. Mixed confidence level: Employs "field-level precise processing". Only low-confidence fields undergo enhanced cleaning / validation, while high-confidence fields are processed directly, saving computational resources.
[0090] The system can be customized to perform ETL processing using a rule engine. This engine allows users to customize "field transformation rule templates" based on their statistical needs. The rule templates are stored in the rule library and can be dynamically called and updated.
[0091] S103: Process resumes based on resume data.
[0092] Optionally, resume processing includes at least one of the following: report generation, data statistics, and polishing suggestion generation. Specifically, based on the structured data processed by ETL, multi-dimensional statistics and visualization report generation for batch resumes can be achieved.
[0093] Optionally, in multi-dimensional statistical analysis, statistics can be performed according to dimensions such as resume format, education level distribution, years of work experience, job preferences, and skill distribution. Batch comparison analysis functions are provided, such as comparing the quality of resumes from different recruitment batches. Specifically, the statistical analysis content includes: Identification and Confidence Analysis: The percentage of high, medium, and low recognition confidence in different batches; the average field recognition confidence; and the percentage of high, medium, and low parsing confidence.
[0094] Completeness rate of core fields: The completeness rate of information in core fields such as name, mobile phone number, email, education, work experience, and skills, as well as the average field completeness of a single resume.
[0095] Validation pass rate: Format validation pass rate (compliance rate of mobile phone numbers, email addresses, etc.), logic validation pass rate (logic compliance rate of timelines, educational levels, etc.), percentage of automatically corrected fields, and percentage of anomalies requiring manual review.
[0096] Basic qualification level: Educational background distribution (percentage of doctoral, master's, bachelor's, and associate's degree holders), percentage of universities in different tiers (985, 211, Double First-Class, and ordinary universities), average years of work experience, distribution of years of work experience range, and age distribution.
[0097] Optionally, various report templates can be built-in, and users can customize report fields and formats; the generated reports can be exported to Excel, PDF and other formats, and online visualization is also supported.
[0098] Optionally, personalized polishing suggestions can be generated based on the parsing results and industry standards for high-quality resumes. The parsing results after ETL processing may include: ① Full set of structured field data: It includes all resume fields after parsing and verification, namely personal basic information, education experience (school, major, degree, time), work experience (company, position, start and end dates, responsibilities, achievements), project experience, skill tags, certificates and qualifications, self-evaluation and other complete structured content, and also includes layout information (column structure, title level, module order).
[0099] ② Data Quality Diagnostic Labels: Output quality judgment results, including field completeness rate, list of low-confidence fields, logical verification anomalies (such as overlapping timelines, contradictory educational levels), and non-standard format items (such as incorrect number of digits in mobile phone numbers, inconsistent date formats), which serve as the basis for locating issues that must be corrected.
[0100] ③ Semantic deep analysis tags: The semantic judgment results of the parsing output include job level judgment, industry classification, skill proficiency level, responsibilities of experience, proportion of achievements, and degree of colloquialism in expression.
[0101] The criteria for a high-quality resume include: ① General Basic Standards: Resume writing guidelines applicable to all positions, including requirements for the completeness of core fields, reverse chronological order and standardized format, principle of objectivity, and no typos or formatting errors, as well as corresponding basic error correction and polishing.
[0102] ② Content Expression Standards: High-quality resumes follow the STAR method for content writing, such as prioritizing quantitative presentation of achievements, using professional verbs to start job descriptions, avoiding colloquialisms and redundant expressions, and placing key information first, with corresponding expression optimization and polishing.
[0103] ③ Industry-specific standards: Differentiated standards for different industries and positions. For example, technical positions focus on the depth of the technology stack, project complexity, and technical achievements; operations positions focus on data indicators, growth effects, and business value; and functional positions focus on process optimization, cost control, and cross-departmental collaboration results, with corresponding industry-specific refinements.
[0104] ④ Structural layout standards: Layout specifications based on layout characteristics, such as module priority ordering, clear information hierarchy, reasonable length control, and visual highlighting of key content, with corresponding layout optimization suggestions.
[0105] Optionally, polishing suggestions can be generated based on a large language model. This involves a collaborative process of rule-based problem identification, structured Prompt (hint word) constraint generation, and rule-based fallback verification. The specific implementation logic is as follows: ① First, through a rule engine and lightweight semantic computation, accurately pinpoint the specific issues in the resume that do not meet the high-quality standards, avoiding the large model from being ineffective: Hard issues: These are determined directly by the rule engine and include missing core fields, format errors, timeline logic anomalies, and non-standard contact information, and are classified as items that must be corrected. Soft issues: These are identified through semantic similarity, keyword ratio, and sentence structure analysis. They include issues such as lack of quantifiable results in experience descriptions, colloquial language, vague job descriptions, and low matching degree between skills and target positions. These issues are classified as optimization and improvement items.
[0106] Once the location is determined, a list of issues is generated, including module affiliation, issue type, and original issue text, which serves as the core input parameter for Prompt.
[0107] ② Structured Prompt Construction: A fixed template and dynamic parameter injection Prompt engineering solution is adopted. Each resume module (work experience, skills, self-evaluation, etc.) corresponds to an independent template to ensure that the generated results are focused, standardized and applicable.
[0108] ③ Module-based generation and invocation strategy: Suggestions are generated for each module based on dimensions such as work experience, education experience, skills modules, and self-evaluation, avoiding information omissions and deviations caused by excessively long inputs in a single session; tiered generation is used for items that must be corrected and items that need to be optimized: corrected items provide clear directions for modification and standard examples, while optimization items provide optional optimization solutions, balancing rigor and flexibility.
[0109] ④ Rule fallback verification after generation Before the large model outputs results, a final fallback check is performed using a rule engine to ensure that the recommendations are compliant and credible. Authenticity Verification: Verify whether the suggested content is based on the existing information in the resume, prohibit the generation of experiences, data and skills that do not exist in the resume, and eliminate "false embellishment"; Compliance verification: Filtering out exaggerated or non-compliant statements to ensure that the advice is professional and compliant; Relevance verification: Ensure that the recommendations strictly correspond to the previously identified problem points, eliminate irrelevant generalities, and guarantee the specificity of the recommendations.
[0110] In one embodiment, the Prompt core includes: 1. Role and Boundary Setting: Clearly define the role of the large language model as "senior recruitment expert and resume optimization consultant", familiar with the recruitment standards of the corresponding industry, and define the boundaries: only make optimization suggestions based on the existing content of the resume, and do not fabricate non-existent experiences, skills and achievements.
[0111] 2. Resume Context Injection: Dynamically inject the original content of the current module, the industry and target position of the resume, and parsed semantic tags (such as the current job level and core skills) to allow the large model to generate suggestions based on the real context.
[0112] 3. High-quality standard constraints: Inject high-quality resume standard items corresponding to the scenario, such as "work experience should follow the STAR method and highlight quantifiable business results" and "skill tags should be classified by proficiency and prioritize core job skills" as the basis for evaluation.
[0113] 4. Define a list of questions: List the specific questions for the preliminary positioning one by one, such as "This work experience lacks quantitative data support" or "The expression is too colloquial and does not start with professional verbs", so that the big model can answer in a targeted manner and avoid generalities.
[0114] 5. Output format requirements: Constrain the output structure, for example, "provide 2-3 optimization suggestions, each suggestion is accompanied by a rewritten example sentence, the language is concise and professional, and not redundant", to ensure a consistent output format and facilitate front-end display.
[0115] Optionally, a resume completeness assessment can be performed based on the resume data. This assessment includes scoring the completeness and clarity of core fields, and identifying missing fields or ambiguous descriptions. Targeted polishing suggestions can be generated based on the specific requirements of the target position to optimize the wording and enhance the resume's persuasiveness.
[0116] Optionally, precise optimization can be achieved through "job competency modeling + defect quantification diagnosis + personalized polishing". Specifically, a structured competency model (target job competency model) is constructed using the job title as an index. This model corresponds to three dimensions: "skills", "experience requirements", and "achievement indicators". Resume defect quantification diagnosis includes: The parsed resume data is matched with the target job competency model in a "dimensional matching" process, and the "matching score" (0-100 points) for each dimension is calculated. Identify dimensions with a matching score <60 and designate them as core deficiencies; further analyze the deficiency types. These are categorized as follows: Missing: The skill is not mentioned; Vague: Mentioned but without specific description; Inefficient: The description lacks quantifiable results. Based on the deficiency type, generate a "Deficiency Quantification Report" and personalized quantitative refinement suggestions: for example, for missing deficiencies, generate "supplementary directions and examples"; for vague deficiencies, generate "quantitative optimization suggestions"; for inefficient deficiencies, generate "keyword enhancement suggestions". Compared with the prior art, this application has the following beneficial effects: Breaking through the format adaptation limitations of traditional OCR technology, it improves the recognition accuracy and efficiency of resumes in complex formats, and achieves full compatibility recognition of resumes in multiple formats such as images, PDFs, and Word documents.
[0117] Improve the parsing accuracy of core resume fields, achieve intelligent adaptation and context-based parsing of non-standard expressions, and output structured and standardized parsing results.
[0118] We have established a collaborative processing mechanism that integrates identification, parsing, ETL, statistics, and application to reduce manual intervention, shorten data processing cycles, and ensure the consistency and availability of batch resume data.
[0119] A new intelligent resume polishing suggestion function has been added, which generates personalized optimization solutions based on the analysis results and industry standards, improves the functional loop, and enhances the added value of resume data.
[0120] Based on the same inventive concept, this application also proposes an electronic device, please refer to [link to relevant documentation]. Figure 3 , Figure 3 This is a structural diagram of an electronic device provided in an embodiment of this application. (In conjunction with...) Figure 3 The electronic device described in this application is described in detail.
[0121] In this embodiment, the smart terminal includes a processor and a memory. The processor and the memory are communicatively connected. The memory stores program data, which is used to execute the method described in the above embodiment.
[0122] In this embodiment, the processor is the control center of the smart terminal. It connects various parts of the smart terminal via various interfaces and lines, and performs various functions and processes data by running or executing software programs and / or modules stored in the memory, and by calling data stored in the memory, thereby providing overall monitoring of the smart terminal. Optionally, the processor may include one or more processing units; preferably, the processor may integrate an application processor and a modem processor, wherein the application processor mainly handles the operating system, user interface, and applications, and the modem processor mainly handles wireless communication. It is understood that the modem processor may also not be integrated into the processor.
[0123] The smart terminal also includes a power supply (such as a battery) to power various components. Preferably, the power supply can be connected to the processor logic through a power management system, thereby enabling functions such as charging, discharging, and power consumption management through the power management system.
[0124] The memory can be used to store software programs and modules. The processor executes various functional applications and data processing of the smart terminal by running the software programs and modules stored in the memory. The memory can mainly include a program storage area and a data storage area. The program storage area can store the operating system, application programs required for at least one function (such as sound playback function, image playback function, etc.), etc.; the data storage area can store data created according to the use of the smart terminal (such as audio data, phone book, etc.). In addition, the memory can include high-speed random access memory, and can also include non-volatile memory, such as at least one disk storage device, flash memory device, or other volatile solid-state storage device.
[0125] Based on the same inventive concept, this application also proposes a computer-readable storage medium, please refer to [link to relevant documentation]. Figure 4 , Figure 4 This is a structural diagram of an embodiment of the computer-readable storage medium of this application, in conjunction with... Figure 4 The computer-readable storage medium of this application is described.
[0126] In this embodiment, a computer-readable storage medium stores a computer program that is used to perform the methods described in the above embodiments.
[0127] The computer-readable storage medium may include, but is not limited to, floppy disks, optical disks, CD-ROMs (compact disc-read-only memory), magneto-optical disks, ROMs (read-only memory), RAMs (random access memory), EPROMs (erasable programmable read-only memory), EEPROMs (electrically erasable programmable read-only memory), magnetic cards or optical cards, flash memory, or other types of media / machine-readable media suitable for storing machine-executable natural language instructions. The computer-readable storage medium may be a product not connected to a computer device or a component used in a computer device.
[0128] The terms "first," "second," "third," "fourth," "1," "2," etc. (if present) in the specification, claims, and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of this application described herein can be implemented in a sequence other than that shown in the illustrations or text descriptions.
[0129] It should be understood that although arrows indicate various operation steps in the flowcharts of this application's embodiments, the order in which these steps are implemented is not limited to the order indicated by the arrows. Unless explicitly stated herein, in some implementation scenarios of this application's embodiments, the implementation steps in each flowchart can be executed in other orders as required. Furthermore, some or all steps in each flowchart, based on the actual implementation scenario, may include multiple sub-steps or multiple stages. Some or all of these sub-steps or stages can be executed at the same time, and each sub-step or stage can also be executed at different times. In scenarios where execution times differ, the execution order of these sub-steps or stages can be flexibly configured according to requirements, and this application's embodiments do not limit this.
[0130] The above description is only an optional implementation method for some implementation scenarios of this application. It should be noted that for those skilled in the art, other similar implementation methods based on the technical concept of this application without departing from the technical concept of this application also fall within the protection scope of the embodiments of this application.
Claims
1. A multimodal fusion resume processing method, characterized in that, The method includes: Extract multimodal features from resume files, and obtain resume recognition results based on the multimodal features and a pre-trained target recognition model. The multimodal features include text features, image features, and layout features. The target recognition model is trained based on the labeled resume and the corresponding multimodal features. Resume data is extracted from the resume recognition results based on a preset knowledge base and a large language model. The preset knowledge base is built based on multidimensional knowledge, which includes field names, synonyms, data formats, and related fields. The resume data is processed, and the resume processing includes at least one of generating reports, data statistics, and generating polishing suggestions.
2. The multimodal fusion resume processing method according to claim 1, characterized in that, The extraction of multimodal features from resume files includes: Obtain the resume file, call the extraction model to extract the multimodal features in parallel, and use different extraction models for different multimodal features.
3. The multimodal fusion resume processing method according to claim 1, characterized in that, The target recognition model includes a teacher model and a student model. The acquisition of the target recognition model includes: Annotate historical resumes, including character-level text annotations, layout area-level annotations, field entity-level annotations, and visual attribute-level annotations; A multimodal backbone network is trained using labeled historical resumes to obtain a teacher model. The loss function of the teacher model is generated based on the loss of the text recognition branch, the loss of the layout detection branch, the loss of the entity extraction branch, and the cross-modal loss in the multimodal backbone network. The teacher model is distilled to obtain the student model.
4. The multimodal fusion resume processing method according to claim 3, characterized in that, The distillation teacher model includes: Obtain the soft label probability distribution and intermediate layer feature distribution corresponding to the teacher model; The soft label distribution and corresponding hard label are used to constrain the output layer of the student model, and feature alignment constraints are applied to the multimodal fusion feature layers of the teacher model and the student model based on the intermediate layer feature distribution.
5. The multimodal fusion resume processing method according to claim 3, characterized in that, The step of obtaining resume recognition results based on the multimodal features and the pre-trained target recognition model includes: The resume file and the multimodal features are input into the student model to obtain the output results of the student model. The output results include the recognition results of each region in the resume file and the confidence scores of each region. The resume recognition results are obtained based on the confidence level and layout features of each region.
6. The resume processing method based on multimodal fusion according to claim 5, characterized in that, The process of obtaining the resume recognition result based on the confidence level and layout features of each region includes: It is determined that the confidence level is less than a preset threshold and the layout feature is a core field; A local multimodal feature package is generated based on the local image, text features, and layout coordinates of the region corresponding to the confidence level. The region recognition result corresponding to the local multimodal feature package is obtained using the teacher model, and a resume recognition result is generated based on the region recognition result and the output result of the student model.
7. The multimodal fusion resume processing method according to claim 1, wherein extracting resume data from the resume recognition result based on a preset knowledge base and a large language model includes: The resume recognition results are semantically understood and processed using a large language model to obtain text fragments. The semantic understanding and processing includes item boundary segmentation, semantic role determination, and association completion. The text fragments are processed by entity linking and standardized expression based on the preset knowledge base to obtain the resume data. Verify the resume data and perform ETL processing on the resume data based on the verification result.
8. The multimodal fusion resume processing method according to claim 7, wherein the ETL processing of the resume data based on the verification result includes: Based on the verification results, obtain the analytical confidence level and the recognition confidence level output by the target recognition model; The resume data is processed using ETL based on the resolution confidence and the identification confidence.
9. An electronic device, characterized in that, The method includes a processor and a memory, the processor being communicatively connected to the memory, the memory storing program data, and the program data being used to execute the method as described in any one of claims 1-8.
10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that is used to perform the method as described in any one of claims 1-8.
