Data processing method based on multi-source interview data aggregation and related equipment

Abstract

The application provides a data processing method based on multi-source interview data aggregation and related equipment, and relates to the technical field of text information processing. In the method, in response to an interview session instruction created by a user, intent recognition is performed on a session purpose to obtain a session intent, and a search request is generated using the session intent and an interviewee. Based on the associated business data of each search and the session intent, a structured session outline is generated. After the interview session is started based on the structured session outline, a multi-modal input stream in the session process is obtained, the multi-modal input stream is converted into a linear text stream, and the linear text stream is mapped to a corresponding hierarchical node of the structured session outline to form a structured session record. Key information extraction is performed on the structured session record, and the extracted key information is respectively written into a heterogeneous data source according to the category of the key information to update the associated business data, so that automatic arrangement, structured recording and archiving of multi-source interview materials can be realized, and a data closed loop is formed.

Description

Technical Field

[0001] This application relates to the field of text information processing technology, specifically to a data processing method and related equipment based on the aggregation of multi-source interview data. Background Technology

[0002] In modern enterprise management, there are numerous dialogue and interview scenarios, such as performance reviews, one-on-one communication, promotion interviews, and exit interviews. The content of these interviews and dialogues is crucial for enterprise management and employee development. Currently, these interviews and dialogues are often recorded via voice or manually written text summaries. These recording methods have significant drawbacks. Taking performance reviews as an example, which are conducted annually or quarterly, managers need to spend considerable time and effort manually compiling data on the interviewee's historical performance, past interview commitments, and current project progress. This data exists in different heterogeneous systems, lacking automatic organization of employee data. Furthermore, the records generated during the interview cannot be aligned with the interview objectives in real time, leading to a shift in focus. Post-interview records are stored in static documents, making it impossible to extract structured information from unstructured data for data processing.

[0003] Therefore, there is an urgent need for a data processing method that can connect heterogeneous data sources and achieve intelligent aggregation before the meeting, real-time structuring during the meeting, and automatic closed-loop processing after the meeting. Summary of the Invention

[0004] To achieve the above objectives, this application provides a data processing method and related equipment based on multi-source interview data aggregation, which can realize the automatic organization, structured recording and archiving of multi-source interview data, and realize a complete closed loop of data processing.

[0005] The technical solution of this application embodiment is as follows: In a first aspect, embodiments of this application provide a data processing method based on multi-source interview data aggregation, the method comprising: In response to a user-created interview session instruction, the interview session instruction includes a target object and a session purpose, the target object including the interviewee; The purpose of the conversation is identified to obtain the conversation intent, and a retrieval request is generated using the conversation intent and the interviewee. The retrieval request is used to retrieve the associated business data corresponding to the interviewee from multiple preset heterogeneous data sources. Based on the associated business data and the session intent, a structured session outline is generated; After the interview session based on the structured conversation outline is initiated, the multimodal input stream during the session is obtained. The multimodal input stream includes at least a voice data stream. The multimodal input stream is converted into a linear text stream and the linear text stream is mapped to the corresponding hierarchical nodes of the structured conversation outline to form a structured conversation record. Key information is extracted from the structured session records and written into the heterogeneous data source according to the category of the extracted key information, so as to update the related business data and form a data closed loop.

[0006] In the above technical solution, the system first responds to the user-created interview session instruction. This instruction includes the target audience and the purpose of the interview. The target audience includes the interviewee. By constructing the interview session instruction, the interviewee is identified, providing support for obtaining accurate data subsequently. The purpose of the interview is then identified through intent recognition. This intent recognition reveals the purpose of the interview, allowing for targeted extraction of heterogeneous multi-source data. A retrieval request is generated using the interview intent and the interviewee. This retrieval request retrieves relevant business data corresponding to the interviewee from multiple preset heterogeneous data sources. This retrieval request defines the search scope, avoids interference from invalid data, and also extracts heterogeneous multi-source data, achieving pre-meeting data aggregation. Based on... Each related business data point and conversation intent is used to generate a structured conversation outline, preparing for automated structured processing during the meeting. After the interview session starts based on the structured conversation outline, the multimodal input stream during the session is acquired. This multimodal input stream includes at least a voice data stream. The multimodal input stream is converted into a linear text stream and mapped to the corresponding hierarchical nodes of the structured conversation outline, forming a structured conversation record. Using the structured conversation outline as a medium, the record can be aligned with the interview objectives in real time, preventing focus deviation. Key information is extracted from the structured conversation record and written to heterogeneous data sources according to the categories of extracted key information to update related business data, forming a data loop and achieving closed-loop processing of document recording and write-back. This process, through automated structured processing of interview materials, helps improve interview efficiency.

[0007] In some embodiments of this application, generating a structured session outline based on the associated business data and the session intent includes: The various related business data are aggregated to obtain multi-source aggregated data; Based on the session intent, a corresponding preset outline template is loaded. The outline template includes multi-level nodes, and each level node is used to indicate different session module topics corresponding to the session intent. The multi-source aggregated data is extracted using a pre-defined text analysis model to obtain key information points and supporting data related to the conversation intent. The key information points and supporting data are filled into the corresponding nodes of the outline template to generate the structured session outline.

[0008] In some embodiments of this application, the associated business data includes human resources system data, historical interview data, project management data, and data from collaboration tools; The aggregation of the various related business data to obtain multi-source aggregated data includes: Entity identification and relationship extraction are performed on the data from the human resources system, the historical interview data, the project management data, and the data from the collaboration tools to construct a conversation knowledge graph of the interviewee; Using the conversation intent, extract the conversation subgraph related to the conversation intent from the conversation knowledge graph to obtain the first intent association data; Create a timeline, using the HR system data, historical interview data, project management data, and collaboration tool data as event cards, anchoring them to the timeline according to the time nodes, and labeling each event card at the same time node to obtain an event line; The conversation knowledge graph and the event line are used to perform causal association analysis on the conversation intent to obtain second intent association data; The first intent-related data and the second intent-related data are fused to obtain the multi-source aggregated data.

[0009] In some embodiments of this application, mapping the linear text stream to the corresponding hierarchical nodes of the structured conversation outline to form a structured conversation record includes: Within a preset context window, the similarity between the linear text stream and the corresponding level nodes of the structured conversation outline is calculated in real time to obtain the similarity value of each level node. The text segments of the linear text stream corresponding to the similarity value greater than the preset similarity threshold are filled under the hierarchical node; In response to the user's action to end the interview, all text segments mapped to nodes at each level are collected to form the structured session record.

[0010] In some embodiments of this application, the step of extracting key information from the structured session records and writing it into the heterogeneous data source according to the category of the extracted key information includes: Multi-dimensional key information is extracted from the structured session records to obtain multi-dimensional structured information; Using preset mapping rules, the multi-dimensional structured information is written into the corresponding heterogeneous data source, wherein the mapping rules include the correspondence between multiple information categories and heterogeneous data sources.

[0011] In some embodiments of this application, the step of extracting key information from the structured session record in multiple dimensions to obtain multi-dimensional structured information includes: The structured session record is divided into each round of dialogue state segments, and a corresponding state label is generated for each round of dialogue state segments, wherein the state label corresponds to the purpose of the session; Named entity recognition and relation extraction are performed on the structured session records to obtain entities and the relationships between them; The status labels, the association relationships, and the dialogue state fragments recorded in the structured session are fused to obtain fused data; The fused data is classified using a pre-defined multi-label intent classifier to obtain the multi-dimensional structured information.

[0012] In some embodiments of this application, the multimodal input stream further includes an interview summary; The step of converting the multimodal input stream into a linear text stream includes: The voice data stream in the multimodal input stream is divided using voiceprint recognition technology to obtain the voice segment of the interviewee and the voice segment of the interviewee. The speech segment of the interviewee is converted into the interview text segment, and the speech segment of the interviewee is converted into the interview text segment. Add timestamps to both the interview text segment and the interviewee text segment, and arrange them in the order of the timestamps to obtain a text segment stream; Obtain the input interview summary, and combine the interview summary and the text segment stream into the linear text stream.

[0013] In a second aspect, this application provides an electronic device including one or more processors and a memory; the memory is coupled to the one or more processors, the memory being used to store computer program code, the computer program code including computer instructions, and the one or more processors invoking the computer instructions to cause the electronic device to perform the method described in the first aspect and any possible implementation thereof.

[0014] Thirdly, this application provides a computer program product containing instructions that, when the computer program product is run on an electronic device, cause the electronic device to perform the method described in the first aspect and any possible implementation thereof.

[0015] Fourthly, this application provides a computer-readable storage medium including instructions that, when executed on an electronic device, cause the electronic device to perform the method described in the first aspect and any possible implementation thereof.

[0016] In summary, one or more technical solutions provided in the embodiments of this application have at least the following technical effects or advantages: 1. Because it adopts a response-first approach to user-created interview session instructions, these instructions include the target audience and the session purpose. The target audience includes the interviewee. Constructing the interview session instructions identifies the interviewee, supporting the subsequent acquisition of accurate data. Intent recognition is performed on the session purpose to obtain the session intent. Intent recognition reflects the purpose of this session, enabling targeted extraction of heterogeneous multi-source data. A retrieval request is generated using the session intent and the interviewee. This retrieval request retrieves relevant business data corresponding to the interviewee from multiple preset heterogeneous data sources. This retrieval request defines the search scope, avoids interference from invalid data, and also extracts heterogeneous multi-source data, achieving pre-meeting data aggregation. Based on various... The system generates a structured conversation outline from related business data and conversation intent, preparing for automated structured processing during the meeting. After the interview session begins based on this structured outline, it acquires the multimodal input streams, which include at least voice data streams. These multimodal input streams are converted into linear text streams and mapped to corresponding hierarchical nodes in the structured conversation outline, forming a structured conversation record. Using the structured conversation outline as a medium, the system can align with the interview objectives in real time, preventing focus deviations. Key information is extracted from the structured conversation record and written to heterogeneous data sources according to the categories of extracted key information to update related business data, forming a data loop and achieving closed-loop processing of document recording and write-back. This process, through automated structured processing of interview materials, helps improve interview efficiency. Therefore, it effectively solves the problems of manually searching for multi-source data and the tedious data recording in related technologies.

[0017] 2. By matching the conversation intent with the outline template, the outline template provides the various process parts of the interview, and information is extracted and filled in to form a structured conversation outline. This transforms unstructured data into structured data, improving the efficiency of data processing.

[0018] 3. Aggregate data from various heterogeneous data sources, and use knowledge graphs and timelines to construct and comprehensively mine the relationships between the data, forming multi-source aggregated data. This automates the interview material preparation process and improves data accuracy.

[0019] 4. Extract multi-dimensional relationships from structured conversation records to match multi-dimensional structured information with heterogeneous data sources. Write the conversation records into the heterogeneous data sources to form a data loop, providing a reference for subsequent interview processing. Attached Figure Description

[0020] Figure 1 This is a flowchart illustrating a data processing method based on multi-source interview data aggregation provided in one embodiment of this application; Figure 2 This is a schematic diagram of a heterogeneous data source structure for a data processing method based on multi-source interview data aggregation provided in one embodiment of this application; Figure 3 yes Figure 1 A flowchart illustrating a sub-step of step S300; Figure 4 This is a schematic diagram of the structure of a data processing system based on multi-source interview data aggregation provided in one embodiment of this application; Figure 5 This is a schematic diagram of the structure of an electronic device provided in one embodiment of this application. Detailed Implementation

[0021] To enable those skilled in the art to better understand the technical solutions in this specification, the technical solutions in the embodiments of this specification will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments.

[0022] In the description of the embodiments of this application, the words "for example" or "for instance" are used to indicate examples, illustrations, or explanations. Any embodiment or design that is described as "for example" or "for instance" in the embodiments of this application should not be construed as being more preferred or advantageous than other embodiments or design options. Rather, the use of the words "for example" or "for instance" is intended to present the relevant concepts in a specific manner.

[0023] In the description of the embodiments of this application, the term "multiple" means two or more. For example, multiple systems means two or more systems, and multiple screen terminals means two or more screen terminals. Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the indicated technical features. Thus, a feature defined with "first" or "second" may explicitly or implicitly include one or more of that feature. The terms "comprising," "including," "having," and variations thereof all mean "including but not limited to," unless otherwise specifically emphasized.

[0024] In modern enterprise management, numerous dialogues and interviews occur, such as performance reviews, one-on-one communications, promotion interviews, and exit interviews. The content of these interviews and dialogues serves as crucial evidence for enterprise management and employee development. However, interview records are often incomplete due to cumbersome processes, or they become silos due to being scattered across different managers or systems, resulting in a lack of correlation between records. This significantly limits the ability to extract relevant data from heterogeneous data sources and perform automated structuring to improve interview efficiency.

[0025] Based on this, embodiments of this application provide a data processing method, electronic device, program product, and readable storage medium based on multi-source interview data aggregation. This data processing method first responds to a user-created interview session instruction. The interview session instruction includes a target object and a session purpose. The target object includes the interviewee. By constructing the interview session instruction, the interviewee is determined, providing support for subsequently obtaining accurate data. Intent recognition is performed on the session purpose to obtain the session intent. Intent recognition reflects the purpose of this session, enabling targeted extraction of heterogeneous multi-source data. A retrieval request is generated using the session intent and the interviewee. This retrieval request is used to retrieve related business data corresponding to the interviewee from multiple preset heterogeneous data sources. This retrieval request defines the retrieval scope and avoids invalid data. Data interference can be mitigated by extracting heterogeneous multi-source data, enabling pre-meeting data aggregation. Based on various related business data and conversation intents, a structured conversation outline is generated to prepare for automated structured processing during the meeting. After the interview session based on the structured conversation outline begins, the multimodal input stream during the session is acquired. This multimodal input stream includes at least a voice data stream. The multimodal input stream is converted into a linear text stream and mapped to the corresponding hierarchical nodes of the structured conversation outline, forming a structured conversation record. Using the structured conversation outline as a medium, it can be aligned with the interview objectives in real time, avoiding focus deviation. Key information is extracted from the structured conversation record and written to heterogeneous data sources according to the categories of extracted key information to update related business data, forming a data loop and achieving closed-loop processing of document recording and write-back. The above process, through automated structured processing of interview materials, helps improve interview efficiency.

[0026] It should be noted that this data processing method based on multi-source interview data aggregation can be applied to various scenarios in enterprise management that require interviews. It can realize the automated extraction and processing of interview materials, real-time alignment with interview objectives, and automated archiving, thus achieving a complete closed loop of automated interview data processing. Through the above-mentioned text information data processing process, it can help improve interview efficiency.

[0027] The technical solutions provided in the embodiments of this application will be further described below with reference to the accompanying drawings.

[0028] Reference Figure 1 , Figure 1 This is a flowchart illustrating a data processing method based on multi-source interview data aggregation provided in an embodiment of this application. The data processing method based on multi-source interview data aggregation is executed by a processor in an electronic device or a readable storage medium, and includes steps S100, S200, S300, S400, and S500.

[0029] Step S100: In response to a user-created interview session instruction, the interview session instruction includes the target object and the purpose of the session, and the target object includes the interviewee.

[0030] In one embodiment, the user corresponds to the interviewee in the target object; that is, the user is the interviewee. During each interview session, an instruction to create an interview session is given through the interviewee. This interviewee can be a qualified employee of the company with the authority to create interview sessions. For example, the interview can be initiated by either the interviewee or the interviewee; however, only the interviewee has the authority to create interview sessions. The target object includes both the interviewee and the interviewee. The core entity targeted in this session refers to the personnel participating in the interview. The session purpose refers to the topic type or business intent of this session, such as textual descriptions like "annual performance appraisal intent," "project progress review intent," or "exit interview intent."

[0031] The product provides a graphical user interface. Interviewees can select the interviewee from a dropdown menu, enter or select the interview topic (conversation purpose), and click "Create Interview Session." The front-end encapsulates this information into a JSON request packet and sends it to the electronic device server via HTTPS. The API gateway receives and parses this request, responding to the user-created interview session instruction. Subsequent intent recognition processes are then initiated, and the created interview session instruction can trigger automated data processing steps such as material preparation (implemented through data retrieval and analysis), intent recognition, and text processing. This process digitally initializes the business scenario, establishes the subject and context of data retrieval, and provides an anchor for subsequent precise retrieval.

[0032] Step S200: The purpose of the conversation is identified to obtain the conversation intent, and a retrieval request is generated using the conversation intent and the interviewee. The retrieval request is used to retrieve the relevant business data corresponding to the interviewee from multiple preset heterogeneous data sources.

[0033] In one embodiment, intent recognition refers to the process of mapping input or selection to predefined standard business tags. Heterogeneous data sources refer to multiple business data systems with different data storage structures, access protocols, and data types. They use a MySQL data storage structure to store data and represent multiple data sources within the enterprise, connected to a data processing system based on multi-source interview data aggregation. Specifically, these include: a human resources system, a historical interview database, a project management system, and collaboration tools. The aforementioned heterogeneous data sources are as follows: Figure 2 As shown, the system retrieves related business data through an interface.

[0034] The HR system stores structured data, such as onboarding date, job level, employee historical performance ratings, and training records. The project management system stores semi-structured data, represented by JSON objects, such as task status, records of all past interviews with the interviewee, memos, and historical to-do items. The project management system also stores a list of projects the interviewee was responsible for within a target period (e.g., the third quarter), task completion rates, and time reports. The collaboration tool stores unstructured data (text / logs), such as work documents, communication records, customer feedback, or performance data. The retrieval request is used to retrieve relevant business data corresponding to the interviewee from multiple preset heterogeneous data sources. Relevant business data refers to the data in the aforementioned heterogeneous data sources that corresponds to the interviewee and whose time range or content attributes match the conversation intent.

[0035] First, a pre-trained BERT (Bidirectional Encoder Representations from Transformers) semantic classification model is used to identify the intent of the conversation, outputting a corresponding intent label vector representing the conversation intent. This conversation intent is the foundation for subsequent data processing; only by understanding the conversation intent can the corresponding data be extracted to meet the requirements of accurate material preparation. For example, this intent label vector can represent the label category of the conversation, expressed in JSON format. If the conversation intent is "to discuss last quarter's performance," then the output labels are "performance interview" and "time window." It should be noted that this pre-trained BERT semantic classification model is a pre-trained model, fine-tuned through a large amount of conversation data to achieve generalization to conversation data. The model is an existing technology model; its structure and training process will not be elaborated here.

[0036] Based on the obtained session intent, the session intent and the interviewee are combined into search keywords. Specifically, the session intent and the interviewee are concatenated as a whole to generate a search request, or the session intent and the interviewee are used as parallel search conditions to generate a search request, so that subsequent API calls can query data. The interviewee serves as a unique identifier, and all records of the interviewee in heterogeneous data sources are retrieved. The session intent represents the focus of this interview. By combining the session intent and the interviewee, the retrieved data is made to match the requirements of this interview. The generation of the search request triggers the API interface to query various heterogeneous data sources within the enterprise, thereby enabling the search request to correspond with the HR system, historical interview database, project management system, and collaboration tools, and to obtain relevant business-related data.

[0037] Using the generated search request, the API interface is invoked to query the enterprise's internal HR system, historical interview database, project management system, and collaboration tools. Specifically, in each heterogeneous data source, the session intent and the interviewee are used as search fields to query data related to the session intent in the interviewee's records, thus obtaining business-related data. For example, in the project management system, records are read based on the interviewee's identifier, and then the similarity between the session intent and each field name is calculated using text similarity. Data with similarity values ​​greater than a preset similarity threshold is extracted to obtain business-related data. The preset similarity threshold can be 50%. By setting a lower similarity threshold, the retrieved business-related data is more comprehensive and consistent with the session intent, while avoiding interference from a small amount of irrelevant data. By performing intent recognition and generating search requests based on the interviewee, business-related data is automatically obtained, solving the problem of low efficiency in manually searching for data across systems. Through "intent" filtering, data irrelevant to the current interview is avoided, achieving on-demand and accurate data aggregation and providing data support for subsequent processing.

[0038] Step S300: Generate a structured session outline based on various related business data and session intents.

[0039] like Figure 3 As shown, a structured conversation outline is generated based on various related business data and conversation intents, including but not limited to the following steps: Step S310: Aggregate the various related business data to obtain multi-source aggregated data.

[0040] Specifically, the related business data includes HR system data, historical interview data, project management data, and collaboration tool data. For example, HR system data could include the interviewee's job level (P6) and start date (2020). Project management data could include the project's "Alpha" status (delayed), the interviewee's role as project development lead, and the duration of the project (six months). Collaboration tool data could include client feedback and provided materials. Historical interview data could include interview records from the previous quarter mentioning participation in training to improve efficiency.

[0041] In one embodiment, the various related business data are aggregated to obtain multi-source aggregated data, including but not limited to the following steps: Step S311 involves performing entity identification and relationship extraction on data from the personnel system, historical interview data, project management data, and collaboration tools to construct a conversation knowledge graph of the interviewee.

[0042] In some possible embodiments of this application, named entity recognition and relationship extraction models are used to process data such as personnel system data, historical interview data, project management data, and collaboration tool data. Extracted entity nodes, exemplarily, could be the interviewee's name, project name, job title, or client name. Relationships are extracted as follows: [Interviewee's Name] -> (Responsible For) -> [Project Alpha]; [Project Alpha] -> (Status) -> [Extended]; [Interviewee's Name] -> (Commitment to Promotion) -> [Attending Training]. Based on the extracted entities and relationships, a topological network structure is constructed to obtain a conversation knowledge graph of the interviewee. This allows for the subsequent extraction of supporting data that better reflects the conversation's intent, improving data accuracy.

[0043] Step S312: Using the conversation intent, extract the conversation subgraph related to the conversation intent from the conversation knowledge graph to obtain the first intent-related data.

[0044] In some possible embodiments of this application, the conversation subgraph is a knowledge graph composed of data strongly related to the conversation intent. Since the extracted data also includes some data with low relevance, a similarity algorithm is used again to calculate the similarity between the conversation intent and each entity node in the conversation knowledge graph, obtaining a similarity value. This similarity algorithm can be a text similarity algorithm or a cosine similarity algorithm. Entity nodes with similarity values ​​greater than a preset threshold are extracted, and the associated edges connected to these entity nodes are also extracted, thereby extracting the conversation subgraph and obtaining the first intent-related data. The preset threshold can be 70%, used to extract strongly related data. This first intent-related data is data strongly related to the conversation intent, and it is stored in JSON format to prepare for subsequent aggregation data. For example, if the conversation intent is a performance evaluation tag, the similarity comparison retains entity nodes and relationships such as performance nodes and project nodes in the conversation knowledge graph, while removing entity nodes such as job level nodes.

[0045] Step S313: Create a timeline by using data from the HR system, historical interview data, project management data, and collaboration tools as event cards. Anchor these event cards to the timeline according to the time nodes in the timeline, and label each event card at the same time node to obtain the event line.

[0046] In some possible embodiments of this application, the timeline is a linear structure of events arranged along a time axis. Creating an empty timeline covers the time period of the session intent, preparing for subsequent event writing. Event cards are various events attached to the timeline, corresponding to entities, and can be project extension cards, project launch cards, etc. Data from the HR system, historical interview data, project management data, and collaboration tools are used as event cards, anchored to the timeline according to time nodes. Specifically, the data is arranged in chronological order; no data may be arranged under a time node, or only one data point may be arranged. Multiple data points may be arranged under the same time node, and different data points are labeled with different parameters to indicate that they originate from different data sources.

[0047] For example, at a certain point in time, there are data such as project launch (project management data), performance targets achieved (HR system data), and on the timeline, project A launches, customer satisfaction feedback or customer complaint feedback is received, and an updated version is submitted. Anchoring each event card to the timeline transforms the timeline into an event line. This event line reflects the temporal order of the data and the chronological order of events, demonstrating a certain causal relationship and facilitating the discovery of potentially related data.

[0048] Step S314: Use the conversation knowledge graph and event lines to perform causal association analysis on the conversation intent to obtain the second intent association data.

[0049] In some possible embodiments of this application, graph algorithms combined with time series analysis are used to perform causal association analysis on the conversation knowledge graph and timeline to determine the conversation intent. For example, it is detected that on the timeline, the "Customer Li Si complains" event is immediately followed by the "Project Alpha code rollback" event, which conforms to the nature of time difference. Based on Project Alpha as a keyword, the potential causality is deduced through the association relationship between [Interviewee]-[Responsible]-[Project Alpha] in the conversation knowledge graph: the complaint may be caused by code quality issues. This deduction result is the second intent association data.

[0050] Specifically, the process begins by chronologically sequencing the event cards in the event timeline. These event cards are then matched with entity nodes in the knowledge graph. Relationship nodes are then identified based on these entity nodes. Causal reasoning is then performed between the chronological order of the event cards and the identified entity nodes, using pre-defined causal rules. These rules are developed based on extensive historical data and are comprised of multiple rule records. The input keywords are the event cards, the time sequence (represented by time difference, calculated as a positive number between the later and earlier occurrences), and the relationships between entity nodes. These keywords are then matched against the causal rules to obtain the derived second-intent association data. This second-intent association data is stored in JSON format for subsequent data aggregation.

[0051] The causal rule can be represented as follows: There exists an event A, and the type of event A is "customer complaint"; there exists an event B, and the type of event B is "code rollback". The time difference between the occurrence of event A and event B is less than a preset time threshold (the time threshold can be 1 day, etc.). There is an association path in the knowledge graph from the projects involved in event A to event B. The above rule data is input into a recurrent neural network model to derive the intent association data. The recurrent neural network model is existing technology and will not be elaborated here. This recurrent neural network model is fine-tuned through a large amount of conversation data to achieve generalization of conversation data.

[0052] Step S315: The first intent-related data and the second intent-related data are fused to obtain multi-source aggregated data.

[0053] In some possible embodiments of this application, the second intent-related data and the first intent-related data are concatenated, that is, the second intent-related data is appended to the first intent-related data, and finally represented in JSON format to obtain multi-source aggregated data. By integrating the graph and timeline, isolated data points are transformed into logical insights, so that the generated interview outline not only has data, but also logical deduction, effectively helping the interviewee to grasp the key points of the interview.

[0054] Step S320: Load the corresponding preset outline template based on the session intent. The outline template includes multi-level nodes, and each level node is used to indicate different session module topics corresponding to the session intent.

[0055] In one embodiment, the preset outline template includes multi-level nodes. Each level of node indicates different conversation module topics corresponding to the conversation intent. These are different template prompts set by experts based on experience, represented in JSON format for easy data reading and storage in the corresponding nodes. For example, if the conversation intent is a performance interview, the multi-level nodes included in the preset outline template to be loaded and matched are: 1. Review last period's goals; 2. Evaluate this period's performance; 3. Discuss strengths and weaknesses; 4. Develop next period's plan. The template description corresponding to the outline template is stored. The conversation intent is matched with the template description of the outline template. If a match is successful, the matched outline template is loaded for subsequent interview recording using structured storage, facilitating data processing. The matching can use text similarity matching; a successful match is considered when the similarity is greater than a preset similarity threshold. The similarity threshold can be 80%. If the similarity is less than the preset similarity threshold, the outline template corresponds to other template introductions. The outline template corresponding to other template introductions is a general template, thus ensuring that the identified conversation intent can be matched with an outline template to guide the interview and provide a basis for subsequent interview record writing.

[0056] It should be noted that for corporate interviews, the categories of conversational intent that can be identified are limited, such as performance review intent, salary review intent, exit review intent, project progress intent, and others. A corresponding outline template is provided for each different conversational intent. By identifying conversational intent and matching and loading outline templates that match the conversational intent, the interview can be guided and the interviewee can be improved, thereby improving interview efficiency.

[0057] Step S330: Use a preset text analysis model to extract information from the multi-source aggregated data to obtain key information points and supporting data related to the conversation intent.

[0058] In one embodiment, the preset text analysis model is a large language model, specifically a recurrent neural network model with an attention mechanism. This model is existing technology, and its structure and training process will not be elaborated here. The model is fine-tuned using a large amount of conversation data to achieve generalization to the conversation data. Based on the multi-source aggregated data obtained in step S310, the graph relationships and causal chains on the event lines are converted into natural language prompts, which are then input into the large language model to extract information from the multi-source aggregated data, obtaining key information and supporting points related to the conversation intent. This key information and supporting points are also stored in JSON format for easy data retrieval later, and a structured conversation outline is generated based on the key information and supporting points.

[0059] Step S340: Fill the key information points and supporting data into the corresponding nodes of the outline template to generate a structured conversation outline.

[0060] In one embodiment, key information points and supporting data are input into a preset classification model. The output classification categories of this model include the various level nodes of the outline template. To achieve this process, when loading the classification model, the various level nodes of the outline template are extracted and used as prompts for supervised learning. These prompts are input into the preset classification model along with the key information points and supporting data. The classification model classifies the input key information points and supporting data according to the outline template. Based on the classification results, the key information points and supporting data are filled into the corresponding nodes, generating a structured conversation outline. Each part of this structured conversation outline contains data recorded from various dimensions of the interviewee. During the interview, the recorded data provides prompts to assist in subsequent interviews, thereby making the interview focused and preventing it from becoming a mere formality.

[0061] It should be noted that the classification model can be a decision tree, support vector machine, or deep neural network model. The above models can adopt existing model structures and retrain the existing models using a large amount of text information and the corresponding outline templates. The specific model training process is an existing technology. The purpose of retraining is to enable the classification model to have better generalization ability to text information and outline templates, so that the generated structured conversation outlines have high accuracy.

[0062] Step S400: After the interview session based on the structured conversation outline is started, the multimodal input stream during the session is obtained. The multimodal input stream includes at least a voice data stream. The multimodal input stream is converted into a linear text stream and mapped to the corresponding hierarchical nodes of the structured conversation outline to form a structured conversation record.

[0063] In one embodiment, the generated structured conversation outline is displayed on the front-end page. In response to the interviewee's selection and click on the structured conversation outline, the interview session is initiated, and the audio device (e.g., a microphone array or online meeting) and input call interface are activated to obtain the multimodal input stream during the session. The multimodal input stream includes at least an audio data stream and an input interview summary. The multimodal input stream is converted into a linear text stream so that it can be mapped to the corresponding hierarchical nodes of the structured conversation outline. The current session is recorded for use in the next interview.

[0064] Specifically, converting a multimodal input stream into a linear text stream includes, but is not limited to, the following steps: Step S410: Use voiceprint recognition technology to divide the speech data stream in the multimodal input stream to obtain the speech segment of the interviewee and the speech segment of the interviewee.

[0065] In one embodiment, before segmenting the speech data stream, the speech data stream is processed to remove silent segments and pure noise segments, resulting in valid audio segments. Voiceprint recognition technology identifies the speaker by extracting biometric features (voiceprint feature vectors) from the speech signal. It can employ a deep neural network model, such as ResNet or x-vector, which are existing technologies. The ResNet model is used to extract features from each valid speech segment in the multimodal input stream, generating a high-dimensional voiceprint feature vector. If the user's voiceprint model is not pre-stored, a clustering algorithm is used to automatically classify the speech segments into sets "Speaker A" and "Speaker B" based on the similarity of the feature vectors. The clustering algorithm can be k-means clustering.

[0066] Furthermore, a pre-defined streaming speech recognition engine is invoked to convert the audio sets of speaker A and speaker B into a first candidate linear text stream and a second candidate linear text stream, respectively, with timestamps. A pre-trained Natural Language Processing (NLP) syntactic intent classifier is introduced to segment the first and second candidate linear text streams according to sentence granularity. Since in standard interactive control sessions (such as business interviews), the interviewee (such as a manager) has the attribute of advancing the conversation process, their speech often exhibits characteristics such as asking questions, guiding topics, and summarizing; while the interviewee (such as an employee) often exhibits characteristics such as stating, explaining, and responding. Therefore, the sentence distribution in the above text streams is statistically analyzed, and the first active guidance feature value corresponding to the first candidate text stream is calculated (e.g., the percentage of effective sentences containing interrogative words, rhetorical questions, and imperative sentences in the total number of sentences in the text stream). Similarly, the second active guidance feature value corresponding to the second candidate text stream is calculated.

[0067] The streaming speech recognition engine can employ a Recurrent Neural Network Transducer (RNN-T) based approach. This engine receives audio segments composed of fixed-length data packets as input and outputs the corresponding text segmentation sequence in real time without waiting for the complete recording to finish. It also uses endpoint detection to attach precise start and end timestamps to each text segment or paragraph. By comparing the timestamps, a precise mapping is established between the first and second candidate linear text streams and the audio clusters of speakers A and B, segmented by k-means clustering. The pre-trained natural language processing syntactic intent classifier can have an underlying network architecture that is either a pre-trained Transformer-based bidirectional encoding representation model (BERT model) or a text convolutional neural network with an attention mechanism (TextCNN). The quantitative extraction process of dialogue behavior features specifically includes the following steps: acquiring a first or second candidate linear text stream, segmenting it into independent sentence sequences, and converting the sentence sequences into high-dimensional semantic vectors using a word embedding layer; using a fully connected layer and a Softmax classification function, performing multi-class probability prediction on the high-dimensional semantic vector of each sentence, and calculating the posterior probability value of the sentence belonging to the interrogative sentence, imperative / command sentence, or regular declarative sentence in the classification label library; extracting the one with the highest probability value as the discourse behavior label of the specific sentence; for the global scope of a single candidate text stream, counting the number of sentences labeled with questioning or dominant features (i.e., interrogative sentences and imperative / command sentences), calculating their proportion to the total number of effective sentences in the text stream, and this proportion is used as the active guidance feature value corresponding to the text stream.

[0068] Then, the topic word vectors (such as business entity words like performance, project, and planning) contained in each level node of the generated structured conversation outline are extracted, and the core keyword vectors of the dialogue in the first and second candidate linear text streams are extracted using the TF-IDF algorithm and word vector models (such as Word2Vec or BERT model). The cosine similarity algorithm is used to calculate the semantic matching score between the core keyword vectors of the first and second candidate linear text streams and the topic word vectors of the outline level nodes. Since the interviewee (manager) is the initiator of the topics according to the outline, their text stream usually triggers first and highly hits the topic words of the outline level nodes. Therefore, through the above calculations, the first topic anchoring score of the first candidate text stream and the second topic anchoring score of the second candidate text stream are obtained respectively. A judgment function S = a × U + b × V is established. Where S is the interviewer's probability comprehensive score, U is the active guidance feature value, V is the topic anchoring score, and a and b are preset dynamic weight coefficients, taking values ​​of 0.4 and 0.6 respectively.

[0069] Substituting the calculated data into the aforementioned decision function, we obtain the interviewee probability score SA for speaker A and the interviewee probability score SB for speaker B. We then compare the values ​​of SA and SB. If SA > SB, it indicates that speaker A is more inclined to lead the conversation and their speech content is more closely aligned with the nodes of the pre-defined conversation outline. In this case, speaker A, who logically represents the controller, is marked as the interviewee, and their corresponding audio set and text stream are labeled as such. Conversely, speaker B is marked as the interviewee. By utilizing the generated structured conversation outline as a semantic reference coordinate system, passive physical clustering is transformed into an identity-adaptive back-inference method that combines contextual logic. This effectively solves the technical problem of not being able to automatically distinguish the identities of interviewees in an environment without an enterprise voiceprint database, ensuring the logical rigor and data accuracy of subsequent corresponding node mapping and data closed-loop write-back.

[0070] If the voiceprint fingerprints of company employees have already been stored, calculate the cosine similarity between the current segment's feature vector and the pre-stored fingerprints. If the similarity with the manager's fingerprint is higher than the fingerprint threshold, it is marked as the interviewee's voice segment; otherwise, it is marked as the interviewee's voice segment. The fingerprint threshold is 85%.

[0071] The "interviewee's audio segment" and the "interviewee's audio segment" refer to discrete audio fragments containing only the voice of a single speaker after segmentation. For example, the "interviewee's audio segment" is identified as the audio of a sentence or paragraph spoken by manager A; the "interviewee's audio segment" is identified as the audio of employee B's speech. By segmenting continuous audio into clearly defined fragments, a foundation is laid for subsequent analysis of the interviewee's questioning / guidance behavior and the interviewee's feedback / commitment behavior, which is a prerequisite for achieving accurate attribution of responsibility.

[0072] Step S420: Convert the interviewee's voice segment into an interview text segment, and convert the interviewee's voice segment into an interview text segment.

[0073] In one embodiment, the interview text segment and the interviewee text segment are represented as natural language text strings. The interviewee's speech segment is input into the speech model, which outputs the interview text segment; conversely, the interviewee's speech segment is input into the speech model, which outputs the interviewee text segment. This output data is used to subsequently form a text segment stream. The speech model can employ a Conformer or Transformer architecture, which is existing technology and will not be elaborated upon here. The model is fine-tuned using a large amount of conversation data to achieve generalization to the conversation data. The conversion transforms unstructured analog / digital speech signals into text data that can be semantically computed by a computer, facilitating subsequent write-back processing.

[0074] In step S430, timestamps are added to both the interview text segment and the interviewee text segment, and the segments are arranged in order of the timestamps to obtain a text segment stream.

[0075] In one embodiment, based on the text segments obtained in step S420, start and end times are added to both the interview text segment and the interviewee text segment, and the speaker of each text segment is marked. The start time, end time, and speaker of each text segment are attached as attribute tags to each text segment. The interview text segment and the interviewee text segment are added to a buffer queue in order of their start times to obtain a text segment stream, which is then used to obtain a linear text stream later.

[0076] Step S440: Obtain the input interview summary and combine the interview summary and text segment stream into a linear text stream.

[0077] In one embodiment, the interview summary is provided on the front-end page where the interviewee has an input interface. The interviewee can input a summary of the interview or select a preset summary field, which is then transmitted to the product to obtain the input interview summary. The interview summary undergoes text preprocessing, which adds a descriptive header identifier to the text and attaches it as an attribute tag to the interview summary. Then, the interview summary and the text segment stream are concatenated to obtain a linear text stream. Specifically, the concatenation process adds the text segment stream after the interview summary. The interview summary provides the background of the interview, and combining these elements limits the boundaries of the analysis, thereby greatly improving the accuracy of subsequent structured extraction.

[0078] Specifically, the linear text stream is mapped to the corresponding hierarchical nodes of the structured conversation outline to form a structured conversation record, including but not limited to the following steps: Step S450: Within a preset context window, calculate the similarity between the corresponding level nodes of the linear text stream and the structured conversation outline in real time to obtain the similarity value of each level node.

[0079] In one embodiment, a preset context window defines the processing range of the linear text stream. The continuity of the text is ensured by moving the context window. The context window can be divided into 15-minute intervals, with the window moving every 5 minutes. A preset similarity algorithm is used to calculate the similarity between the linear text stream and corresponding level nodes of the structured conversation outline in real time, obtaining similarity values ​​for each level node. These similarity values ​​quantify the semantic proximity between the current dialogue content and a topic of a certain outline node. First, the BERT model converts the current linear text stream into a high-dimensional vector, and the descriptive text of all nodes in the structured conversation outline is converted into node vectors. The cosine similarity algorithm is used to calculate the similarity between the high-dimensional vector and the node vectors, obtaining similarity values ​​that provide a foundation for subsequent filling of the structured conversation outline.

[0080] Step S460: Fill the text segments of the linear text stream with similarity values ​​greater than the preset similarity threshold under the hierarchical node.

[0081] In one embodiment, the preset similarity threshold is an empirical value, such as 75% or 80%, used to determine whether the current dialogue belongs to a node. If the similarity value is greater than the preset similarity threshold, it indicates that the linear text stream has entered the node, and the linear text stream is filled under that level node. This achieves automated classification and organization of interview records, eliminating the need for manual transcription of the recording sentence by sentence after the interview.

[0082] It should be noted that if multiple nodes exceed the threshold simultaneously, the node with the highest similarity is selected as the primary filling level node, for example, when the topic transitions from A to B; or, based on the hierarchical logic of the outline, child nodes under the current stage are matched first, for example, if we are currently in the "Performance Review" stage. For example, the hierarchical nodes of the structured conversation outline are: 1. Review last period's goals; 2. Evaluate this period's performance; 3. Discuss strengths and weaknesses; 4. Develop next period's plan. The linear text stream of the performance review is then filled under the "1.1 Items to be improved" node as the text stream for this current session.

[0083] In one embodiment, if the similarity of all nodes is below the similarity threshold, the text segment is marked as "uncategorized" or "casual / other" and temporarily stored in other nodes to avoid noise data interfering with core business records.

[0084] In step S470, in response to the user's end interview operation, all text segments mapped to each level node are collected to form a structured session record.

[0085] In one embodiment, the front-end page includes an "End Interview" button, which users can click to end the interview. In response to the user's click on the "End Interview" button, all text segments mapped to each hierarchical node are collected. A depth-first traversal of the structured conversation outline is performed, and for each node, its text segments are extracted in chronological order. The text segments under each node are then recombined into a conversation record, forming a structured conversation record. This structured conversation record corresponds to the outline template format and is also stored in JSON format, serving as an electronic document or data package with hierarchical logical relationships. This structured data facilitates subsequent text calculations, improving accuracy.

[0086] For example, a structured session record is represented as follows: {"Introduction":"Third Quarter Performance Review",} "Hierarchical Node":[{"Node1":"1. Performance Review", "Content":[{"Interviewee":"How is Project A going?"},{"Interviewee":"Delivered on time."}]}, {"Node":"2. Evaluate the current period's performance",} "content":[...]}]} Step S500: Extract key information from the structured session records and write it into heterogeneous data sources according to the categories of the extracted key information to update the related business data and form a data closed loop.

[0087] In one embodiment, key information is extracted from structured session records and written into heterogeneous data sources according to the categories of the extracted key information, including but not limited to the following steps: Step S510: Extract key information from the structured session records in multiple dimensions to obtain multi-dimensional structured information.

[0088] Specifically, key information is extracted from structured session records in multiple dimensions to obtain multi-dimensional structured information, including but not limited to the following steps: Step S511: Divide the structured session record into each round of dialogue state segments, and generate a corresponding state label for each round of dialogue state segments, wherein the state label corresponds to the purpose of the session.

[0089] In some possible embodiments of this application, a dialogue state fragment refers to the smallest interactive unit with complete semantic logic under a certain level node of the structured record. A state label refers to metadata describing the current fragment's stage in the business process; the set of state labels is strongly correlated with the purpose of the conversation. For example, if the purpose of the conversation is a "performance interview," the state labels might include [goal setting], [problem diagnosis], [consensus reaching], etc. The structured conversation record is divided into several conversation state fragments based on speaker transitions and pause times. According to the purpose of the conversation, a pre-trained text classification model is used to input the conversation intent corresponding to the current dialogue state fragment and the conversation purpose, outputting the probability that the dialogue state fragment belongs to a business state, thus obtaining the state label. Decomposing lengthy interview records into atomic units with business attributes allows us to know not only "what was said" but also "at what stage it was said," providing support for subsequent processing.

[0090] Step S512: Perform named entity recognition and relation extraction on the structured session records to obtain the entities and the relationships between them.

[0091] In some possible embodiments of this application, a pre-defined named entity recognition model is used to scan the structured conversation records, locate the boundaries and types of entities, and for sentences containing multiple entities, a relation extraction model is used to analyze the dependency syntax tree between them, find the shortest path on the tree, and construct a clear relationship between the entities. The structured conversation record text data is converted into knowledge graph triples to prepare for subsequent extraction of relationships between data and accurate write-back. The named entity recognition model can be a BiLSTM-CRF or Transformer model, fine-tuned using a large amount of structured conversation data. This data can be obtained through web crawling technology to achieve generalization of conversation data.

[0092] Step S513: The status labels, associations, and dialogue state fragments from the structured session records are fused to obtain fused data.

[0093] In some possible embodiments of this application, feature concatenation processing is performed on state labels, association relationships, and dialogue state fragments from structured session records to form an input sequence, resulting in fused data. This fused data contains high-dimensional data vectors or composite data objects with rich contextual features, improving the accuracy of the multi-label intent classifier.

[0094] Step S514: Use a preset multi-label intent classifier to classify the intent of the fused data to obtain multi-dimensional structured information.

[0095] In some possible embodiments of this application, the multi-label intent classifier is a deep learning model capable of determining which attributes a sentence simultaneously possesses. Unlike single-classification, multi-labeling allows a sentence to be both a "to-do task" and a "risk warning," and its model adopts the BERT structure. The fused data is input into the BERT model, which outputs the confidence score of each dialogue state segment. Categories with confidence scores greater than a confidence threshold are output as JSON objects, resulting in multi-dimensional structured information. Key information categories are extracted to prepare for subsequent writing to heterogeneous data sources. The confidence threshold can be 0.5.

[0096] Step S520: Using preset mapping rules, multi-dimensional structured information is written into the corresponding heterogeneous data source. The mapping rules include the correspondence between multiple information categories and heterogeneous data sources.

[0097] In one embodiment, the mapping rules include the correspondence between multiple information categories and heterogeneous data sources. These correspondences are pre-configured for professionals through statistical analysis of a large amount of interview data. For example, Rule A: If the category is "To-Do" and the entity contains "Project Code," then it is mapped to -> Project Management System. Rule B: If the category is "Performance Evaluation," then it is mapped to -> Human Resources System. Rule C: If the category is "Memo," then it is mapped to -> Historical Interview Database.

[0098] It calls pre-defined API interfaces (RESTful interfaces) to perform data creation operations, writing multi-dimensional structured information into corresponding heterogeneous data sources. This replaces the tedious work of manually compiling minutes and creating tasks after interviews. It ensures that valuable data generated from interviews is written into the company's core business processes, rather than remaining in documents or recordings, providing direct historical data input for the next interview and achieving fully automated loops.

[0099] In one embodiment, multi-dimensional structured information is written into a heterogeneous data source according to the above method to update business-related data. This also realizes the updating of the heterogeneous data source, forming a data closed loop and facilitating the automatic preparation and recording of the next interview. It should be noted that the above update is only an update to increase data, in order to write multi-dimensional structured information into the heterogeneous data source to increase the current session data, enrich the amount of data in the heterogeneous data source, and provide a reliable foundation for the next session.

[0100] To verify the effectiveness of the above method (hereinafter referred to as "this method"), the following control experiment was designed: Experimental group (Group A): 20 managers (interviewees) who used and implemented "this method".

[0101] Control group (Group B): 20 managers, using traditional methods (manually looking up information and manually recording data).

[0102] Experiment content: Group A and Group B each conducted a performance review with 5 subordinates (the interviewees) for one cycle.

[0103] 1. Verification Indicators Indicator 1: Interview preparation efficiency (verified by querying historical interview data) Test: Record the average time that managers in Group A and Group B spend "finding and organizing information" before each interview.

[0104] Expected: The average preparation time for Group A is significantly lower than that for Group B (reduced by 80%).

[0105] Indicator 2: Interview Effectiveness (Verifying the "Smart Outline") Test: After the interview, an anonymous survey was conducted among the interviewed employees (Group A and Group B) to assess whether "the interview grasped the key points" and "whether the manager communicated based on facts and data".

[0106] Expected: Group A employees' positive ratings are significantly higher than those of Group B.

[0107] Indicator 3: Efficiency of interview result processing (validating "structured results") Test: Record the average time that managers in Group A and Group B spent "organizing and archiving interview records and entering to-do items" after the interview.

[0108] Expected: The average processing time for Group A is significantly lower than that for Group B (which is close to automated).

[0109] Indicator 4: Information extraction accuracy (verifying the quality of "structured" data) Test: Randomly select interview records from Group A and manually verify the accuracy and recall of the "to-do tasks" and "performance facts" automatically extracted by the system.

[0110] Expected: Both accuracy and recall rates will reach acceptable industry standards (above 90%).

[0111] Indicator 5: Data closed-loop verification (verifying "correlation") Test: When the next interview cycle starts, check whether the Group A system can automatically retrieve the "Key Memos" and "To-Do Tasks Completion Status" that were processed and archived by "this method" in the previous cycle.

[0112] Expected outcome: Group A will be retrieved 100% automatically, while Group B will still require manual recall or searching.

[0113] like Figure 4As shown, this application embodiment provides a data processing system 100 based on multi-source interview data aggregation. This system 100 responds to user-created interview session instructions via a response processing module 110. The interview session instructions include a target object and a session purpose; the target object includes the interviewee. Then, an analysis and retrieval module 120 identifies the session purpose to obtain the session intent, and uses the session intent and the interviewee to generate a retrieval request. This retrieval request is used to retrieve related business data corresponding to the interviewee from multiple preset heterogeneous data sources. An outline generation module 130 is then used. Based on various related business data and conversation intents, a structured conversation outline is generated. After the interview conversation based on the structured conversation outline is started, the interview conversation recording module 140 acquires the multimodal input stream during the conversation process. The multimodal input stream includes at least a voice data stream. The multimodal input stream is converted into a linear text stream and mapped to the corresponding hierarchical nodes of the structured conversation outline to form a structured conversation record. Finally, the extraction and writing module 150 extracts key information from the structured conversation record and writes it into heterogeneous data sources according to the categories of the extracted key information to update the related business data and form a data closed loop.

[0114] It should be noted that the response processing module 110 is connected to the analysis and retrieval module 120, the analysis and retrieval module 120 is connected to the outline generation module 130, the outline generation module 130 is connected to the interview session recording module 140, and the interview session recording module 140 is connected to the extraction and writing module 150. The above-mentioned data processing based on multi-source interview data aggregation is applied to the data processing system 100 based on multi-source interview data aggregation. The data processing system 100 responds to user-created interview session instructions. The interview session instructions include the target object and the session purpose. The target object includes the interviewee. By constructing the interview session instructions, the interviewee is determined, providing support for obtaining accurate data subsequently. Intent recognition is performed on the session purpose to obtain the session intent. Intent recognition reflects the purpose of this session, enabling targeted extraction of heterogeneous multi-source data. A retrieval request is generated using the session intent and the interviewee. The retrieval request is used to retrieve related business data corresponding to the interviewee from multiple preset heterogeneous data sources. This retrieval request defines the retrieval scope and avoids interference from invalid data. It can also extract heterogeneous multi-source data to achieve pre-meeting data aggregation; based on various related business data and conversation intents, it generates a structured conversation outline to prepare for automatic structured processing during the meeting; after the interview conversation based on the structured conversation outline starts, it acquires the multimodal input stream during the conversation process, which includes at least voice data streams, converts the multimodal input streams into linear text streams, and maps the linear text streams to the corresponding hierarchical nodes of the structured conversation outline to form a structured conversation record. Using the structured conversation outline as a medium, it can be aligned with the interview objectives in real time to avoid shifting the focus; it extracts key information from the structured conversation record and writes it into heterogeneous data sources according to the categories of extracted key information to update related business data, forming a data closed loop and realizing closed-loop processing of document recording and write-back. The above process improves interview efficiency through automated structured processing of interview materials.

[0115] It should also be noted that the apparatus provided in the above embodiments is only illustrated by the division of the above functional modules. In actual applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above. In addition, the apparatus and method embodiments provided in the above embodiments belong to the same concept, and their specific implementation process can be found in the method embodiments, which will not be repeated here.

[0116] The following describes an exemplary electronic device 700 provided in an embodiment of this application. Figure 5 This is an exemplary hardware structure diagram of the electronic device 700 provided in the embodiments of this application.

[0117] In some embodiments, the electronic device 700 is a computer device or includes a computer device. The computer device includes a processor, memory, and a network interface connected via a system bus. The processor of the computer device provides computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and internal memory. The non-volatile storage medium stores an operating system, computer programs, and a database. The internal memory provides an environment for the operation of the operating system and computer programs in the non-volatile storage medium. The database of the computer device stores data. The network interface of the computer device is used to communicate with other external terminals or servers via a network connection. In some embodiments, the network interface can be a wired network interface; in some embodiments, the network interface can also be a wireless network interface. When the computer program is executed by the processor, it implements the methods in the embodiments of this application.

[0118] Those skilled in the art will understand that Figure 5 The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the computer device to which the present application is applied. Specific computer devices may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0119] The above embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit it. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of this application.

[0120] In the above embodiments, implementation can be achieved, in whole or in part, through software, hardware, firmware, or any combination thereof. When implemented in software, it can be implemented, in whole or in part, as a computer program product. A computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, all or part of the flow or function according to the embodiments of this application is generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired (e.g., coaxial cable, fiber optic, digital subscriber line) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that integrates one or more available media. The available medium can be a magnetic medium (e.g., floppy disk, hard disk, magnetic tape), an optical medium (e.g., DVD), or a semiconductor medium (e.g., solid-state drive), etc.

[0121] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. This program can be stored in a computer-readable storage medium, and when executed, it can include the processes described in the above method embodiments. The aforementioned storage medium includes various media capable of storing program code, such as ROM or random access memory (RAM), magnetic disks, or optical disks.

[0122] In the above embodiments, the descriptions of each embodiment have different focuses. For parts not described in detail in a certain embodiment, please refer to the relevant descriptions in other embodiments.

[0123] In the various embodiments provided in this application, it should be understood that the disclosed apparatus can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some service interface; the indirect coupling or communication connection between apparatuses or units may be electrical or other forms.

[0124] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0125] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0126] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage device (CMD). Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a memory and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods of the various embodiments of this application. The aforementioned memory includes various media capable of storing program code, such as USB flash drives, portable hard drives, magnetic disks, or optical disks.

[0127] The above are merely exemplary embodiments of this disclosure and should not be construed as limiting the scope of this disclosure. Any equivalent changes and modifications made in accordance with the teachings of this disclosure shall still fall within the scope of this disclosure. Other embodiments of this disclosure will readily conceive of those skilled in the art upon consideration of the specification and the disclosure of practical truths.

[0128] This application is intended to cover any variations, uses, or adaptations of this disclosure that follow the general principles of this disclosure and include common knowledge or customary techniques in the art not described in this disclosure. The specification and embodiments are to be considered exemplary only, and the scope and spirit of this disclosure are defined by the claims.

Claims

1. A data processing method based on multi-source interview data aggregation, characterized in that, The method includes: In response to a user-created interview session instruction, the interview session instruction includes a target object and a session purpose, the target object including the interviewee; The purpose of the conversation is identified to obtain the conversation intent, and a retrieval request is generated using the conversation intent and the interviewee. The retrieval request is used to retrieve the associated business data corresponding to the interviewee from multiple preset heterogeneous data sources. Based on the associated business data and the session intent, a structured session outline is generated; After the interview session based on the structured conversation outline is initiated, the multimodal input stream during the session is obtained. The multimodal input stream includes at least a voice data stream. The multimodal input stream is converted into a linear text stream and the linear text stream is mapped to the corresponding hierarchical nodes of the structured conversation outline to form a structured conversation record. Key information is extracted from the structured session records and written into the heterogeneous data source according to the category of the extracted key information, so as to update the related business data and form a data closed loop.

2. The method according to claim 1, characterized in that, The generation of a structured session outline based on the associated business data and the session intent includes: The various related business data are aggregated to obtain multi-source aggregated data; Based on the session intent, a corresponding preset outline template is loaded. The outline template includes multi-level nodes, and each level node is used to indicate different session module topics corresponding to the session intent. The multi-source aggregated data is extracted using a preset text analysis model to obtain key information points and supporting data related to the conversation intent; The key information points and supporting data are filled into the corresponding nodes of the outline template to generate the structured session outline.

3. The method according to claim 2, characterized in that, The related business data includes human resources system data, historical interview data, project management data, and data from collaboration tools; The aggregation of the various related business data to obtain multi-source aggregated data includes: Entity identification and relationship extraction are performed on the data from the human resources system, the historical interview data, the project management data, and the data from the collaboration tools to construct a conversation knowledge graph of the interviewee; Using the conversation intent, extract the conversation subgraph related to the conversation intent from the conversation knowledge graph to obtain the first intent association data; Create a timeline, using the HR system data, historical interview data, project management data, and collaboration tool data as event cards, anchoring them to the timeline according to the time nodes, and labeling each event card at the same time node to obtain an event line; The conversation knowledge graph and the event line are used to perform causal association analysis on the conversation intent to obtain second intent association data; The first intent-related data and the second intent-related data are fused to obtain the multi-source aggregated data.

4. The method according to claim 2, characterized in that, The step of mapping the linear text stream to the corresponding hierarchical nodes of the structured conversation outline to form a structured conversation record includes: Within a preset context window, the similarity between the linear text stream and the corresponding level nodes of the structured conversation outline is calculated in real time to obtain the similarity value of each level node. The text segments of the linear text stream corresponding to the similarity value greater than the preset similarity threshold are filled under the hierarchical node; In response to the user's action to end the interview, all text segments mapped to nodes at each level are collected to form the structured session record.

5. The method according to claim 1, characterized in that, The step of extracting key information from the structured session records and writing it into the heterogeneous data source according to the categories of the extracted key information includes: Multi-dimensional key information is extracted from the structured session records to obtain multi-dimensional structured information; Using preset mapping rules, the multi-dimensional structured information is written into the corresponding heterogeneous data source, wherein the mapping rules include the correspondence between multiple information categories and heterogeneous data sources.

6. The method according to claim 5, characterized in that, The process of extracting key information from the structured session records in multiple dimensions yields multi-dimensional structured information, including: The structured session record is divided into each round of dialogue state segments, and a corresponding state label is generated for each round of dialogue state segments, wherein the state label corresponds to the purpose of the session; Named entity recognition and relation extraction are performed on the structured session records to obtain entities and the relationships between them; The status labels, the association relationships, and the dialogue state fragments recorded in the structured session are fused to obtain fused data; The fused data is classified using a pre-defined multi-label intent classifier to obtain the multi-dimensional structured information.

7. The method according to claim 1, characterized in that, The multimodal input stream also includes an interview summary; The step of converting the multimodal input stream into a linear text stream includes: The voice data stream in the multimodal input stream is divided using voiceprint recognition technology to obtain the voice segment of the interviewee and the voice segment of the interviewee. The speech segment of the interviewee is converted into the interview text segment, and the speech segment of the interviewee is converted into the interview text segment. Add timestamps to both the interview text segment and the interviewee text segment, and arrange them in the order of the timestamps to obtain a text segment stream; Obtain the input interview summary, and combine the interview summary and the text segment stream into the linear text stream.

8. An electronic device, characterized in that, It includes one or more processors and a memory; the memory is coupled to the one or more processors, the memory is used to store computer program code, the computer program code including computer instructions, and the one or more processors call the computer instructions to cause the electronic device to perform the method as described in any one of claims 1-7.

9. A computer program product containing instructions, characterized in that, When the computer program product is run on an electronic device, it causes the electronic device to perform the method as described in any one of claims 1-7.

10. A computer-readable storage medium comprising instructions, characterized in that, When the instructions are executed on an electronic device, the electronic device causes the electronic device to perform the method as described in any one of claims 1-7.

Images