Construction method of global talent flow dynamic corpus

Abstract

The invention provides a construction method of a global talent flow dynamic corpus. The global talent flow condition can be effectively and accurately analyzed. The method comprises the following steps: (1) collecting global talent flow dynamic diachronic narrative texts through a network; (2) extracting talent flow information from the diachronic narrative text by adopting a large language model LLMs (Language Language Model); (3) the extraction results of different models in the large language model LLMs are aligned; and (4) compiling the aligned extraction results into a global talent flow dynamic corpus.

Description

Technical Field

[0001] The invention relates to a method for constructing a global talent flow dynamic corpus. Background Art

[0002] Talent mobility, which refers to the phenomenon of individuals with specialized skills moving across geographic, organizational, or national boundaries to meet labor demand and achieve the redistribution of expertise, has always been an important issue in the global competition to attract and retain talent. In today's knowledge-based society, talent mobility is a long-standing concern in the global competition for talent. Highly skilled talents, including researchers, professionals, and practitioners, not only promote the integration of existing information, but also promote the dissemination of knowledge in various fields. Given the importance of talent mobility, research in this field has grown rapidly, covering a wide range of topics, such as immigrants' integration and return intentions, transnational networks and practices, and international student mobility. However, existing research usually adopts a single perspective, generally defining developing countries as "talent outflows" and economically developed countries as "talent inflows", thus ignoring the bilateral relationship in talent mobility. In addition, many studies have deeply analyzed the brain drain and inflow of specific countries, but there is a lack of research exploring the links between countries from a global perspective, resulting in a lack of understanding of how countries and regions at different levels of development can successfully attract highly skilled talents.

[0003] Recent studies have used geospatial metadata analysis methods to reveal talent mobility patterns. Geospatial metadata can be obtained from large-scale data sources including social networks, or from mobile devices that provide location-based services. For example, the change in location information provided by more than 300 million smartphones is used to study talent mobility. However, geospatial metadata analysis methods only superficially capture the flow between countries and regions from the changes in signal points, while ignoring the relevant attributes of mobile individuals. Therefore, the conclusions drawn from geospatial metadata analysis alone are difficult to accurately apply to discussions related to high-skilled talents.

[0004] Quantitative analysis methods driven by static data sets are another common means of studying talent mobility. For example, the latest research uses the GAIA data set to depict the annual dynamics of the global human footprint from 2000 to 2018. However, although the static data set driven method provides richer features than single signal changes, it has significant limitations when dealing with unquantifiable features such as geographic entities. In addition, the method is limited to static data at fixed time intervals and cannot capture the rapid changes and latest trends in talent mobility.

[0005] Both geospatial metadata analysis methods and quantitative analysis methods driven by static data sets suffer from insufficient analytical preparation and cannot be effectively used for talent mobility analysis on a global scale. Summary of the invention

[0006] The purpose of the present invention is to overcome the above-mentioned deficiencies in the prior art and to provide a method for constructing a reasonably designed dynamic corpus of global talent flow, which can effectively and accurately analyze the talent flow situation on a global scale.

[0007] The technical solution adopted by the present invention to solve the above problem is: a method for constructing a dynamic corpus of global talent flow, characterized in that it includes the following steps: 1. Collecting diachronic narrative texts of global talent mobility dynamics through the Internet; (ii) Using large language models (LLMs) to extract talent mobility information from diachronic narrative texts; (3) Align the extraction results of different models in the large language model LLMs; (IV) Compile the aligned extraction results into a dynamic corpus of global talent flow.

[0008] The diachronic narrative text described in the present invention can record information and track the long-term dynamics of talent flow.

[0009] Before the extraction in step (ii), the present invention constructs prior information and performs structural fine-tuning on the large language models LLMs.

[0010] For the C4 dataset in the large language model LLMs, the steps of constructing prior information and fine-tuning the structure are as follows: (1) Distinguishing diachronic narrative texts based on the conditions of explicit and implicit talent mobility texts; (2) Select prompt words that are suitable for the talent flow extraction task to guide the response of the large language model LLMs; (3) Responses containing necessary talent flow features are selected from the generated texts, and the LoRa method is used to further fine-tune the large language model (LLMs).

[0011] The corpus described in the present invention is divided into three different parts: a time series corpus, a static corpus and a negative sample corpus.

[0012] In step (ii) of the present invention, the talent flow information includes the country of departure, country of destination and timestamp of the talent flow.

[0013] The timestamp described in the present invention is standardized into the YYYY-MM format.

[0014] In step (ii) of the present invention, the geographic entities identified by the large language models (LLMs) are standardized and mapped to the country level for extraction.

[0015] In step (iii), the present invention only retains the outputs that are completely consistent between all models and removes any inconsistent parts.

[0016] In step (iii), the present invention adopts an integration method to combine the prediction results of multiple large language models (LLMs) by outputting an average value or using a voting mechanism to fuse the predictions of multiple models.

[0017] Compared with the prior art, the present invention has the following advantages and effects: 1. It can characterize the areas with advantages and disadvantages in terms of talent mobility in various countries and provide a reference for policy making; policymakers can have a deeper understanding of the complex dynamics behind talent mobility and make wise decisions to address related challenges.

[0018] 2. It highlights the potential of data-driven approaches in policy making and helps develop more efficient talent mobility strategies. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] Figure 1 It is a schematic diagram of an embodiment of the present invention using large language models (LLMs) to extract talent flow information from diachronic narrative texts. DETAILED DESCRIPTION

[0020] The present invention will be further described in detail below with reference to the accompanying drawings and by way of examples. The following examples are intended to explain the present invention but the present invention is not limited to the following examples.

[0021] 1. The embodiment of the present invention includes the following steps: 1. Collect diachronic narrative texts on the dynamics of global talent mobility on the Internet.

[0022] Diachronic narrative texts have the ability to record information and track the dynamics of long-term talent flow, making them an ideal source for long-term research. This type of text has been widely used in multiple tasks, such as word frequency statistics and syntactic parsing.

[0023] (ii) Use large language models (LLMs) to extract talent flow information from diachronic narrative texts.

[0024] Talent mobility information includes the country of departure, country of destination, and timestamp of the talent flow. To ensure consistency and accuracy, the timestamp is standardized to YYYY-MM format, providing monthly resolution.

[0025] Geographic entities (such as city names, organization names, landmarks, etc.) identified by large language models (LLMs) are standardized and mapped to the country level for extraction to ensure consistency and accuracy.

[0026] For tasks with complex relationships, common processing methods in diachronic narrative texts (such as dependency parsing and part-of-speech tagging) have difficulty capturing the relationships between entities, events, and concepts because they lack the ability to understand the context. To address this problem, large language models (LLMs) are introduced. Large language models (LLMs) perform well in understanding semantic relationships in text and can generate structured outputs based on input. For example, large language models (LLMs) have been shown to perform well in tasks such as named entity recognition and relation extraction.

[0027] In order to improve the extraction performance of large language models LLMs on specific tasks, this embodiment constructs prior information and fine-tunes the response structure of large language models LLMs before extraction to ensure that talent flow information is fully extracted from diachronic narrative texts. This method combines the semantic understanding ability of large language models LLMs with task-specific training to more accurately capture the talent flow characteristics in the text, providing strong data support for further analysis and policy making.

[0028] For the C4 dataset in the large language model LLMs, the steps for constructing prior information and structural fine-tuning are as follows: (1) As one of the largest language datasets compiled from the Common Crawl web archive, we distinguish diachronic narrative texts based on the conditions of explicit and implicit talent flow texts and demonstrate the advantages of large language models over previous natural language processing (NLP) methods in this regard. Obviously, due to the huge information reserves and semantic analysis capabilities of large language models (LLMs), they are able to perform tasks such as geographic entity mapping and talent flow subject analysis well. (2) In order to further improve the performance of large language models (LLMs) in the task, we first select prompt words that are suitable for the talent flow extraction task to guide the responses of large language models (LLMs), making them more sensitive to the task and reducing the occurrence of hallucination phenomena. (3) Subsequently, we screen out responses containing the necessary talent flow features from the generated text, and further fine-tune the large language models (LLMs) in combination with the LoRa (low-rank) method so that they can output structured data.

[0029] (III) In order to reduce the bias and uncertainty of model output, the extraction results of different models in the large language model LLMs are aligned. This is necessary because differences in training data, architecture, and randomness may cause the outputs of multiple models to be different. Relying on one model alone may have potential risks, and ignoring the inference results of other models may cause a single model to introduce errors or biases, thereby affecting the accuracy of feature extraction.

[0030] An integrated approach is used to combine the predictions of multiple large language models (LLMs) by averaging the output or using a voting mechanism to fuse the predictions of multiple models. Recent studies on GPT-3 have shown that combining models of different sizes and configurations is more advantageous than a single model. In addition, Multitask Prompted Training has been shown to align the outputs of a single model between different tasks, thereby ensuring consistency and accuracy. In this embodiment, only outputs that are completely consistent between all models are retained, and any inconsistent parts are eliminated. This strict filtering reduces semantic bias and improves the accuracy of feature extraction, because different models may emphasize different aspects of the text. By leveraging the advantages of multiple models and aligning their outputs, the reliability and robustness of feature extraction results can be improved.

[0031] (IV) The aligned extraction results are compiled into a dynamic corpus of global talent flow, which has fine spatiotemporal resolution and covers a wide range of transfer-related features, including diachronic narrative text, source URL, flow timestamp and location, see Table 1.

[0032] Table 1. Corpus summary information.

[0033] By comparing data from multiple large language models (LLMs), the corpus is divided into three different parts: a time series corpus, a static corpus, and a negative sample corpus. This classification method can effectively organize and analyze data, making it easier to identify patterns and trends in talent mobility. The resulting corpus provides a solid foundation for studying the dynamics of talent mobility and its underlying mechanisms.

[0034] 2. Inspection.

[0035] The main contribution of the Global Talent Mobility Dynamic Corpus is the innovative method of extracting text-specific features using large language models (LLMs) and providing a set of talent mobility features with great temporal and spatial coverage. However, the differences in the extraction performance of these models may be magnified in specific tasks, so it is particularly important to verify the extraction performance of the model after task-specific fine-tuning.

[0036] 11479 random narrative texts were used as the test set (Table 2). The results show that the CR scores of all models are over 0.957, which is a significant improvement compared to the highest CR score of 0.051 in the untuned model. This shows that after task-specific fine-tuning, the structure of the reasoning results is roughly consistent with the task requirements. In addition, the average improvement in TPR scores is about 0.09, which is a positive optimization for all models. Therefore, fine-tuning large language models (LLMs) can significantly improve extraction performance, which in turn helps enrich existing corpora and promote more accurate and informed decision making.

[0037] Table 2. Verify the fine-tuning results of reasoning performance on specific tasks.

[0038] In the table, * CR (Compliance rate) indicates the proportion of structured information output by the large language model LLMs according to the extraction task requirements; TPR (True Positive Rate) indicates the proportion of all positive samples that are correctly predicted as positive samples.

[0039] Next, we sampled 10% of the inference results in the positive samples and conducted manual reasoning verification checks. By grouping the text segmentation size of the time-series narrative text, we manually annotated the entity reasoning results and calculated the recall, precision, and F1 scores, as shown in Table 3. The recall rate indicates the proportion of relevant instances successfully retrieved by the model, while the precision rate refers to the proportion of relevant instances identified by the model that are truly correct. The F1 score represents the harmonic mean of the recall rate and precision rate, which provides a balanced measure for evaluating the accuracy and comprehensiveness of the model in identifying relevant relationships.

[0040] Table 3. Manual evaluation results of task-specific extraction using large language models (LLMs).

[0041] The best F1 score for each entity is about 0.9, which is significantly higher than the performance of large language model-based entity extraction methods used in most recent studies. The results also show that the increase in token size has a positive effect on the performance of entity extraction. Among the performance results for each entity, the feature group with longer token size performs best. These tokens are reflected as rich contextual information in narrative data, which provides LLM with more prior knowledge to help it better identify entities, thereby improving the F1 score.

[0042] In addition, when the token size exceeds 200 and continues to increase, the F1 score of all entity groups rises by about 0.04. However, as the token size approaches the processing limit of large language models (LLMs), the F1 score decreases slightly, resulting in the best performance in the range of 1000 to 3000 tokens. This forms an inverted U-shaped curve, indicating that too much contextual information may lead to redundancy, which in turn affects performance.

[0043] In summary, the performance scores demonstrate the potential of large language models (LLMs) in entity recognition tasks and emphasize the importance of token size and context information. The analysis shows that the performance of entity extraction shows an inverted U-shaped curve with token size, highlighting the importance of token size optimization in entity extraction tasks. By optimizing these factors, the accuracy and efficiency of entity extraction can be improved, resulting in higher quality information and having a profound impact on various applications.

[0044] In addition, it should be noted that the shapes and names of the parts and components of the specific embodiments described in this specification may be different, and the above content described in this specification is only an example of the structure of the present invention. All equivalent changes or simple changes made based on the structure, features and principles described in the patent concept of the present invention are included in the protection scope of the patent of the present invention. Technicians in the technical field of the present invention can make various modifications or supplements to the specific embodiments described or replace them in a similar manner, as long as they do not deviate from the structure of the present invention or exceed the scope defined by the claims, they should all fall within the protection scope of the present invention.

Claims

1. A method for constructing a dynamic corpus of global talent flow, characterized by: The steps include:

1. Collecting diachronic narrative texts of global talent mobility dynamics through the Internet; (ii) Using large language models (LLMs) to extract talent mobility information from diachronic narrative texts; (3) Align the extraction results of different models in the large language model LLMs; (IV) Compile the aligned extraction results into a dynamic corpus of global talent flow.

2. The method for constructing a dynamic corpus of global talent flow according to claim 1 is characterized by: The diachronic narrative text described is capable of recording information and tracking the dynamics of long-term talent flows.

3. The method for constructing a dynamic corpus of global talent flow according to claim 1 is characterized by: Before the extraction in step (ii), the large language model LLMs is subjected to prior information construction and structural fine-tuning.

4. The method for constructing a dynamic corpus of global talent flow according to claim 1 is characterized by: For the C4 dataset in the large language model LLMs, the steps for constructing prior information and structural fine-tuning are as follows: (1) Distinguishing diachronic narrative texts based on the conditions of explicit and implicit talent mobility texts; (2) Select prompt words that are suitable for the talent flow extraction task to guide the response of the large language model LLMs; (3) Responses containing necessary talent flow features are selected from the generated texts, and the LoRa method is used to further fine-tune the large language model (LLMs).

5. The method for constructing a dynamic corpus of global talent flow according to claim 1 is characterized by: The corpus is divided into three different parts: time series corpus, static corpus and negative sample corpus.

6. The method for constructing a dynamic corpus of global talent flow according to claim 1 is characterized by: In step (ii), the talent flow information includes the country of departure, country of destination and timestamp of the talent flow.

7. The method for constructing a dynamic corpus of global talent flow according to claim 6 is characterized by: The timestamps are standardized to the YYYY-MM format.

8. The method for constructing a dynamic corpus of global talent flow according to claim 6 is characterized by: In step (ii), the geographic entities identified by the large language models (LLMs) are standardized and mapped to the country level for extraction.

9. The method for constructing a dynamic corpus of global talent flow according to claim 1 is characterized by: In step (iii), only the outputs that are completely consistent between all models are retained, and any inconsistent parts are discarded.

10. The method for constructing a dynamic corpus of global talent flow according to claim 1 is characterized by: In step (iii), an ensemble method is used to combine the prediction results of multiple large language models (LLMs) by outputting the average value or using a voting mechanism to fuse the predictions of multiple models.

Images