Method for sensing and processing key information in correlation model

By extending Query and Magic Box information, optimizing the model structure, and using multi-task self-training and fie l d embedd i ng technology, the problem of correlation understanding between Query and Magic Box in automotive APP is solved, and the accuracy of correlation discrimination and user experience is improved.

CN118939759APending Publication Date: 2024-11-12TIANJIN AUTOHOME DATA INFORMATION TECH CO LTD
View PDF 2 Cites 0 Cited by

Patent Information

Application Number
CN202410993520.0
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-07-23
Publication Date
2024-11-12

AI Technical Summary

Technical Problem

In the automotive APP scenario, the Query entered by the user is short but rich in demand. It is difficult for the existing correlation model to deeply understand the complex relationship between Query and the magic box, resulting in information imbalance, affecting the accuracy of the search and recommendation system and user experience. .

Method used

By extending the information of Query and Magic Box, optimizing the model structure, using multi-task self-training and fie l d embedd i ng technology, the model's understanding and distinction of multi-field information is enhanced, the extended sample format is constructed, and the multi-task self-training is adopted The training framework improves the implicit perception of the model.

Benefits of technology

It significantly improves the correlation discrimination accuracy and stability of the model, enhances the understanding of complex data, improves the correlation between user experience and search results, and ensures higher discrimination accuracy and system performance.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118939759A_ABST
    Figure CN118939759A_ABST
Patent Text Reader

Abstract

The invention belongs to the technical field of information, and discloses a method for sensing and processing key information in a correlation model, and the method comprises the following specific steps: S1, defining a problem and a target, S2, carrying out query side explicit information extension, S3, carrying out magic box side explicit information extension, S4, designing field embed, S5, constructing an expanded sample format, S6, implementing multi-task self-training, and S7, carrying out evaluation and optimization. By constructing the auxiliary task, introducing the soft label and combining a multi-task training framework and a self-training strategy, the performance of the model can be remarkably improved, the soft label and the model generated by the pre-training model can be used for implicitly capturing and learning deep and rich information in a sample, the ability of understanding complex data is enhanced, and meanwhile, the method has the advantages of being simple in structure and convenient to operate. The multi-task learning framework promotes the model to be optimized on a plurality of related tasks at the same time, and the generalization ability and robustness of the model are effectively improved.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention belongs to the field of information technology, and specifically relates to a method for sensing and processing key information in a correlation model. Background Art

[0002] In the automotive APP scenario, relevance judgment faces significant information imbalance challenges. Although the query entered by the user is short, it often contains rich needs and intentions, requiring the system to capture quickly and respond accurately. Unlike general scenarios, the relevance model of the automotive APP not only needs to carefully compare the direct match between the query and the magic box title, but also needs to deeply consider the complex relationship between the query and the magic box type, the embedded model title and even the detailed attribute fields of the model. This multi-dimensional and deep matching mechanism is designed to ensure that users can immediately obtain the car information that is closest to their needs and preferences, thereby greatly improving user experience and satisfaction. In the search and recommendation system of the automotive APP, it is crucial to process the precise matching of QP information such as query intent and entity with the magic box content. The short form of the query often contains rich intentions and needs, while the magic box carries detailed model information, attribute data and specific type descriptions. It is difficult to deeply explore and accurately understand the internal connection between these key information by relying solely on the surface matching of text information. Summary of the invention

[0003] The purpose of the present invention is to provide a method for sensing and processing key information in a correlation model to solve the problems raised in the above background technology.

[0004] In order to achieve the above object, the present invention provides the following technical solution: a method for sensing and processing key information in a correlation model, the method comprising the following specific steps:

[0005] S1: Define the problem and goal: To address the imbalance between query and magic box information, we will expand information and optimize models to enhance the recognition of key information and improve the ability to discriminate relevance.

[0006] S2: Query-side explicit information expansion: collects query intent, entity, and alias extension information, and integrates it into the sample in a structured manner to enhance query information richness and improve model processing efficiency;

[0007] S3: Explicit information expansion on the magic box side: The magic box side introduces displacement and power structured attributes, expands the magic box sample information, and improves the sample expression richness and correlation discrimination ability;

[0008] S4: Design field embedding: optimize Bert, integrate field embedding to distinguish fields, fuse multiple embedding inputs, and enhance the model's understanding and discrimination of multi-field information;

[0009] S5: Construct the extended sample format: format the extended query and magic box information, organize the samples in a specific format, and use them as training data to prepare for model training;

[0010] S6: Implement multi-task self-training: Construct QP intent and entity soft labels as auxiliary tasks, combine them with main relevance discrimination, and enhance the model's implicit perception of scene information through multi-task self-training;

[0011] S7: Evaluation and Optimization: Evaluate model performance, optimize and adjust based on the results, and iterate to improve discrimination performance and stability.

[0012] Preferably, the specific steps of defining the problem and the goal in S1 are as follows:

[0013] Step 1: Information expansion: First, analyze the information difference between the query and the magic box side, enrich the query side with intent and entity information, and introduce certain structured attribute data to the magic box side to narrow the information gap between the two.

[0014] Step 2: Model optimization: At the model level, the field embedding mechanism is introduced to enhance the model's ability to distinguish and perceive the expanded information. At the same time, a multi-task self-training strategy is adopted to enable the model to implicitly learn and understand the deep relationship between the query and the magic box;

[0015] Step 3: Performance evaluation and iteration: Evaluate the correlation discrimination performance of the optimized model.

[0016] Preferably, the query-side explicit information expansion in S2 refers to deeply mining the rich information behind each query, parsing the query intent, identifying key entities and their aliases, collecting matching keywords, and extracting entities corresponding to the query title from natural high-flow articles. Subsequently, these carefully collected extended information are integrated into the query sample in a rigorous structured form to ensure that each query is accompanied by detailed context and related data.

[0017] Preferably, the explicit information expansion on the Magic Box side in S3 refers to paying special attention to the information improvement on the Magic Box side when optimizing search relevance. To this end, a series of highly relevant structured attribute data such as displacement, power, brand, and manufacturer are carefully introduced. These data not only accurately depict the core characteristics of the Magic Box, but also greatly enrich the dimensions of the Magic Box samples, by seamlessly integrating these structured attribute data into the Magic Box samples in the form of extended fields.

[0018] Preferably, the design of field embedding in S4 refers to the innovative definition of field embedding in the process of deep optimization of the Bert model, aiming to further distinguish the complex field information in the sample, and successfully construct a more delicate and powerful information representation framework by seamlessly integrating field embedding into the embedding system of the existing Bert model and deeply integrating it with Token embedding, Segment embedding and Position embedding.

[0019] Preferably, the extended sample format constructed in S5 is to integrate the extended query information and its extended fields with the title, type and structured data of the magic box to form a standard format of "query[SEP] extended fields...\tmagic box title[SEP] type[SEP] structured data...".

[0020] Preferably, the specific steps of implementing the multi-task self-training in S6 are as follows:

[0021] Step 1: Construct auxiliary tasks and generate soft labels

[0022] Model selection: Select the trained QP intent recognition model and entity extraction model;

[0023] Soft label generation: For each query in a sample, use the above model to predict intent and entity and generate the corresponding soft label;

[0024] Task definition: clarify the auxiliary learning tasks, that is, the model needs to learn and optimize the two auxiliary goals, namely intent recognition and entity extraction, at the same time during the training process;

[0025] Step 2: Design a multi-task training framework

[0026] Main task setting: Determine the main relevance judgment task, that is, the model needs to judge the relevance between the query and the magic box;

[0027] Model architecture: Design a model architecture that supports multi-task learning, including a shared bottom layer for extracting common features of text and specific layers for each task;

[0028] Loss function: define a corresponding loss function for each task and optimize these loss functions simultaneously during training;

[0029] Step 3: Perform multi-task self-training

[0030] Initialization: Initialize shared layers using pre-trained model parameters and randomly initialize parameters for specific layers;

[0031] Joint training: During the training process, the data of the main task and the auxiliary task are processed simultaneously. For each batch of data, features are extracted through the shared layer, and then relevance discrimination, intent recognition, and entity extraction are performed through specific layers respectively. The gradient is calculated according to the loss function of each task, and the model parameters are updated through back propagation.

[0032] Iterative optimization: Through multiple rounds of iterative training, the model parameters are continuously optimized to improve the performance of the model on the main task while implicitly learning the intent and entity scene information in the sample. During the training process, the learning rate, weight decay hyperparameters, and the weights of the main task and auxiliary task loss functions are adjusted as needed to achieve the best training effect.

[0033] Preferably, the specific steps of evaluation and optimization in S7 are:

[0034] Step 1: Model Evaluation

[0035] Prepare the test set: Select a test set that is independent of the training set and validation set to ensure that the data distribution of the test set is similar to the actual application scenario;

[0036] Run the test: Use the test set to test the trained model and record the performance indicators of the model on the relevance discrimination task, such as accuracy, recall, and F1 score;

[0037] Performance analysis: Compare the performance indicators of the model on the test set with the previous baseline model and the results of the previous iteration to check whether the model performance has improved and analyze the possible reasons;

[0038] Step 2: Optimization and Adjustment

[0039] Parameter tuning: Based on the evaluation results, adjust the model's hyperparameters to optimize model performance;

[0040] Strategy improvement: If the evaluation results show that the model's performance is insufficient in some aspects, consider improving the sample expansion strategy, adding more diverse extension fields, and optimizing the soft label generation method;

[0041] Model architecture adjustment: adjust the model architecture, add more hidden layers, change activation functions, and improve the complexity and expressiveness of the model;

[0042] Step 3: Iterative Improvement

[0043] Retrain: Retrain the model using the adjusted model parameters, sample expansion strategy, and model architecture;

[0044] Re-evaluate: Use the test set to evaluate the trained model again to check whether the performance has improved.

[0045] Continuous iteration: If the model performance still does not reach the expected level, continue to repeat the above steps for iterative improvement. Through continuous evaluation, adjustment and optimization, the performance and stability of the model can be gradually improved. The beneficial effects of the present invention are as follows:

[0046] The present invention can significantly improve model performance by constructing auxiliary tasks and introducing soft labels, combining a multi-task training framework and a self-training strategy. By using the soft labels generated by the pre-trained model, the model can implicitly capture and learn the deep and rich information in the samples, enhancing the ability to understand complex data. At the same time, the multi-task learning framework enables the model to be optimized on multiple related tasks at the same time, effectively improving the generalization ability and robustness of the model. In the iterative optimization process, the model parameters and training strategies are continuously adjusted to ensure continuous improvement of the model, ultimately achieving higher discrimination accuracy and more stable performance output. BRIEF DESCRIPTION OF THE DRAWINGS

[0047] Figure 1 It is a schematic diagram of the overall process structure of the present invention;

[0048] Figure 2 This is a schematic diagram of explicit information expansion of the present invention;

[0049] Figure 3 This is a flowchart of the multi-task self-training of the present invention. DETAILED DESCRIPTION

[0050] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0051] like Figures 1 to 3 As shown, an embodiment of the present invention provides a method for sensing and processing key information in a correlation model, and the method comprises the following specific steps:

[0052] S1: Define the problem and goal: To address the imbalance between query and magic box information, we will expand information and optimize models to enhance the recognition of key information and improve the ability to discriminate relevance.

[0053] S2: Query-side explicit information expansion: collects query intent, entities, aliases and other extended information, and integrates it into the sample in a structured manner to enhance query information richness and improve model processing efficiency;

[0054] S3: Explicit information expansion on the magic box side: The magic box side introduces structured attributes such as displacement and power to expand the magic box sample information and improve the sample expression richness and correlation judgment capabilities.

[0055] S4: Design field embedding: Optimize Bert, integrate field embedding to distinguish fields, fuse multiple embedding inputs, and enhance the model's understanding and discrimination of multi-field information.

[0056] S5: Construct the extended sample format: format the extended query and magic box information, organize the samples in a specific format, and use them as training data to prepare for model training;

[0057] S6: Implementing multi-task self-training

[0058] Constructing QP intent and entity soft labels as auxiliary tasks, combined with main relevance discrimination, and through multi-task self-training, the model's implicit perception ability of scene information is enhanced.

[0059] S7: Evaluation and Optimization: Evaluate model performance, optimize and adjust based on the results, and iterate to improve discrimination performance and stability.

[0060] The specific steps for defining the problem and the goal in S1 are as follows:

[0061] Step 1: Information expansion: First, analyze the information difference between the query and the magic box side, enrich the intent and entity information on the query side, and introduce necessary structured attribute data to the magic box side to narrow the information gap between the two.

[0062] Step 2: Model optimization: At the model level, the field embedding mechanism is introduced to enhance the model's ability to distinguish and perceive the expanded information. At the same time, strategies such as multi-task self-training are adopted to enable the model to implicitly learn and understand the deep relationship between the query and the magic box;

[0063] Step 3: Performance evaluation and iteration: Evaluate the correlation discrimination performance of the optimized model, adjust the information expansion strategy and model optimization method according to the evaluation results, and continuously improve the overall performance of the model through iteration.

[0064] Through information expansion and model optimization, the information gap between the query and the magic box is effectively narrowed, and the model's perception and understanding of complex information is enhanced. The introduction of field embedding and multi-task self-training strategies not only improves the model's accuracy in distinguishing extended information, but also promotes implicit learning of deep associations between the query and the magic box. Continuous performance evaluation and iterative optimization ensure the steady improvement of model performance, providing a solid guarantee for improving the accuracy of search relevance judgment and user experience.

[0065] The query-side explicit information expansion in S2 refers to the in-depth mining of the rich information behind each query, including parsing the query intent, identifying key entities and their aliases, collecting matching keywords (match words), and extracting entities corresponding to the query title from natural flow high-point articles. Subsequently, these carefully collected extended information are integrated into the query sample in a rigorous structured form to ensure that each query is accompanied by detailed context and related data. Such processing not only enhances the information density of the query, but also lays a solid foundation for the deep learning and precise matching of the subsequent model.

[0066] The query-side explicit information expansion strategy significantly enhances the information density and context relevance of the query by deeply mining the rich information behind the query. This not only improves the richness and expressiveness of query samples, but also provides strong data support for the deep learning and precise matching of subsequent models. The implementation of this strategy effectively promotes the search system's accurate understanding of user needs, improves the relevance and satisfaction of search results, and lays a solid foundation for optimizing user experience and improving search efficiency and effectiveness.

[0067] Among them, the explicit information expansion on the Magic Box side in S3 refers to paying special attention to the information improvement on the Magic Box side when optimizing search relevance. To this end, a series of highly relevant structured attribute data such as displacement, power, brand, and manufacturer are carefully introduced. These data not only accurately depict the core characteristics of the Magic Box, but also greatly enrich the dimensions of the Magic Box samples. By seamlessly integrating these structured attribute data into the Magic Box samples in the form of extended fields, the information density and expressiveness of the samples are successfully improved, providing more comprehensive and detailed data support for subsequent model learning and relevance judgment.

[0068] The Magic Box side-explicit information expansion strategy greatly enriches the dimensions and information density of the Magic Box samples by introducing highly relevant structured attribute data. This move not only accurately depicts the core features of the Magic Box, but also enhances the expressiveness of the samples, providing comprehensive and detailed data support for subsequent model learning and relevance judgment. This move effectively improves the intelligence level of the search system, making relevance judgment more accurate and efficient, bringing users a better search experience, and promoting the innovation and development of search technology.

[0069] Among them, the design of field embedding in S4 refers to the innovative definition of field embedding in the process of deep optimization of Bert model, which aims to further distinguish the complex field information in the sample, and successfully build a more delicate and powerful information representation framework by seamlessly integrating field embedding into the embedding system of the existing Bert model, deeply integrating with Token embedding, Segment embedding and Position embedding. This improvement significantly enhances the model's ability to accurately capture and efficiently distinguish multi-field information, laying a solid foundation for improving the accuracy of search relevance discrimination.

[0070] Designing field embedding and integrating it into the Bert model brings multiple significant advantages. This innovation not only deepens the model's ability to understand complex field information, but also achieves efficient distinction and accurate capture between fields through a refined information representation framework. This deep fusion enhances the model's representation ability, making it more delicate and powerful when processing multi-field data. Ultimately, this improvement provides solid support for improving the accuracy of search relevance judgment, significantly optimizes the user experience, enhances the intelligence and practicality of the system, and is of great significance for promoting the application and development of AI technology in the search field.

[0071] The extended sample format constructed in S5 is to integrate the extended query information and its extended fields (such as intent, entity, etc.) with the title, type and structured data of the magic box (such as displacement, power, etc.) to form a standard format of "query [SEP] extended field...\t magic box title [SEP] type [SEP] structured data..." This formatting process ensures the clarity and completeness of the sample information and provides high-quality data support for subsequent training.

[0072] Integrating the expanded query information and Magic Box data into a standardized format has significant advantages. This formatting process not only ensures the clarity and completeness of the sample information, but also greatly facilitates subsequent data processing and analysis. By clearly defining the boundaries of each field, such as using the "[SEP]" delimiter, information confusion is effectively avoided, and the readability and maintainability of the data are improved. At the same time, high-quality data support lays a solid foundation for model training, helps to improve the accuracy and stability of the model in the relevance judgment task, thereby optimizing the user experience and enhancing the market competitiveness of the product.

[0073] The specific steps of implementing multi-task self-training in S6 are as follows:

[0074] Step 1: Construct auxiliary tasks and generate soft labels

[0075] Model selection: Select the trained QP intent recognition model and entity extraction model;

[0076] Soft label generation: For each query in a sample, use the above model to predict intent and entity and generate corresponding soft labels (i.e. probability distribution rather than hard labels). These soft labels will serve as the target of auxiliary learning tasks;

[0077] Task definition: clarify the auxiliary learning tasks, that is, the model needs to learn and optimize the two auxiliary goals, namely intent recognition and entity extraction, at the same time during the training process;

[0078] Step 2: Design a multi-task training framework

[0079] Main task setting: Determine the main relevance judgment task, that is, the model needs to judge the relevance between the query and the magic box;

[0080] Model architecture: Design a model architecture that supports multi-task learning. Typically, this includes a shared bottom layer (such as BERT's Transformer layer) for extracting common features of text; and specific layers for each task (such as a fully connected layer for relevance discrimination and a sequence annotation layer for intent and entity recognition).

[0081] Loss function: Define a corresponding loss function for each task and optimize these loss functions simultaneously during training; typically, the total loss is a weighted sum of the main task and auxiliary task loss functions.

[0082] Step 3: Perform multi-task self-training

[0083] Initialization: Initialize shared layers using pre-trained model parameters and randomly initialize parameters for specific layers;

[0084] Joint training: During the training process, the data of the main task and the auxiliary task are processed simultaneously. For each batch of data, features are extracted through the shared layer, and then relevance discrimination, intent recognition, and entity extraction are performed through specific layers respectively. The gradient is calculated according to the loss function of each task, and the model parameters are updated through back propagation.

[0085] Iterative optimization: Through multiple rounds of iterative training, the model parameters are continuously optimized to improve the performance of the model on the main task. At the same time, the model can also implicitly learn the scene information such as the intent and entity in the sample. During the training process, the hyperparameters such as learning rate and weight decay, as well as the weights of the main task and auxiliary task loss functions are adjusted as needed to achieve the best training effect.

[0086] Building auxiliary tasks and soft labels, designing a multi-task training framework, and performing self-training can significantly improve model performance. By using pre-trained models to generate soft labels, the model can implicitly learn the rich information in the samples. The multi-task learning framework promotes the model to optimize multiple goals at the same time and enhance generalization capabilities. The iterative optimization process continuously adjusts parameters and strategies to ensure continuous optimization of the model, ultimately achieving higher discrimination accuracy and stability, providing strong support for practical applications.

[0087] Among them, the specific steps of evaluation and optimization in S7 are:

[0088] Step 1: Model Evaluation

[0089] Prepare the test set: Select a test set that is independent of the training set and validation set to ensure that the data distribution of the test set is similar to the actual application scenario;

[0090] Run the test: Use the test set to test the trained model and record the performance indicators of the model on the relevance discrimination task, such as accuracy, recall, and F1 score;

[0091] Performance analysis: Compare the performance indicators of the model on the test set with the previous baseline model and the results of the previous iteration to check whether the model performance has improved and analyze the possible reasons;

[0092] Step 2: Optimization and Adjustment

[0093] Parameter tuning: Based on the evaluation results, adjust the model's hyperparameters (such as learning rate, batch size, regularization coefficient, etc.) to optimize model performance;

[0094] Strategy improvement: If the evaluation results show that the model's performance is insufficient in some aspects, consider improving the sample expansion strategy, such as adding more diverse extension fields and optimizing the soft label generation method;

[0095] Model architecture adjustment: If necessary, the model architecture can also be adjusted, such as adding more hidden layers, changing activation functions, etc., to improve the complexity and expressiveness of the model;

[0096] Step 3: Iterative Improvement

[0097] Retrain: Retrain the model using the adjusted model parameters, sample expansion strategy, and model architecture;

[0098] Re-evaluate: Use the test set to evaluate the trained model again to check whether the performance has improved.

[0099] Continuous iteration: If the model performance still does not reach the expected level, continue to repeat the above steps for iterative improvement. Through continuous evaluation, adjustment and optimization, the performance and stability of the model can be gradually improved.

[0100] Following the above steps of model evaluation, optimization adjustment and iterative improvement can significantly improve the performance and stability of the model. Through independent test set evaluation, the effectiveness of the model in actual application scenarios is ensured; precise adjustments based on evaluation results promote continuous optimization of model performance; the continuous iteration process ensures that the model can adapt to complex and changing data environments and achieve more efficient correlation discrimination. This process not only improves the accuracy of the model, but also enhances its generalization ability and stability, providing more reliable guarantees for practical applications.

[0101] It should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device.

[0102] Although embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that various changes, modifications, substitutions and variations may be made to the embodiments without departing from the principles and spirit of the present invention, and that the scope of the present invention is defined by the appended claims and their equivalents.

Claims

1. A method for sensing and processing key information in a correlation model, characterized in that: The specific steps of this method are as follows: S1: Define the problem and goal: To address the imbalance between query and magic box information, we will expand information and optimize models to enhance the recognition of key information and improve the ability to discriminate relevance. S2: Query-side explicit information expansion: collects query intent, entity, and alias extension information, and integrates it into the sample in a structured manner to enhance query information richness and improve model processing efficiency; S3: Explicit information expansion on the magic box side: The magic box side introduces displacement and power structured attributes, expands the magic box sample information, and improves the sample expression richness and correlation discrimination ability; S4: Design field embedding: optimize Bert, integrate field embedding to distinguish fields, fuse multiple embedding inputs, and enhance the model's understanding and ability to distinguish multi-field information; S5: Construct the extended sample format: format the extended query and magic box information, organize the samples in a specific format, and use them as training data to prepare for model training; S6: Implement multi-task self-training: Construct QP intent and entity soft labels as auxiliary tasks, combine them with main relevance discrimination, and enhance the model's implicit perception of scene information through multi-task self-training; S7: Evaluation and Optimization: Evaluate model performance, optimize and adjust based on the results, and iterate to improve discrimination performance and stability.

2. The method for sensing and processing key information in a correlation model according to claim 1, characterized in that: The specific steps for defining the problem and goal in S1 are as follows: Step 1: Information expansion: First, analyze the information difference between the query and the magic box side, enrich the query side with intent and entity information, and introduce certain structured attribute data to the magic box side to narrow the information gap between the two. Step 2: Model optimization: At the model level, the field embedding mechanism is introduced to enhance the model's ability to distinguish and perceive the expanded information. At the same time, a multi-task self-training strategy is adopted to enable the model to implicitly learn and understand the deep relationship between the query and the magic box; Step 3: Performance evaluation and iteration: Evaluate the correlation discrimination performance of the optimized model.

3. The method for sensing and processing key information in a correlation model according to claim 1, characterized in that: The query-side explicit information expansion in S2 refers to the in-depth mining of the rich information behind each query, including parsing the query intent, identifying key entities and their aliases, collecting matching keywords, and extracting entities corresponding to the query title from natural high-ranking articles. Subsequently, these carefully collected extended information are integrated into the query sample in a rigorous structured form to ensure that each query is accompanied by detailed context and related data.

4. The method for sensing and processing key information in a correlation model according to claim 1, characterized in that: The explicit information expansion on the Magic Box side in S3 refers to paying special attention to the information improvement on the Magic Box side when optimizing search relevance. To this end, a series of highly relevant structured attribute data such as displacement, power, brand, and manufacturer are carefully introduced. These data not only accurately depict the core characteristics of the Magic Box, but also greatly enrich the dimensions of the Magic Box samples, by seamlessly integrating these structured attribute data into the Magic Box samples in the form of extended fields.

5. The method for sensing and processing key information in a correlation model according to claim 1, characterized in that: The design of field embedding in S4 refers to the innovative definition of field embedding in the process of deep optimization of the Bert model, which aims to further distinguish the complex field information in the sample. By seamlessly integrating field embedding into the embedding system of the existing Bert model, and deeply integrating it with Token embedding, Segment embedding and Position embedding, a more delicate and powerful information representation framework is successfully constructed.

6. The method for sensing and processing key information in a correlation model according to claim 1, characterized in that: The extended sample format constructed in S5 is to integrate the extended query information and its extended fields with the title, type and structured data of the magic box to form a standard format of "query[SEP] extended fields...\tmagic box title[SEP] type[SEP] structured data...".

7. The method for sensing and processing key information in a correlation model according to claim 1, characterized in that: The specific steps of implementing multi-task self-training in S6 are as follows: Step 1: Construct auxiliary tasks and generate soft labels Model selection: Select the trained QP intent recognition model and entity extraction model; Soft label generation: For each query in a sample, use the above model to predict intent and entity and generate the corresponding soft label; Task definition: clarify the auxiliary learning tasks, that is, the model needs to learn and optimize the two auxiliary goals, namely intent recognition and entity extraction, at the same time during the training process; Step 2: Design a multi-task training framework Main task setting: Determine the main relevance judgment task, that is, the model needs to judge the relevance between the query and the magic box; Model architecture: Design a model architecture that supports multi-task learning, including a shared bottom layer for extracting common features of text and specific layers for each task; Loss function: define a corresponding loss function for each task and optimize these loss functions simultaneously during training; Step 3: Perform multi-task self-training Initialization: Initialize shared layers using pre-trained model parameters and randomly initialize parameters for specific layers; Joint training: During the training process, the data of the main task and the auxiliary task are processed simultaneously. For each batch of data, features are extracted through the shared layer, and then relevance discrimination, intent recognition, and entity extraction are performed through specific layers respectively. The gradient is calculated according to the loss function of each task, and the model parameters are updated through back propagation. Iterative optimization: Through multiple rounds of iterative training, the model parameters are continuously optimized to improve the performance of the model on the main task while implicitly learning the intent and entity scene information in the sample. During the training process, the learning rate, weight decay hyperparameters, and the weights of the main task and auxiliary task loss functions are adjusted as needed to achieve the best training effect.

8. The method for sensing and processing key information in a correlation model according to claim 1, characterized in that: The specific steps of evaluation and optimization in S7 are: Step 1: Model Evaluation Prepare the test set: Select a test set that is independent of the training set and validation set to ensure that the data distribution of the test set is similar to the actual application scenario; Run the test: Use the test set to test the trained model and record the performance indicators of the model on the relevance discrimination task, such as accuracy, recall, and F1 score; Performance analysis: Compare the performance indicators of the model on the test set with the previous baseline model and the results of the previous iteration to check whether the model performance has improved and analyze the possible reasons; Step 2: Optimization and Adjustment Parameter tuning: Based on the evaluation results, adjust the model's hyperparameters to optimize model performance; Strategy improvement: If the evaluation results show that the model's performance is insufficient in some aspects, consider improving the sample expansion strategy, adding more diverse extension fields, and optimizing the soft label generation method; Model architecture adjustment: adjust the model architecture, add more hidden layers, change activation functions, and improve the complexity and expressiveness of the model; Step 3: Iterative Improvement Retrain: Retrain the model using the adjusted model parameters, sample expansion strategy, and model architecture; Re-evaluate: Use the test set to evaluate the trained model again to check whether the performance has improved. Continuous iteration: If the model performance still does not reach the expected level, continue to repeat the above steps for iterative improvement. Through continuous evaluation, adjustment and optimization, the performance and stability of the model can be gradually improved.

Citation Information

Patent Citations

  • Unmanned ship small sample target detection method based on self-distillation data enhancement

    CN118196391A

  • A valve for preventing backflow

    KR1020230009052A