Model input information processing method and device, equipment and storage medium

Abstract

The invention relates to the technical field of data processing, and discloses a model input information processing method and device, equipment and a storage medium. Judging the information type of the initial input information, and determining a feature extraction mode based on the information type; extracting a plurality of keywords from the initial input information according to the feature extraction mode; performing information retrieval in a preset database based on the plurality of keywords by utilizing a similarity algorithm to obtain an information retrieval result; and combining the initial input information and the information retrieval result to generate model input information. According to the method, more refined data analysis is realized, the data analysis efficiency is improved, and the comprehensiveness and accuracy of model input information are improved.

Description

Technical Field

[0001] The present invention relates to the field of data processing technology, and in particular to a model input information processing method, device, equipment and storage medium. Background Art

[0002] Traditional NLP models mainly rely on large-scale pre-trained models, which have limitations in the use of domain-specific knowledge and do not handle data sparsity well. Traditional models require a large amount of labeled data to achieve good performance, which is particularly difficult in certain professional fields. Although pre-trained models can capture certain semantic information, they are still insufficient in terms of complex context understanding and reasoning. When encountering new domains or specific tasks, the effectiveness of the model is reduced and additional time and resources are required for fine-tuning. The difficult-to-understand decision-making process of deep learning models poses a challenge to application scenarios that require transparency.

[0003] Therefore, current traditional models suffer from low data efficiency, limited context understanding, poor adaptability and poor interpretability. Summary of the invention

[0004] The present invention provides a model input information processing method, device, equipment and storage medium, which realizes more refined data analysis, speeds up data analysis efficiency, and improves the comprehensiveness and accuracy of model input information.

[0005] In order to solve the above technical problems, the present invention provides a model input information processing method, comprising:

[0006] Get initial input information;

[0007] Determining the information type of the initial input information, and determining a feature extraction method based on the information type; wherein the information type includes text information, video information, image information and audio information;

[0008] According to the feature extraction method, extracting a number of keywords from the initial input information;

[0009] Using a similarity algorithm, based on a number of the keywords, information retrieval is performed in a preset database to obtain information retrieval results;

[0010] The initial input information and the information retrieval result are combined to generate model input information.

[0011] The present invention obtains initial input information; determines which type of information the initial input information belongs to among text information, video information, image information and audio information, and determines different feature extraction methods according to different information types, and extracts a number of keywords from the initial input information based on the determined feature extraction method, thereby realizing more refined data analysis and improving the accuracy and efficiency of keyword extraction; based on a number of keywords, performs information retrieval in a preset database to obtain information retrieval results, thereby improving the comprehensiveness of model input information; merges the initial input information and the information retrieval results to generate model input information, and improves the relevance and accuracy of subsequent model output results by combining the initial input information and the information retrieval results obtained by the retrieval.

[0012] Further, the determining of the information type of the initial input information and determining the feature extraction method based on the information type is specifically:

[0013] When the initial input information is text information, determining the feature extraction method to be a text recognition algorithm;

[0014] When the initial input information is video information or image information, determining the feature extraction method is an image recognition algorithm;

[0015] When the initial input information is audio information, the feature extraction method is determined to be speech-to-text technology and text recognition algorithm.

[0016] Furthermore, the extracting of several keywords from the initial input information according to the feature extraction method includes:

[0017] When the initial input information is text information, the initial input information is subjected to text preprocessing; wherein the text preprocessing includes word segmentation processing and stemming processing;

[0018] A text recognition algorithm is used to extract features from the initial input information after text preprocessing to obtain several keywords.

[0019] Furthermore, the extracting of several keywords from the initial input information according to the feature extraction method includes:

[0020] When the initial input information is video information or image information, performing image preprocessing on the initial input information;

[0021] The image recognition algorithm is used to extract features from the initial input information after image preprocessing to obtain several keywords.

[0022] Furthermore, the extracting of several keywords from the initial input information according to the feature extraction method includes:

[0023] When the initial input information is audio information, converting the initial input information from audio information into text information using speech-to-text technology;

[0024] Performing text preprocessing on the initial input information converted into text information;

[0025] A text recognition algorithm is used to extract features from the initial input information after text preprocessing to obtain several keywords.

[0026] Furthermore, the similarity algorithm is used to perform information retrieval in a preset database based on a number of the keywords to obtain information retrieval results, specifically:

[0027] Converting the plurality of keywords into a plurality of keyword vectors;

[0028] Using a similarity algorithm, respectively calculating the vector similarity between each of the keyword vectors and each of the preset vectors in the preset vector database;

[0029] The preset vectors having the greatest vector similarity to the keyword vectors are determined as information retrieval results.

[0030] Furthermore, the initial input information and the information retrieval result are combined to generate model input information, specifically:

[0031] Based on the vector similarity of each preset vector in the information retrieval result, a weighted average method is used to perform weighted merging of several keywords in the initial input information and each preset vector to form an initial merged result set;

[0032] Obtaining a weight value of each preset vector in a preset vector database;

[0033] Based on the weight value of each preset vector, a Rerank algorithm is used to reorder each preset vector to form model input information.

[0034] Accordingly, the present invention provides a model input information processing device, comprising: an acquisition module, a determination module, an extraction module, a retrieval module and a generation module;

[0035] The acquisition module is used to acquire initial input information;

[0036] The determination module is used to determine the information type of the initial input information and determine the feature extraction method based on the information type; wherein the information type includes text information, video information, image information and audio information;

[0037] The extraction module is used to extract a number of keywords from the initial input information according to the feature extraction method;

[0038] The retrieval module is used to perform information retrieval in a preset database based on a number of the keywords using a similarity algorithm to obtain information retrieval results;

[0039] The generation module is used to merge the initial input information and the information retrieval result to generate model input information.

[0040] The present invention also provides a model input information processing device, comprising a processor, a memory, and a computer program stored in the memory and configured to be executed by the processor, wherein the processor implements the model input information processing method as described above when executing the computer program.

[0041] The present invention also provides a computer-readable storage medium, which includes a stored computer program, wherein when the computer program is running, the device where the computer-readable storage medium is located is controlled to execute the model input information processing method as described above. BRIEF DESCRIPTION OF THE DRAWINGS

[0042] Figure 1 A schematic diagram of a flow chart of an embodiment of the model input information processing method provided by the present invention;

[0043] Figure 2 A schematic diagram of the structure of an embodiment of a model input information processing device provided by the present invention;

[0044] Figure 3 A schematic structural diagram of an embodiment of a model input information processing device provided by the present invention. DETAILED DESCRIPTION

[0045] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0046] The flowcharts shown in the accompanying drawings are only examples and do not necessarily include all the contents and operations / steps, nor must they be executed in the order described. For example, some operations / steps may also be decomposed, combined or partially merged, so the actual execution order may change according to actual conditions.

[0047] Some embodiments of the present invention are described in detail below in conjunction with the accompanying drawings. In the absence of conflict, the following embodiments and features in the embodiments can be combined with each other.

[0048] like Figure 1FIG. 1 is a flow chart of an embodiment of a method for processing model input information provided by the present invention. The method includes steps 101 to 105, and each step is specifically as follows:

[0049] Step 101: Obtain initial input information.

[0050] In the first embodiment of the present invention, the initial input information is the initial information of the model input. If the information is directly used as the input information of the model, the model result obtained will have the defect of poor accuracy. Therefore, after obtaining the initial input information, it is necessary to perform data processing on the initial input information to improve the accuracy of the model input information, thereby obtaining a more accurate model output result.

[0051] Step 102: Determine the information type of the initial input information, and determine a feature extraction method based on the information type; wherein the information type includes text information, video information, image information, and audio information.

[0052] Furthermore, in the first embodiment of the present invention, the information type of the initial input information is determined, and the feature extraction method is determined based on the information type, specifically:

[0053] When the initial input information is text information, determining the feature extraction method to be a text recognition algorithm;

[0054] When the initial input information is video information or image information, determining the feature extraction method is an image recognition algorithm;

[0055] When the initial input information is audio information, the feature extraction method is determined to be speech-to-text technology and text recognition algorithm.

[0056] In the first embodiment of the present invention, for multimodal initial input information, corresponding feature extraction methods can be used to extract features from the initial input information. When the initial input information is text information, a text recognition algorithm can be used to extract features, and information retrieval can be performed through text parsing and semantic understanding. Among them, the text recognition algorithm is trained according to a preset database. When the initial input information is video information or image information, OCR technology and image recognition algorithm are used to extract key information. When the initial input information is audio information, the audio information is converted into text information through speech-to-text technology, and then further processed. The present invention supports multimodal information retrieval and generation, covering multiple media types such as text, images, and audio. Different feature extraction methods are used for feature extraction for different types of information. These differentiated processing methods ensure that different types of data can be processed efficiently and accurately, providing richer and more accurate services.

[0057] Step 103: extracting a number of keywords from the initial input information according to the feature extraction method.

[0058] Furthermore, in the first embodiment of the present invention, according to the feature extraction method, several keywords are extracted from the initial input information, including:

[0059] When the initial input information is text information, the initial input information is subjected to text preprocessing; wherein the text preprocessing includes word segmentation processing and stemming processing;

[0060] A text recognition algorithm is used to extract features from the initial input information after text preprocessing to obtain several keywords.

[0061] In the first embodiment of the present invention, for text input, documents in various formats, such as PDF, Word, Excel, etc., are first intelligently identified and parsed, and text information is extracted from these unstructured data. Pre-processing operations such as word segmentation and stemming are performed on the text information, and then a pre-trained text recognition algorithm is used to extract features from the pre-processed text information to accurately obtain key information and generate a number of keywords.

[0062] Furthermore, in the first embodiment of the present invention, according to the feature extraction method, several keywords are extracted from the initial input information, including:

[0063] When the initial input information is video information or image information, performing image preprocessing on the initial input information;

[0064] The image recognition algorithm is used to extract features from the initial input information after image preprocessing to obtain several keywords.

[0065] In the first embodiment of the present invention, for video input or image input, an image is first extracted from the video, and preprocessing operations such as image denoising, image enhancement and image size adjustment are performed on the image. Then, OCR technology and image recognition algorithm are used to extract features from the image to accurately obtain key information and generate a number of keywords.

[0066] Furthermore, in the first embodiment of the present invention, according to the feature extraction method, several keywords are extracted from the initial input information, including:

[0067] When the initial input information is audio information, converting the initial input information from audio information into text information using speech-to-text technology;

[0068] Performing text preprocessing on the initial input information converted into text information;

[0069] A text recognition algorithm is used to extract features from the initial input information after text preprocessing to obtain several keywords.

[0070] In the first embodiment of the present invention, for audio input, the audio content is first converted into text using speech-to-text technology, and the text is preprocessed such as word segmentation and stemming. Then, a pre-trained text recognition algorithm is used to extract features from the preprocessed text to accurately obtain key information and generate a number of keywords.

[0071] The present invention adopts different keyword extraction methods for different types of data, which can not only understand the text content, but also parse image data and audio data, ensuring the high quality of input data. The present invention supports multimodal information retrieval and generation, and can select appropriate templates according to different data types to achieve more accurate and efficient text segmentation, thereby improving processing efficiency, making the entire processing process more transparent and controllable, generating more accurate model input information, and improving the relevance and accuracy of model output results.

[0072] Step 104: Using a similarity algorithm, based on a number of the keywords, perform information retrieval in a preset database to obtain information retrieval results.

[0073] Furthermore, in the first embodiment of the present invention, a similarity algorithm is used to perform information retrieval in a preset database based on a number of the keywords to obtain information retrieval results, specifically:

[0074] Converting the plurality of keywords into a plurality of keyword vectors;

[0075] Using a similarity algorithm, respectively calculating the vector similarity between each of the keyword vectors and each of the preset vectors in the preset vector database;

[0076] The preset vectors having the greatest vector similarity to the keyword vectors are determined as information retrieval results.

[0077] In the first embodiment of the present invention, the relevant document fragments are searched in the preset database according to the extracted keywords, and the relevant document fragments are determined as information retrieval results. The preset database is a vector database, so before performing information retrieval in the preset database, the extracted keywords are first converted into vector format representation, and then the most relevant document fragments are found in the vector database by calculating vector similarity to obtain information retrieval results.

[0078] The present invention adopts a hybrid retrieval technology, combining the traditional rule-based retrieval method and the semantic understanding ability of the deep learning model. Compared with traditional retrieval, the present invention can perform more refined analysis and processing of model input information, and can achieve efficient retrieval through the deep learning model.

[0079] Step 105: Merge the initial input information and the information retrieval result to generate model input information.

[0080] Furthermore, in the first embodiment of the present invention, the initial input information and the information retrieval result are combined to generate model input information, specifically:

[0081] Based on the vector similarity of each preset vector in the information retrieval result, a weighted average method is used to perform weighted merging of several keywords in the initial input information and each preset vector to form an initial merged result set;

[0082] Obtaining a weight value of each preset vector in a preset vector database;

[0083] Based on the weight value of each preset vector, a Rerank algorithm is used to reorder each preset vector to form model input information.

[0084] In the first embodiment of the present invention, after the information retrieval results are screened out in the preset database through information retrieval, each preset vector and its corresponding keyword can be weighted and merged based on the vector similarity of each preset vector in the information retrieval results to form an initial merged result set. The initial merged result set illustrates that when determining the model input information, the importance of the original query is considered and the supporting information from external resources is fully utilized. Then, the weight value of each preset vector is obtained in the preset vector database, and the weight value is determined based on multiple factors, such as the relevance score of the document to which the preset vector belongs, the release time and other factors. Based on the weight value of each preset vector, the Rerank algorithm can be used to rearrange each vector in the initial merged result set to form the model input information so that the vector that is more in line with the user's intention is sorted in front to obtain a query result that is more in line with the user's intention. Therefore, the model input information contains carefully selected and organized content, which helps to improve the quality and accuracy of the model's answers.

[0085] In the first embodiment of the present invention, after the model input information is generated, the model input information is input into the model, and a more accurate model output result can be obtained. In the process of using the model to input the model information, the structure and parameters of the model are optimized in real time, and data enhancement, network structure optimization, training strategy adjustment and other means can be used, such as using deep separable convolution to reduce the amount of calculation, using the attention mechanism to focus on important information, implementing a dynamic learning rate scheduling strategy to prevent overfitting, and reasonably setting the batch size and sequence length to improve the model bandwidth utilization (MBU), ensuring that the model is both efficient and stable when processing complex tasks, so as to better process the model input information.

[0086] See also Figure 2, is a schematic diagram of the structure of an embodiment of a model input information processing device provided by the present invention, the device includes an acquisition module, a determination module, an extraction module, a retrieval module and a generation module;

[0087] The acquisition module 201 is used to acquire initial input information;

[0088] The determination module 202 is used to determine the information type of the initial input information and determine the feature extraction method based on the information type;

[0089] The extraction module 203 is used to extract a number of keywords from the initial input information according to the feature extraction method;

[0090] The search module 204 is used to search for information in a preset database based on a number of the keywords to obtain information search results;

[0091] The generation module 205 is used to merge the initial input information and the information retrieval result to generate model input information.

[0092] Furthermore, in the second embodiment of the present invention, the information type of the initial input information is determined, and the feature extraction method is determined based on the information type, specifically:

[0093] When the initial input information is text information, determining the feature extraction method to be a text recognition algorithm;

[0094] When the initial input information is video information or image information, determining the feature extraction method is an image recognition algorithm;

[0095] When the initial input information is audio information, the feature extraction method is determined to be speech-to-text technology and text recognition algorithm.

[0096] Furthermore, in the second embodiment of the present invention, according to the feature extraction method, several keywords are extracted from the initial input information, including:

[0097] When the initial input information is text information, the initial input information is subjected to text preprocessing; wherein the text preprocessing includes word segmentation processing and stemming processing;

[0098] A text recognition algorithm is used to extract features from the initial input information after text preprocessing to obtain several keywords.

[0099] Furthermore, in the second embodiment of the present invention, according to the feature extraction method, a number of keywords are extracted from the initial input information, including:

[0100] When the initial input information is video information or image information, performing image preprocessing on the initial input information;

[0101] The image recognition algorithm is used to extract features from the initial input information after image preprocessing to obtain several keywords.

[0102] Furthermore, in the second embodiment of the present invention, according to the feature extraction method, a number of keywords are extracted from the initial input information, including:

[0103] When the initial input information is audio information, converting the initial input information from audio information into text information using speech-to-text technology;

[0104] Performing text preprocessing on the initial input information converted into text information;

[0105] A text recognition algorithm is used to extract features from the initial input information after text preprocessing to obtain several keywords.

[0106] Furthermore, in the second embodiment of the present invention, based on the plurality of keywords, information retrieval is performed in a preset database to obtain information retrieval results, specifically:

[0107] Converting the plurality of keywords into a plurality of keyword vectors;

[0108] Calculating respectively the vector similarity between each of the keyword vectors and each preset vector in the preset vector database;

[0109] The preset vectors having the greatest vector similarity to the keyword vectors are determined as information retrieval results.

[0110] Furthermore, in the second embodiment of the present invention, the initial input information and the information retrieval result are combined to generate model input information, specifically:

[0111] Based on the vector similarity of each preset vector in the information retrieval result, the initial input information and the information retrieval result are merged by using a weighted average method to obtain a merged result;

[0112] The combined results are reordered using the Rerand algorithm to obtain model input information.

[0113] See also Figure 3 , is a structural diagram of an embodiment of a model input information processing device provided by the present invention. The device may be a server, including a processor, a memory and a network interface connected through a system bus, wherein the memory may include a non-volatile storage medium and an internal memory.

[0114] The processor is used to provide computing and control capabilities and support the operation of the entire computer equipment.

[0115] The non-volatile storage medium can store an operating system and a computer program. The computer program includes program instructions, and when the program instructions are executed, the processor can execute any one of the P2P-based broadcast system data transmission methods.

[0116] The internal memory provides an environment for the operation of the computer program in the non-volatile storage medium. When the computer program is executed by the processor, the processor can execute any P2P-based broadcast system data transmission method.

[0117] The network interface is used for network communication, such as sending assigned tasks.

[0118] Those skilled in the art will understand that Figure 3 The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0119] It should be understood that the processor may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. Among them, the general-purpose processor may be a microprocessor or the processor may also be any conventional processor, etc.

[0120] An embodiment of the present invention also provides a computer-readable storage medium, including a processor, a memory, and a computer program stored in the memory and configured to be executed by the processor, and when the processor executes the computer program, the model input information processing method described in any of the above embodiments is implemented.

[0121] In summary, the present invention provides a model input information processing method, device, equipment and storage medium to obtain initial input information; determine which type of information the initial input information belongs to among text information, video information, image information and audio information, and determine different feature extraction methods according to different information types, extract a number of keywords from the initial input information based on the determined feature extraction method, thereby achieving more refined data analysis and improving the accuracy and efficiency of keyword extraction; based on a number of keywords, perform information retrieval in a preset database to obtain information retrieval results, thereby improving the comprehensiveness of the model input information; merge the initial input information and the information retrieval results to generate model input information, and improve the relevance and accuracy of subsequent model output results by combining the initial input information and the information retrieval results obtained by the retrieval.

[0122] The specific embodiments described above further illustrate the purpose, technical solutions and beneficial effects of the present invention. It should be understood that the above description is only a specific embodiment of the present invention and is not intended to limit the scope of protection of the present invention. It is particularly pointed out that for those skilled in the art, any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention should be included in the scope of protection of the present invention.

Claims

1. A model input information processing method, characterized in that: include: Get initial input information; Determining the information type of the initial input information, and determining a feature extraction method based on the information type; wherein the information type includes text information, video information, image information and audio information; According to the feature extraction method, extracting a number of keywords from the initial input information; Using a similarity algorithm, based on a number of the keywords, information retrieval is performed in a preset database to obtain information retrieval results; The initial input information and the information retrieval result are combined to generate model input information.

2. The model input information processing method according to claim 1, characterized in that: The determining of the information type of the initial input information and determining the feature extraction method based on the information type is specifically: When the initial input information is text information, determining the feature extraction method to be a text recognition algorithm; When the initial input information is video information or image information, determining the feature extraction method is an image recognition algorithm; When the initial input information is audio information, the feature extraction method is determined to be speech-to-text technology and text recognition algorithm.

3. The model input information processing method according to claim 2, characterized in that: According to the feature extraction method, a number of keywords are extracted from the initial input information, including: When the initial input information is text information, the initial input information is subjected to text preprocessing; wherein the text preprocessing includes word segmentation processing and stemming processing; A text recognition algorithm is used to extract features from the initial input information after text preprocessing to obtain several keywords.

4. The model input information processing method according to claim 2, characterized in that: According to the feature extraction method, a number of keywords are extracted from the initial input information, including: When the initial input information is video information or image information, performing image preprocessing on the initial input information; The image recognition algorithm is used to extract features from the initial input information after image preprocessing to obtain several keywords.

5. The model input information processing method according to claim 2, characterized in that: According to the feature extraction method, a number of keywords are extracted from the initial input information, including: When the initial input information is audio information, converting the initial input information from audio information into text information using speech-to-text technology; Performing text preprocessing on the initial input information converted into text information; A text recognition algorithm is used to extract features from the initial input information after text preprocessing to obtain several keywords.

6. The model input information processing method according to claim 1, characterized in that: The similarity algorithm is used to perform information retrieval in a preset database based on a number of the keywords to obtain information retrieval results, specifically: Converting the plurality of keywords into a plurality of keyword vectors; Using a similarity algorithm, respectively calculating the vector similarity between each of the keyword vectors and each of the preset vectors in the preset vector database; The preset vectors having the greatest vector similarity to the keyword vectors are determined as information retrieval results.

7. The model input information processing method according to claim 6, characterized in that: The initial input information and the information retrieval result are combined to generate model input information, specifically: Based on the vector similarity of each preset vector in the information retrieval result, a weighted average method is used to perform weighted merging of several keywords in the initial input information and each preset vector to form an initial merged result set; Obtaining a weight value of each preset vector in a preset vector database; Based on the weight value of each preset vector, a Rerank algorithm is used to reorder each preset vector to form model input information.

8. A model input information processing device, characterized in that: include: Acquisition module, determination module, extraction module, retrieval module and generation module; The acquisition module is used to acquire initial input information; The determination module is used to determine the information type of the initial input information and determine the feature extraction method based on the information type; wherein the information type includes text information, video information, image information and audio information; The extraction module is used to extract a number of keywords from the initial input information according to the feature extraction method; The retrieval module is used to perform information retrieval in a preset database based on a number of the keywords using a similarity algorithm to obtain information retrieval results; The generation module is used to merge the initial input information and the information retrieval result to generate model input information.

9. A model input information processing device, characterized in that: The method comprises a processor, a memory, and a computer program stored in the memory and configured to be executed by the processor, wherein the processor implements the model input information processing method as claimed in any one of claims 1 to 7 when executing the computer program.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium includes a stored computer program, wherein when the computer program is executed, the device where the computer-readable storage medium is located is controlled to execute the model input information processing method according to any one of claims 1 to 7.

Images