A knowledge base-based SVN cross-modal retrieval method and system

Abstract

The present invention discloses a knowledge base-based SVN cross-modal retrieval method and system, which mainly relates to the field of natural language processing technology. The method comprises the following steps: capturing change data, obtaining SVN version change information when a file is submitted to the SVN version library; obtaining corresponding change documents based on the SVN version change information; converting the change documents into corresponding vector retrieval and full-text retrieval; obtaining documents that meet the conditions through vector retrieval and full-text retrieval based on user input; rearranging the multiple retrieved documents to find documents that meet the user's search conditions; inputting the file names obtained after rearrangement into the SVN version library for retrieval, obtaining file version related items, calculating the related items, and obtaining the final recall result. The beneficial effect of the present invention is that it achieves a balance between precision and recall, while significantly improving the accuracy and applicability of the retrieval.

Description

Technical Field

[0001] The present invention relates to the technical field of natural language processing, and in particular to a knowledge base-based SVN cross-modal retrieval method and system. Background Art

[0002] SVN, or Subversion, is an open-source version control system. As an open-source version control system, Subversion manages data that changes over time. This data is stored in a central repository. This repository is much like a regular file server, but it remembers every file change. This allows you to restore files to older versions and browse the history of file changes.

[0003] Subversion is a general-purpose system that can be used to manage any type of file, including program source code. However, Subversion does not provide full-text or cross-modal search capabilities. Users who need to find a document must manually search file by file. This manual search becomes increasingly difficult as the number of documents managed increases. In short, there is currently a lack of a method for quickly retrieving required documents from SVN.

[0004] Therefore, there is an urgent need for a knowledge base-based SVN cross-modal retrieval method and system to solve the above problems. Summary of the Invention

[0005] The purpose of the present invention is to provide a knowledge base-based SVN cross-modal retrieval method and system, which achieves a balance between precision and recall, while significantly improving the accuracy and applicability of retrieval.

[0006] To achieve the above-mentioned purpose, the present invention is implemented through the following technical solutions:

[0007] In one aspect, the present invention provides a knowledge base-based SVN cross-modal retrieval method, comprising the following steps:

[0008] Step S1: Capture change data. When a file is submitted to the SVN repository, obtain the SVN version change information.

[0009] Step S2: Obtain the corresponding change document according to the SVN version change information;

[0010] Step S3: Convert the changed document into corresponding vector retrieval and full-text retrieval;

[0011] Step S4: Based on the user input, documents that meet the conditions are obtained through vector search and full-text search;

[0012] Step S5: re-arrange the retrieved documents to find documents that meet the user's search criteria;

[0013] Step S6: The rearranged file names are input into the SVN version library for retrieval to obtain file version related items, and the related items are calculated to obtain the final recall result.

[0014] Preferably, the step S1 is specifically:

[0015] Through the post-commit interface of the SVN repository, you can obtain version change information after the user submits data to the SVN repository and successfully creates a version.

[0016] Preferably, the step S2 is specifically as follows:

[0017] According to the version number of the version change information, the commit record is obtained through svnlog to identify the changed documents. The commit record includes a list of newly added documents, a list of modified documents, and a list of deleted documents.

[0018] Preferably, the step S3 includes:

[0019] Step S31: For an audio file, convert the audio signal into text using a speech recognition model. The audio file format includes but is not limited to: MP3 format and WAV format;

[0020] Step S32: For image files and video files, use the multimodal large model to understand the image or video content and obtain the content text corresponding to the image or video. The image file format includes but is not limited to: PNG format, JPG format; the video file format includes but is not limited to: MP4 format, AVI format;

[0021] Step S33: For a text file, directly obtain the text corresponding to the file, wherein the text file format includes but is not limited to: TXT format, DOC format;

[0022] Step S34: construct a full-text search for the text obtained after processing the various types of files obtained in steps S31-S33, and vectorize the text data using the text vectorization method of natural language processing.

[0023] Preferably, in step S4, the vector search is specifically:

[0024] The user input text is converted into a high-dimensional vector through the Embedding model, and the semantic similarity between the texts is measured by cosine similarity to obtain the vector retrieval result set;

[0025] The full-text search is specifically:

[0026] Based on keywords and user input based on word frequency and inverse document frequency, the BM25 formula is used to perform weighted calculation on keyword matching, and the full-text retrieval result set is obtained through the correlation between the query and the document.

[0027] Preferably, the step S5 is specifically as follows:

[0028] Use the reciprocal ranking fusion algorithm to combine the full-text search ranking and vector search ranking of the same document to obtain a new ranking, and assign a new reciprocal ranking score to each document;

[0029] The reciprocal ranking fusion algorithm uses the ranking of documents in the full-text search result set and the vector search result set to calculate the reciprocal of the ranking of each document in different search result lists, and adds these reciprocals to obtain the fusion score of each document. The documents are reordered according to the fusion score. The calculation process is as follows:

[0030] ;

[0031] in, It's a document. It is a document The inverse ranking score of is the number of search result lists, It is a document In the Ranking in the search result list, is a constant used to smooth the ranking.

[0032] Preferably, the step S6 is specifically as follows:

[0033] Use SVN to query the time and submitter of historical versions;

[0034] Use the svnlog command to display the commit history of a repository, which includes the author, date, revision number, commit message, and changed path of each commit.

[0035] On the other hand, a retrieval system based on the above-mentioned knowledge base-based SVN transmembrane search method is provided, comprising:

[0036] The version change information collection module is used to capture change data and obtain SVN version change information when a file is submitted to the SVN repository.

[0037] The change document acquisition module is used to obtain the corresponding change document according to the SVN version change information;

[0038] Data change module, used to convert change documents into corresponding vector retrieval and full-text retrieval;

[0039] The data retrieval module is used to obtain documents that meet the requirements through vector search and full-text search based on user input;

[0040] The data rearrangement module is used to rearrange the retrieved documents to find the documents that meet the user's search criteria;

[0041] The result output module is used to: input the rearranged file names into the SVN version library for retrieval, obtain file version related items, calculate the related items, and obtain the final recall result.

[0042] Compared with the prior art, the beneficial effects of the present invention are:

[0043] The present invention provides a knowledge base-based SVN cross-modal retrieval method and system, which implements SVN repository document retrieval. By inputting text, voice, image, or video files, it can retrieve not only plain text documents, but also audio files such as mp3 and avi, and image / video files such as mp4 and jpg. Cross-modal semantic matching is achieved through multimodal data such as text, images, or voice. Hybrid retrieval combines the exact matching of keyword-based full-text retrieval with the fuzzy matching of vector retrieval, balancing precision and recall, significantly improving retrieval accuracy and applicability. Ultimately, the results of the hybrid retrieval are recalled using rules to obtain documents that meet the user's query intent and have accurate content. BRIEF DESCRIPTION OF THE DRAWINGS

[0044] Figure 1 is a flow chart of the method of the present invention;

[0045] Figure 2 It is a flowchart of the fusion sorting process of the present invention;

[0046] Figure 3 It is a knowledge base hybrid retrieval flow chart of the present invention;

[0047] Figure 4 It is a candidate file information retrieval flow chart of the present invention;

[0048] Figure 5 This is a display diagram of the recall test results of the present invention;

[0049] Figure 6 It is a schematic diagram of the system structure of the present invention. DETAILED DESCRIPTION

[0050] Below in conjunction with specific embodiment, further set forth the present invention.Should be understood that these embodiments are only used to illustrate the present invention and are not used in limiting the scope of the present invention.In addition, should be understood that after reading the content taught by the present invention, those skilled in the art can make various changes or modifications to the present invention, and these equivalent forms fall within the scope limited by the application equally.

[0051] In the present invention, terms such as "upper", "lower", "left", "right", "front", "rear", "vertical", "horizontal", "side", "bottom", etc. indicate the orientation or positional relationship based on the orientation or positional relationship shown in the drawings. They are only relational terms determined for the convenience of describing the structural relationships of various components or elements of the present invention, and do not specifically refer to any component or element in the present invention, and should not be construed as a limitation to the present invention.

[0052] Embodiment:

[0053] As Figure 1 shown, the present embodiment provides a knowledge - base - based SVN cross - modal retrieval method, including the following steps:

[0054] Step S1: Capture change data. When a file is submitted to the SVN repository, obtain the SVN version change information.

[0055] Step S2: Obtain the corresponding changed document according to the SVN version change information.

[0056] Step S3: Convert the changed document into corresponding vector retrieval and full - text retrieval.

[0057] Step S4: According to the user input, obtain the documents that meet the conditions through vector retrieval and full - text retrieval.

[0058] Step S5: Rearrange the retrieved multiple documents to find the document that meets the user's retrieval conditions.

[0059] Step S6: Input the file names obtained after rearrangement into the SVN repository for retrieval, obtain the file version - related items, and calculate the related items to obtain the final recall result.

[0060] According to the above process, the implementation method is as follows:

[0061] 1. The change data capture module receives the SVN repository version change notice, and obtains the set of changed documents for this change according to the change version number in the change notice:

[0062] The SVN repository server is configured with a post - commit notification program. After the user submits to the svn repository, the post - commit notification is triggered, and the change version information is sent to the change data capture module. The change version information includes the current change version number. After receiving the change version information, the change data capture module uses the version commit log command (svnlog - r <version number> - v <svn address>) to obtain the set of changed documents for this change version.

[0063] 2. Screen out the documents changed this time according to the set of changed documents:

[0064] Receive the changed document set and the change version number, update the local files to the current changed version through the version update command (svn update -r <version number> -v <svn address>), and filter out the newly added documents and updated documents according to the changed document set.

[0065] 3. Extract content according to the file type of the changed documents:

[0066] Classify the newly added documents and updated documents according to the file type, into picture type files such as png and jpg, and text type files such as txt, word, and pdf. For audio documents such as mp3 and wav, use an ASR model (such as Whisper, DeepSpeech, Kaldi, etc.) to identify the information in the audio to obtain the text content; for picture files such as jpg and png and video files such as mp4 and avi, use a multi-modal visual recognition large model (such as Cogvlm-video, Apollo, Video-XL, etc.) to understand the content and output the picture / video corresponding picture description text; for text type documents such as txt and word, use a text processing tool to directly extract the content to obtain the text.

[0067] 4. Vectorize the text obtained from processing various documents, and store it in the knowledge base together with the original text for vector indexing and full-text indexing:

[0068] In the knowledge base, use the bce-embedding model to convert the input content text into a vector representation. Taking each independent document as a unit, store the text content corresponding to the document and the vectorized data representation in the svn document knowledge base.

[0069] 5. Receive the user input, convert the user input into vector features for vector search to obtain a vector search document set, and perform full-text search for keywords based on the original text of the user input to obtain a full-text search document set, as Figure 2 shown:

[0070] The user input can be plain text, speech, picture or video. The hybrid retrieval module receives the user input:

[0071] (1) When the user inputs plain text, extract keywords through word segmentation technology and send them to the svn document knowledge base to perform full-text retrieval to generate a full-text result set. At the same time, after the text is vectorized, perform similarity matching based on the feature vector and output a vector retrieval result set;

[0072] (2) When a user uploads an image or video, the multimodal large model understands the content of the image and outputs the corresponding content text. After the text is segmented to generate keywords, a full-text search is performed to generate a full-text result set. Secondly, the text is vectorized and similarity matching is performed based on the feature vector, and a vector search result set is output;

[0073] (3) When the user inputs voice, the voice is converted into text using speech recognition technology, and keywords are extracted through word segmentation technology and full-text search is performed to generate a full-text result set. At the same time, the text is vectorized and similarity matching is performed based on the feature vector, and the vector search result set is output.

[0074] 6. Rearrange the retrieved documents to find documents that better meet the user's search criteria:

[0075] The query content entered by the user is input into the SVN document knowledge base and a hybrid search strategy is used for search and matching. The hybrid search strategy combines traditional databases and vector databases to perform full-text search and vector search at the same time.

[0076] On the one hand, the SBERT model is used to directly calculate the vector of the query text and perform vector retrieval in the built vector index; on the other hand, the BM25 algorithm is used for full-text retrieval, and finally the reciprocal ranking fusion (RRF) is used for hybrid retrieval, such as Figure 3 As shown in the figure, the final result is a comprehensive ranking of the two search results, and the location is given. The advantage of the reciprocal sort fusion is that it does not use the correlation score, but relies solely on the ranking calculation. It can effectively integrate the two different search methods. According to the relevant document content that the user wants to retrieve, the candidate results of the full-text keyword and vector search are re-ranked to obtain the matching items with the highest comprehensive similarity and output the corresponding file name of the content, achieving the effect of precise positioning and cross-modal retrieval.

[0077] 7. Evaluate the relevance of the sorted documents and select candidate documents 1 to n whose relevance is greater than p:

[0078] Set a relevance of size p. If the relevance value of the best result is greater than the relevance p we set, it means that there are documents in the knowledge base that are related to the current user query. Sort by relevance and provide all documents greater than p.

[0079] 8. If Figure 4 As shown in the figure, the file names of candidate documents 1 to n are input into the SVN version library for retrieval, and the file version related items are obtained. The related items are calculated to obtain the final recall result:

[0080] By using Subversion (SVN) to query the historical version submission information of the candidate file (svnlog candidate file name), the query will display the submission history of the candidate file, including the author, date, revision number (revision number) and other information of each submission.

[0081] The file recall score is calculated based on the returned submission information to obtain the final Top-k search results. The obtained indicator data can be used to calculate the time, submitter, and version difference. The calculation process is as follows:

[0082] ;

[0083] When using SVN for project document management, the document version is usually updated quickly. is the time decay coefficient of the document. A 5% daily decay rate can quickly eliminate outdated content. For example, 7 days after the document is submitted (70% weight retained), 30 days after document submission (Retain 22% weight);

[0084] ;

[0085] When the query condition entered by the user contains the submitter information, the candidate file that matches the submitter is given a weight of 1, otherwise it is 0. Is a flag indicating whether the candidate file matches the submitter. Finally, the line change ratio is used to calculate the version difference of the text document:

[0086] ;

[0087] in To add new rows, To delete the number of rows, The total number of lines in the file. and number of deleted rows The sum of the total number of lines in the file When the ratio is low, the difference score A value closer to 1 indicates greater stability or consistency of the file content. The comprehensive scoring formula is:

[0088]

[0089] The Score value ranges from [0 to 3]. A higher Score indicates that the candidate document is recent, the author matches, and the content is stable. Conversely, a higher Score indicates that the document is old, the author does not match, and the content has been significantly modified. Finally, the documents after the hybrid search are recalled based on the Score, and the top-K most relevant documents that match the user's query are returned.

[0090] In this embodiment, when text documents, audio documents, pictures, and video documents are listed separately and the user query input is set to "Please help me find documents related to tigers", the specific process of the above method is as follows:

[0091] For newly changed text documents in the SVN repository, uploading a ".txt" text file to SVN triggers a post-commit notification, sending the changed version information to the change data capture module. The change module uses the command svn log -r $REV -v $SVN_URL to obtain the changed document set, where $REV is the version number and $SVN_URL is the file address. The command svn update -r $REV $SVN_URL is then executed to update the local working copy to the specified version. The local working copy identifies the file type based on the file suffix. If a text file with the suffix .txt is identified, the text is directly vectorized using the bce-embedding model and saved together with the text content to the SVN document knowledge base.

[0092] For newly modified audio files in the SVN repository, uploading the "Forest King.mp3" audio file to SVN triggers a post-commit notification, sending the revision change information to the change data capture module. The change module then uses the command svn log -r $REV -v $SVN_URL to obtain the changed document set, where $REV is the revision number and $SVN_URL is the file address. The command svn update -r $REV $SVN_URL is then executed to update the local working copy to the specified revision. The local working copy identifies the file type based on the file suffix and finds an audio file with the .mp3 suffix. The Whisper model is used to identify the text content: "The tiger, king of the forest, is draped in gorgeous orange-yellow fur with black stripes, like a robe woven by nature. Its sharp eyes gleam with a calm and majestic light, and every gaze seems to see through everything. Its body is strong and its steps are steady, and every stride is powerful and graceful, as if it can shake the silence of the entire forest." The resulting text content is vectorized using the bce-embedding model and saved together with the text content identified by the Whisper model to the SVN document repository.

[0093] For newly changed images and videos in the SVN repository, uploading the tiger.png image or video.mp4 video file to SVN triggers a post-commit notification, sending the changed version information to the change data capture module. The change module then uses the command svn log -r $REV -v $SVN_URL to obtain the changed document set, where $REV is the version number and $SVN_URL is the file address. The command svn update -r $REV $SVN_URL is then executed to update the local working copy to the specified revision. The local working copy identifies the file type based on the file suffix and identifies an image or video file with a .png or mp4 suffix. The Cogvlm-video model is used to identify the image content as: "This picture shows a majestic Siberian tiger. It is lying leisurely on the green grass. Its fur is mainly orange with typical black stripes. The tiger's face is a mixture of white and orange, with sharp eyes and erect ears, and the overall image is spectacular. The background is a green vegetation, creating a natural and tranquil atmosphere for the entire picture." The Cogvlm-video model is used to identify the video content as: "Video The video shows a large, colorful tiger sculpture made of various materials, including feathers and fabric. The sculpture is displayed at various locations on a city street lined with buildings with Asian influences. Throughout the video, people pass by, stand near the sculpture, or shop. At one point, a shower of pink petals falls, adding to the festive atmosphere. This scene captures the fusion of traditional art and modern urban life, creating a dynamic visual experience…” The resulting text content is vectorized using the bce-embedding model and saved together with the text content recognized by the Cogvlm-video model to the SVN document repository.

[0094] When a user makes a query, for example, "Please help me find documents related to tigers", the knowledge base will perform full-text search and vector search simultaneously based on the user input, and apply the inverse fusion re-ranking step to select the best result that matches the user's question from the two types of query results. Figure 5 As shown in the figure, after hybrid retrieval and re-ranking, the contents of three files are greater than the set threshold p (p=0.50 in this example). We obtain that the file corresponding to candidate file 1 (forest king.txt) is forest king.mp3; the file corresponding to candidate file 2 (tiger.txt) is tiger.png; and the file corresponding to candidate file 3 (video.txt) is video.mp4.

[0095] like Figure 6 As shown, this embodiment also provides a knowledge base-based SVN cross-modal retrieval system, including:

[0096] The version change information collection module is used to capture change data and obtain SVN version change information when a file is submitted to the SVN repository.

[0097] The change document acquisition module is used to obtain the corresponding change document according to the SVN version change information;

[0098] Data change module, used to convert change documents into corresponding vector retrieval and full-text retrieval;

[0099] The data retrieval module is used to obtain documents that meet the requirements through vector search and full-text search based on user input;

[0100] The data rearrangement module is used to rearrange the retrieved documents to find the documents that meet the user's search criteria;

[0101] The result output module is used to: input the rearranged file names into the SVN version library for retrieval, obtain file version related items, calculate the related items, and obtain the final recall result.

[0102] The above is a specific description of the preferred implementation of the present invention, but the present invention is not limited to the embodiments. Those skilled in the art can make various equivalent modifications or substitutions without violating the spirit of the present invention. These equivalent modifications or substitutions are all included in the scope defined by the claims of this application.

Claims

1. A knowledge base-based SVN cross-modal retrieval method, characterized in that: The following steps are involved: Step S1: Capture change data. When a file is submitted to the SVN repository, obtain the SVN version change information. Step S2: Obtain the corresponding change documents based on the SVN version change information, classify the newly added and updated documents according to file type, and use the ASR model to recognize the audio information to obtain the text content for audio documents; use a multimodal visual recognition model to understand the content of image files and video files, and output the corresponding picture description text for the image / video; use text processing tools to directly extract the content of text documents to obtain the text; Step S3: Convert the changed document into corresponding vector retrieval and full-text retrieval; Step S4: Based on the user input, documents that meet the conditions are obtained through vector search and full-text search; Step S5: re-arrange the retrieved documents to find documents that meet the user's search criteria; Step S6: The rearranged file names are input into the SVN version library for retrieval to obtain file version related items, and the related items are calculated to obtain the final recall result. The calculation of the related items includes: The file recall score is calculated based on the returned submission information to obtain the final Top-k search results. The obtained indicator data can be used to calculate the time, submitter, and version difference. The calculation process is as follows: , where t is the number of days of release; When using SVN for project document management, the document version is usually updated quickly. is the time decay coefficient of the document; ； When the query condition entered by the user contains the submitter information, the candidate file that matches the submitter is given a weight of 1, otherwise it is 0. It is a flag that indicates whether the candidate file matches the submitter. The line change ratio is used to calculate the version difference of the text document: ； in To add new rows, To delete the number of rows, The total number of lines in the file. and number of deleted rows The sum of the total number of lines in the file When the ratio is low, the difference score A value close to 1 indicates that the file content is more stable or consistent. The comprehensive scoring formula is: ； The Score value range is between [0,3]. A higher Score value indicates that the candidate document is new, the author matches, and the file content is stable. Conversely, the document is old, the author does not match, and the file content has changed significantly. The documents after hybrid retrieval are recalled based on the Score, and the Top-K best content-related documents that meet the user query are returned.

2. The SVN cross-modal retrieval method based on a knowledge base according to claim 1 is characterized in that: The step S1 is specifically as follows: Through the post-commit interface of the SVN repository, you can obtain version change information after the user submits data to the SVN repository and successfully creates a version.

3. The SVN cross-modal retrieval method based on a knowledge base according to claim 1, characterized in that: The step S2 is specifically as follows: According to the version number of the version change information, the commit record is obtained through svn log to identify the changed documents. The commit record includes a list of newly added documents, a list of modified documents, and a list of deleted documents.

4. The SVN cross-modal retrieval method based on a knowledge base according to claim 1 is characterized in that: The step S3 includes: Step S31: For an audio file, convert the audio signal into text using a speech recognition model. The formats of the audio file include but are not limited to: MP3 format and WAV format; Step S32: For image files and video files, use the multimodal large model to understand the image or video content and obtain the content text corresponding to the image or video. The formats of the image files include but are not limited to PNG and JPG formats, and the formats of the video files include but are not limited to MP4 and AVI formats. Step S33: For a text file, directly obtain the text corresponding to the file. The format of the text file includes but is not limited to: TXT format, DOC format; Step S34: construct a full-text search for the text obtained after processing the various types of files obtained in steps S31-S33, and vectorize the text data using the text vectorization method of natural language processing.

5. The SVN cross-modal retrieval method based on a knowledge base according to claim 1, characterized in that: In step S4, the vector search is specifically as follows: The user input text is converted into a high-dimensional vector through the Embedding model, and the semantic similarity between the texts is measured by cosine similarity to obtain the vector retrieval result set; The full-text search is specifically: Based on keywords and user input based on word frequency and inverse document frequency, the BM25 formula is used to perform weighted calculation on keyword matching, and the full-text retrieval result set is obtained through the correlation between the query and the document.

6. The SVN cross-modal retrieval method based on a knowledge base according to claim 1, characterized in that: The step S5 is specifically as follows: Use the reciprocal ranking fusion algorithm to combine the full-text search ranking and vector search ranking of the same document to obtain a new ranking, and assign a new reciprocal ranking score to each document; The reciprocal ranking fusion algorithm uses the ranking of documents in the full-text search result set and the vector search result set to calculate the reciprocal of the ranking of each document in different search result lists, and adds these reciprocals to obtain the fusion score of each document. The documents are reordered according to the fusion score. The calculation process is as follows: ； in, It's a document. It is a document The inverse ranking score of is the number of search result lists, It is a document In the Ranking in the search result list, is a constant used to smooth the ranking.

7. The SVN cross-modal retrieval method based on a knowledge base according to claim 1 is characterized in that: The step S6 is specifically as follows: Use SVN to query the time and submitter of historical versions; Use the svn log command to display the commit history of a repository, including the author, date, revision number, commit message, and changed path of each commit.

8. A retrieval system based on the knowledge base-based SVN cross-modal retrieval method according to claim 1, characterized in that: include: The version change information collection module is used to capture change data and obtain SVN version change information when a file is submitted to the SVN repository. The change document acquisition module is used to obtain the corresponding change document according to the SVN version change information; Data change module, used to convert change documents into corresponding vector retrieval and full-text retrieval; The data retrieval module is used to obtain documents that meet the requirements through vector search and full-text search based on user input; The data rearrangement module is used to rearrange the retrieved documents to find the documents that meet the user's search criteria; The result output module is used to: input the rearranged file names into the SVN version library for retrieval, obtain file version related items, calculate the related items, and obtain the final recall result.

Images