SVN cross-modal retrieval method and system based on knowledge base

Abstract

The invention discloses an SVN cross-modal retrieval method and system based on a knowledge base, and mainly relates to the technical field of natural language processing. Comprising the following steps: capturing change data, and obtaining SVN version change information when a file is submitted to an SVN version library; obtaining a corresponding change document according to the SVN version change information; converting the changed document into corresponding vector retrieval and full-text retrieval; according to user input, obtaining documents meeting conditions through vector retrieval and full-text retrieval; the retrieved documents are rearranged, and the documents meeting the retrieval conditions of the user are found; file names obtained after rearrangement are input into an SVN version library to be retrieved, file version related items are obtained, the related items are calculated, and a final recall result is obtained. The method has the beneficial effects that the balance between the precision ratio and the recall ratio is realized, and meanwhile, the accuracy and the applicability of retrieval are remarkably improved.

Description

Technical Field

[0001] The present invention relates to the technical field of natural language processing, and specifically, to a knowledge base-based SVN cross-modal retrieval method and system. Background Art

[0002] SVN, namely Subversion, is an open-source version control system. As an open-source version control system, Subversion manages data that changes over time. This data is placed in a central repository. This repository is very much like an ordinary file server, but it remembers every change to a file. This allows the repository to be restored to an old version or to browse the change history of a file.

[0003] Subversion is a general-purpose system that can be used to manage any type of file, including program source code. However, Subversion does not provide full-text retrieval or cross-modal retrieval functions. When a user needs to find a certain document, they can only manually search through each file one by one. As the number of managed document materials gradually increases, the difficulty of this manual search for document materials also increases. In short, there is currently a lack of a method for quickly retrieving the required document materials from SVN.

[0004] Therefore, there is an urgent need for a knowledge base-based SVN cross-modal retrieval method and system to solve the above problems. Summary of the Invention

[0005] The purpose of the present invention is to provide a knowledge base-based SVN cross-modal retrieval method and system, which achieves a balance between precision and recall, and at the same time significantly improves the accuracy and applicability of retrieval.

[0006] To achieve the above object, the present invention is realized through the following technical solutions: On the one hand, the present invention provides a knowledge base-based SVN cross-modal retrieval method, including the following steps: Step S1: Capture change data. When a file is submitted to the SVN repository, obtain the SVN version change information. Step S2: Obtain the corresponding changed document according to the SVN version change information. Step S3: Convert the changed document into corresponding vector retrieval and full-text retrieval. Step S4: According to the user input, obtain the documents that meet the conditions through vector retrieval and full-text retrieval. Step S5: Rearrange the retrieved multiple documents to find the document that meets the user's retrieval conditions. Step S6: Input the file names obtained after rearrangement into the SVN repository for retrieval, obtain the file version-related items, and calculate the relevant items to obtain the final recall result.

[0007] Preferably, step S1 is specifically as follows: After the user successfully creates a version by submitting data to the SVN repository, obtain version change information through the post-commit interface provided by the SVN repository.

[0008] Preferably, step S2 is specifically as follows: According to the version number of the version change information, obtain the commit record through svnlog and identify the changed documents. The commit record includes a list of newly added documents, a list of modified documents, and a list of deleted documents.

[0009] Preferably, step S3 includes: Step S31: For audio files, use a speech recognition model to convert the audio signal into text. The audio file formats include but are not limited to: MP3 format, WAV format; Step S32: For picture files and video files, use a multimodal large model to understand the picture or video content and obtain the corresponding content text. The picture file formats include but are not limited to: PNG format, JPG format. The video file formats include but are not limited to: MP4 format, AVI format; Step S33: For text files, directly obtain the text corresponding to the file. The text file formats include but are not limited to: TXT format, DOC format; Step S34: Construct a full-text search for the text obtained by processing each type of file in steps S31 - S33, and use the text vectorization method of natural language processing to vectorize the text data.

[0010] Preferably, in step S4, the vector search is specifically as follows: Convert the user input text into a high-dimensional vector through an Embedding model, and obtain a vector search result set by measuring the semantic similarity between texts through cosine similarity, etc.; The full-text search is specifically as follows: Based on keywords and according to the user input, perform weighted calculation on keyword matching using the BM25 formula based on term frequency and inverse document frequency, and obtain a full-text search result set through the relevance between the query and the documents.

[0011] Preferably, step S5 is specifically as follows: Use the reciprocal rank fusion algorithm to combine the full-text search rank and the vector search rank of the same document to obtain a new rank, and assign a new reciprocal rank score to each document; The reciprocal ranking fusion algorithm uses the rankings of documents in the full-text retrieval result set and the vector retrieval result set, calculates the reciprocals of the rankings of each document in different retrieval result lists, and adds these reciprocals to obtain the fusion score for each document. The documents are then re-sorted based on the fusion scores. The calculation process is as follows: ; where is the document, is the document 's reciprocal ranking score, is the number of retrieval result lists, is the document in the th retrieval result list, is a constant used to smooth the ranking.

[0012] Preferably, step S6 is specifically as follows: Use SVN to query the time and submitter of historical version submissions; Use the svnlog command to display the commit history of a repository, where the commit history includes the author, date, revision number, commit message, and changed paths for each commit.

[0013] On the other hand, a retrieval system based on the above-mentioned knowledge base-based SVN transmembrane retrieval method is provided, including: A version change information collection module for: capturing change data and obtaining SVN version change information when a file is submitted to the SVN repository; A changed document acquisition module for: obtaining the corresponding changed document according to the SVN version change information; A data change module for: converting the changed document into corresponding vector retrieval and full-text retrieval; A data retrieval module for: obtaining documents that meet the conditions through vector retrieval and full-text retrieval according to user input; A data re-ranking module for: re-ranking multiple retrieved documents to find documents that meet the user's retrieval conditions; A result output module for: inputting the file name obtained after re-ranking into the SVN repository for retrieval, obtaining file version-related items, calculating the related items, and obtaining the final recall result.

[0014] Compared with the prior art, the beneficial effects of the present invention are as follows: The present invention provides a knowledge - base - based SVN cross - modal retrieval method and system, which realizes the retrieval of document materials in the SVN repository. By inputting text, voice, pictures or video files, it is possible to retrieve pure text documents, as well as audio documents such as mp3 and avi, and picture / video documents such as mp4 and jpg. Through multi - modal data such as text, images or voices, cross - modal semantic matching is achieved. The hybrid retrieval combines the exact matching of keyword - based full - text retrieval and the fuzzy matching of vector retrieval, balancing the precision rate and the recall rate, and significantly improving the accuracy and applicability of the retrieval. Finally, the results of the hybrid retrieval are recalled through rules to obtain documents that meet the user's query intent and have accurate content. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] Figure 1 is the flowchart of the method of the present invention; Figure 2 is the block diagram of the fusion sorting process of the present invention; Figure 3 is the flowchart of the knowledge - base hybrid retrieval of the present invention; Figure 4 is the flowchart of the candidate file information retrieval of the present invention; Figure 5 is the diagram showing the recall test results of the present invention; Figure 6 is the schematic diagram of the system structure of the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0016] The present invention will be further described below in conjunction with specific embodiments. It should be understood that these embodiments are only used to illustrate the present invention and not to limit the scope of the present invention. In addition, it should be understood that after reading the content taught by the present invention, those skilled in the art can make various changes or modifications to the present invention, and these equivalent forms also fall within the scope defined by this application.

[0017] In the present invention, terms such as "upper", "lower", "left", "right", "front", "rear", "vertical", "horizontal", "side", "bottom", etc. indicate the orientation or positional relationship based on the orientation or positional relationship shown in the drawings. They are only relationship words determined for the convenience of describing the structural relationship of each component or element of the present invention, and do not specifically refer to any component or element of the present invention, and should not be construed as a limitation of the present invention.

[0018] Embodiment: As Figure 1 shown, this embodiment provides a knowledge - base - based SVN cross - modal retrieval method, including the following steps: Step S1: Capture change data. When a file is submitted to the SVN repository, obtain the SVN version change information; Step S2: Obtain the corresponding change document according to the SVN version change information; Step S3: Convert the change document into corresponding vector retrieval and full-text retrieval; Step S4: According to the user input, obtain the documents that meet the conditions through vector retrieval and full-text retrieval; Step S5: Rearrange the multiple retrieved documents to find the document that meets the user's retrieval conditions; Step S6: Input the file name obtained after rearrangement into the SVN version library for retrieval, obtain the file version-related items, calculate the related items, and obtain the final recall result.

[0019] According to the above process, the implementation method is as follows: 1. The change data capture module receives the SVN version library version change notification, and obtains the set of change documents for this change according to the change version number in the change notification: The SVN version library server configures a post-commit notification program. After the user submits to the svn version library, the post-commit notification is triggered, and the change version information is sent to the change data capture module. The change version information includes the change version number for this time. After receiving the change version information, the change data capture module uses the version commit log command (svnlog -r<version number> -v<svn address>) to obtain the set of change documents for this change version.

[0020] 2. Screen out the documents changed this time according to the set of change documents: After receiving the set of change documents and the change version number, update the local files to this change version through the version update command (svnupdate -r<version number> -v<svn address>), and screen out the newly added documents and updated documents according to the set of change documents.

[0021] 3. Extract the content according to the file type of the change document: Classify the newly added documents and updated documents according to the file type, which are divided into picture type files such as png and jpg, and text type files such as txt, word, and pdf. For audio documents such as mp3 and wav, use an ASR model (such as Whisper, DeepSpeech, Kaldi, etc.) to recognize the information in the audio to obtain the text content; for picture files such as jpg and png and video files such as mp4 and avi, use a multi-modal visual recognition large model (such as Cogvlm-video, Apollo, Video-XL, etc.) to understand the content and output the picture description text corresponding to the picture / video; for text type documents such as txt and word, use a text processing tool to directly extract the content therein to obtain the text.

[0022] 4. Vectorize the text obtained from processing various types of documents and store it in the knowledge base together with the original text for convenient vector indexing and full-text indexing: In the knowledge base, use the bce-embedding model to convert the input content text into a vector representation. Taking each independent document as a unit, store the corresponding text content of the document and the vectorized data representation in the svn document knowledge base.

[0023] 5. Receive user input, convert the user input into vector features for vector search to obtain a vector search document set, and perform full-text search for keywords based on the original text of the user input to obtain a full-text search document set, as Figure 2 shown: The user input can be plain text, speech, pictures, or videos. The hybrid retrieval module receives the user input: (1) When the user inputs plain text, extract keywords through the word segmentation technology and send them to the svn document knowledge base to perform full-text retrieval to generate a full-text result set. At the same time, after the text is vectorized, perform similarity matching based on the feature vector and output the vector retrieval result set; (2) When the user uploads a picture / video, use the multimodal large model to understand the content in the image and output the corresponding content text. After the text is generated into keywords through the word segmentation technology, perform full-text retrieval to generate a full-text result set. Secondly, after the text is vectorized, perform similarity matching based on the feature vector and output the vector retrieval result set; (3) When the user inputs speech, use speech recognition technology to convert the speech into text, extract keywords through the word segmentation technology and perform full-text retrieval to generate a full-text result set. At the same time, after the text is vectorized, perform similarity matching based on the feature vector and output the vector retrieval result set.

[0024] 6. Rearrange the retrieved multiple documents to find documents that better meet the user's retrieval conditions: Input the query content entered by the user into the svn document knowledge base and use a hybrid retrieval strategy for retrieval matching. The hybrid retrieval strategy combines a traditional database and a vector database, and performs full-text retrieval and vector retrieval simultaneously; On the one hand, directly calculate the vector of the query text using the SBERT model and perform vector retrieval in the built vector index; on the other hand, perform full-text retrieval using the BM25 algorithm. Finally, use reciprocal rank fusion (RRF) for hybrid retrieval, as Figure 3As shown, the comprehensive ranking of the two retrieval results is finally obtained and the location is given. The advantage of inverse ranking fusion is that it does not utilize the relevance scores, but only relies on ranking calculations, and can well integrate two different retrieval methods. According to the relevant document content that the user wants to retrieve, the results are re-ranked by combining the full-text keywords and the candidate results of vector search to obtain the matching item with the highest comprehensive similarity, and the file name corresponding to its content is output to achieve the effects of accurate positioning and cross-modal retrieval.

[0025] 7. Evaluate the relevance of the sorted documents, and select the candidate documents 1 to n with a relevance greater than p: Set a relevance of size p. If the relevance value of the best result is greater than the set relevance p, it indicates that there are documents in the knowledge base related to the current user query. Sort by relevance and provide all documents greater than p.

[0026] 8. As Figure 4 shown, input the file names of the candidate documents 1 to n into the SVN repository for retrieval, obtain the file version-related items, and calculate the relevant items to obtain the final recall result: By using Subversion (SVN) to query the historical version submission-related information of the candidate files (svnlog candidate file names), the submission history of the candidate files will be displayed after the query, including information such as the author, date, and revision number of each submission.

[0027] Calculate the file recall score based on the returned submission information to obtain the final top-k retrieval results. The obtained metric data can be used to calculate the time, submitter, and version difference degree. The calculation process is as follows: ; When using SVN for project document management, the document versions are usually updated quickly. Among them, is the time decay coefficient of the document. A daily decay rate of 5% can quickly eliminate outdated content. For example: on the 7th day after the file is submitted (retain 70% weight), on the 30th day after the file is submitted (retain 22% weight); ; When the query condition input by the user contains the submitter information, give a weight of 1 to the candidate files that match the submitter, otherwise 0. Among them is the flag bit indicating whether the candidate file matches the submitter. Finally, use the line change ratio to calculate the version difference degree of the text document: ; Among them is the number of newly added lines, For the number of lines to be deleted, it is the total number of lines in the file. When the number of newly added lines and the number of lines to be deleted together account for a relatively low proportion of the total number of lines in the file the difference score tends to 1, indicating stronger stability or consistency of the file content. The comprehensive scoring formula is:

[0028] The value range of Score is between [0, 3]. The higher the value of Score, the newer the candidate document, the better the author match, and the stronger the stability of the file content; conversely, the older the document, the worse the author match, and the greater the change in the file content. Finally, the documents after hybrid retrieval are recalled according to the score of Score, and the top-K best content-related and user query-compliant documents are returned.

[0029] In this embodiment, text documents, audio documents, pictures, and video documents are listed respectively. When the user query input is set to "Please help me find documents related to tigers", the specific process implemented by the above method is as follows: For the newly changed text documents in the SVN repository, after uploading the ".txt" text file to the SVN, it triggers a post-commit notification, and sends the change version information to the change data capture module. The change module uses the command svn log -r $REV -v $SVN_URL to obtain the set of changed documents, where $REV is the version number and $SVN_URL is the file address. Subsequently, the command svn update -r $REV $SVN_URL is executed to update the local working copy to the specified version. The local working copy identifies the file type according to the file suffix. When it identifies a text file with the suffix ".txt", it directly uses the bce-embedding model to vectorize the text and saves it together with the text content to the SVN document knowledge base; For the newly changed audio documents in the SVN repository, after uploading the "King of the Forest.mp3" audio file to SVN, it triggers a post-commit notification, and sends the change version information to the change data capture module. The change module uses the command svn log -r $REV -v $SVN_URL to obtain the set of changed documents, where $REV is the version number and $SVN_URL is the file address. Subsequently, execute the command svn update -r $REV $SVN_URL to update the local working copy to the specified version. The local working copy identifies the file type according to the file suffix. When it identifies that the file is an audio file with the.mp3 suffix, it uses the Whisper model to identify the text content: "The tiger, the king of the forest, is covered in magnificent orange-yellow fur with black stripes, like a battle robe woven by nature. Its sharp eyes shine with a calm and majestic light, and every gaze seems to penetrate everything. Its body is strong and its steps are steady. Every step it takes is full of strength and grace, as if it can shake the silence of the entire forest." Save the obtained text content and the text content identified by the Whisper model after vectorizing the text using the bce-embedding model to the SVN document knowledge base; For newly changed images and video documents in the SVN repository, uploading the tiger.png image or video.mp4 video file to SVN triggers a post-commit notification, and sends the changed version information to the change data capture module. The change module obtains the changed document set through the command svn log -r $REV -v $SVN_URL, where $REV is the version number and $SVN_URL is the file address. Then execute the command svn update -r $REV $SVN_URL to update the local working copy to the specified version. The local working copy identifies the file type according to the file suffix, and identifies a picture or video file with a .png or mp4 suffix. The Cogvlm-video model is used to identify the picture content as: "This picture shows a majestic Siberian tiger. It is lying leisurely on the green grass. Its fur is mainly orange with typical black stripes. The tiger's face is a mixture of white and orange, with sharp eyes and erect ears. The overall image is spectacular. The background is a green vegetation, creating a natural and tranquil atmosphere for the whole picture."; The Cogvlm-video model is used to identify the video content as: "Video A large colorful tiger sculpture made of various materials such as feathers and fabric is shown in the video. The sculpture is displayed in different locations on the city streets, which are lined with buildings with Asian logos. Throughout the video, people pass by and stand near the sculpture or shops. At a certain moment, there is a burst of pink petals falling, adding to the festive atmosphere. This scene captures the fusion of traditional art and modern city life, creating a dynamic visual experience..." The obtained text content is vectorized using the bce-embedding model and saved together with the text content recognized by the Cogvlm-video model to the SVN document knowledge base; When a user makes a query, for example, when the user enters "Please help me find documents related to tigers", the knowledge base will perform full-text search and vector search simultaneously based on the user input, and apply the inverse fusion re-ranking step to select the best result that matches the user's question from the two types of query results. Figure 5 As shown in the figure, after hybrid retrieval and re-ranking, the contents of three files are greater than the set threshold p (in this case p=0.50). We get that the file corresponding to candidate file 1 (Forest King.txt) is Forest King.mp3; the file corresponding to candidate file 2 (tiger.txt) is tiger.png; and the file corresponding to candidate file 3 (video.txt) is video.mp4.

[0030] like Figure 6 As shown, this embodiment also provides a knowledge base-based SVN cross-modal retrieval system, including: The version change information collection module is used for: capturing change data and obtaining SVN version change information when a file is submitted to the SVN repository; The change document acquisition module is used for: obtaining the corresponding change document according to the SVN version change information; The data change module is used for: converting the change document into corresponding vector retrieval and full-text retrieval; The data retrieval module is used for: obtaining the documents that meet the conditions through vector retrieval and full-text retrieval according to the user input; The data rearrangement module is used for: rearranging multiple retrieved documents to find the documents that meet the user's retrieval conditions; The result output module is used for: inputting the file name obtained after rearrangement into the SVN repository for retrieval, obtaining the file version related items, calculating the related items, and obtaining the final recall result.

[0031] The above has specifically described the preferred embodiments of the present invention, but the present invention is not limited to the described embodiments. Those skilled in the art can also make various equivalent deformations or substitutions without departing from the spirit of the present invention, and these equivalent deformations or substitutions are all included in the scope defined by the claims of this application.

Claims

1. A knowledge base-based SVN cross-modal retrieval method, characterized in that It includes the following steps: Step S1: Capture change data. When a file is submitted to the SVN repository, obtain the SVN version change information; Step S2: Obtain the corresponding change document according to the SVN version change information; Step S3: Convert the change document into corresponding vector retrieval and full-text retrieval; Step S4: According to the user input, obtain the documents that meet the conditions through vector retrieval and full-text retrieval; Step S5: Rearrange the retrieved multiple documents to find the documents that meet the user's retrieval conditions; Step S6: Input the file names obtained after rearrangement into the SVN repository for retrieval, obtain the file version-related items, calculate the relevant items, and obtain the final recall result.

2. A knowledge-base-based SVN cross-modal retrieval method according to claim 1, wherein The specific content of step S1 is: Through the post-commit interface provided by the SVN repository, after the user successfully creates a version by submitting data to the SVN repository, obtain the version change information.

3. A method for cross-modal retrieval of SVN based on a knowledge base according to claim 1, characterized in that, The specific content of step S2 is: According to the version number of the version change information, obtain the commit record through svn log, and identify the change document. The commit record includes the list of newly added documents, the list of modified documents, and the list of deleted documents.

4. A knowledge base-based SVN cross-modal retrieval method according to claim 1, characterized in that, Step S3 includes: Step S31: For audio files, use a speech recognition model to convert the audio signal into text. The formats of the audio files include but are not limited to: MP3 format, WAV format; Step S32: For picture files and video files, use a multi-modal large model to understand the picture or video content, and obtain the content text corresponding to the picture or video. The formats of the picture files include but are not limited to: PNG format, JPG format. The formats of the video files include but are not limited to: MP4 format, AVI format; Step S33: For text files, directly obtain the text corresponding to the file. The formats of the text files include but are not limited to: TXT format, DOC format; Step S34: Construct a full-text retrieval for the text obtained by processing each type of file in steps S31 - S33, and use the text vectorization method of natural language processing to vectorize the text data.

5. A knowledge base-based SVN cross-modal retrieval method according to claim 1, characterized in that In step S4, the specific content of the vector retrieval is: Convert the user input text into a high-dimensional vector through the Embedding model, and obtain the vector retrieval result set by measuring the semantic similarity between texts through cosine similarity, etc.; The specific content of the full-text retrieval is: Based on keywords and according to the user input, based on the term frequency and inverse document frequency, use the BM25 formula to perform weighted calculation on keyword matching, and obtain the full-text retrieval result set through the relevance between the query and the document.

6. A method for SVN cross-modal retrieval based on a knowledge base according to claim 1, characterized in that The specific content of step S5 is: Use the reciprocal rank fusion algorithm to combine the full-text retrieval rank and the vector retrieval rank of the same document to obtain a new rank, and assign a new reciprocal rank score to each document; The reciprocal rank fusion algorithm uses the ranks of the documents in the full-text retrieval result set and the vector retrieval result set, calculates the reciprocal of the rank of each document in different retrieval result lists, and adds these reciprocals to obtain the fusion score of each document. Reorder the documents according to the fusion score. The calculation process is: ； Among them, is the document, is the document 's reverse sorting score, is the number of the retrieval result list, is the document at the th position in the retrieval result list, is a constant used to smooth the ranking.

7. A method for cross-modal retrieval of SVN based on a knowledge base according to claim 1, characterized in that The specific content of step S6 is: Use SVN to query the time and committer of historical version submissions; Use the svn log command to display the commit history of a repository, where the commit history includes the author, date, revision number, commit message, and changed paths for each commit.

8. A retrieval system based on the SVN cross-modal retrieval method based on a knowledge base as described in claim 1, characterized in that, It includes: A version change information collection module, which is used to: capture change data and obtain SVN version change information when a file is submitted to the SVN repository; A change document acquisition module, which is used to: obtain the corresponding change document according to the SVN version change information; A data change module, which is used to: convert the change document into corresponding vector retrieval and full-text retrieval; A data retrieval module, which is used to: obtain documents that meet the conditions through vector retrieval and full-text retrieval according to user input; A data rearrangement module, which is used to: rearrange multiple retrieved documents to find documents that meet the user's retrieval conditions; A result output module, which is used to: input the file name obtained after rearrangement into the SVN repository for retrieval, obtain file version-related items, calculate the related items, and obtain the final recall result.

Images