Audio and video electronic file element intelligent analysis method and system

Abstract

The application discloses an audio and video electronic file case element intelligent analysis method and system, which comprises the following steps: obtaining an audio and video electronic file target file, dividing the target file into a first segment and a second segment; splitting the first segment and the second segment into a plurality of first sub-segments and a plurality of second sub-segments; generating first text information and second text information based on the first sub-segments and the second sub-segments, and splitting the first text information and the second text information into a plurality of independent sentences; extracting first keywords from the independent sentences and performing retrieval to obtain second keywords, wherein the first keywords and the second keywords constitute a keyword set; extracting third keywords based on comparison of the keyword set; generating a basic sentence based on the third keywords, and taking a basic sentence corresponding to a template text as a theme label of the corresponding first sub-segment; and the application solves the problem that the subject label function does not exist in the prior art audio and video electronic file case, thereby reducing work efficiency.

Description

Technical Field

[0001] This invention belongs to the field of data processing technology, specifically relating to an intelligent analysis method and system for audio and video electronic case file elements. Background Technology

[0002] With the continuous development of information technology, audio and video electronic case files have become a new type of judicial document management method in the judicial field. This method can not only improve work efficiency, but also ensure judicial fairness.

[0003] Existing technologies already exist for managing audio and video electronic case files. These typically involve using intelligent management systems and technologies such as speech recognition and semantic analysis to perform structured management of the files, enabling the storage, management, retrieval, and viewing of audio and video data. For example, Chinese application document CN110135715A discloses an intelligent courtroom management method based on a cloud-based intelligent courtroom trial support system. The goal of building an intelligent courtroom is to focus on application scenarios, achieving centralized management, in-depth analysis, and utilization of trial video data. By combining existing courtroom technology equipment and related systems, and using technologies and methods such as speech recognition, semantic analysis, and big data analysis, it performs structured transformation, centralized management, and in-depth application of trial audio and video data. This generates intelligent deconstructed electronic archives that match trial subtitles, audio and video recordings, and the trial process, achieving integrated recording, transmission, storage, management, and viewing of audio and video recordings. It also enables intelligent supervision and analysis of the trial proceedings, truly providing intelligent services and management for judges. For example, Chinese application document with publication number "CN101636328A" provides an audio-visual electronic case file system. The system adopts a three-layer architecture, including a database layer, a middleware service layer, and a client layer. The database layer uses an Oracle database, installed on a database server. It uses feature extraction algorithms to extract features from audio-visual resources, create electronic case files, and store them in the feature database. The middleware service layer is installed on the database server, either on a separate server or on multiple servers, and is dedicated to handling user data requests. A database connection is established when a user requests data, and released immediately after the user's data request is completed to reduce the occupation of system resources. The client layer adopts a B / S model, where users can search, download, and view audio-visual electronic case files through a browser.

[0004] However, for audio and video electronic case files, due to the complexity and large quantity of audio and video information, simply viewing and searching audio and video electronic case files can no longer meet the current needs of relevant personnel. The various segments in audio and video electronic case files lack thematic tag functions. When relevant personnel only know that the segment they want to find exists in a certain video, but do not know where the segment will appear, they still need to browse the entire video to find it, which reduces work efficiency. Summary of the Invention

[0005] To address the aforementioned issues, this invention provides an intelligent analysis method and system for audio and video electronic case files, thereby resolving the problem that existing audio and video electronic case files lack a main tag function, which reduces work efficiency.

[0006] To achieve the aforementioned objectives, this invention proposes an intelligent analysis method for audio and video electronic case file elements, comprising:

[0007] Obtain the target file, which is an audio and video electronic file; parse the target file to divide it into at least one first segment and one second segment, wherein the first segment contains human voice and the second segment does not contain human voice;

[0008] Based on the human voice in the first segment, the first segment is divided into multiple first sub-segments, each of which includes only one timbre; based on the video in the second segment, the second segment is divided into multiple second sub-segments, each of which includes the same target object.

[0009] First text information is generated based on the human voice in the first sub-segment, and second text information is generated based on the video in the second sub-segment. The first text information and the second text information are then split into multiple independent statements.

[0010] Extract the first keyword from the independent statement, filter the first keyword, use the filtered first keyword to search in sequence to obtain multiple second keywords, and combine the first keyword with the second keywords retrieved from it to form a keyword set;

[0011] The keyword sets belonging to the same first text information are compared to extract the third keyword in the same first text information, and a topic database is established, which includes multiple template texts;

[0012] Based on the third keyword, a basic statement is generated. If the template text corresponding to the basic statement is obtained from the topic database, the basic statement serves as the topic tag for the corresponding first sub-segment.

[0013] Furthermore, extracting the third keyword from the same first text information includes the following steps:

[0014] Based on the order of occurrence of the independent statements, each keyword set is numbered from 1 to N. The first keyword and the second keyword in keyword set 1 and keyword set 2 are compared to obtain semantically similar keywords, which are used as candidate keyword 1.

[0015] Extract the first keyword and the second keyword that are semantically similar to the candidate keyword 1 from the keyword set 3, and use them as candidate keyword 2. Continue to extract the first keyword and the second keyword that are semantically similar to the candidate keyword 2 from the keyword set 4. Repeat this step until no candidate keywords can be extracted. Count the first number of the extracted keyword sets. If the first number is less than the second threshold, locate the last extracted keyword set and define it as keyword set n. Compare each of the first keyword and the second keyword in keyword set n and keyword set n+1 to obtain candidate keyword n. Repeat this step until no candidate keywords can be extracted again. Continue to obtain the first number and compare it with the second threshold.

[0016] If the first number is greater than or equal to the second threshold, then all the extracted candidate keywords are defined as the third keyword, and the second number of the remaining unextracted keyword set is counted. If the second number is less than the third threshold, then the candidate keywords are extracted from the remaining unextracted keyword set.

[0017] Furthermore, generating basic statements based on the third keyword includes the following steps:

[0018] The basic statement includes a first statement and a second statement. The third keyword is divided into a first type and a second type. The independent statement that simultaneously includes three of the third keywords is located and defined as the target statement. The third keyword included in the target statement is defined as the fourth keyword. One of the fourth keywords is selected from the target statement and defined as the main keyword 1. The remaining fourth keywords are defined as sub-keyword 1 and sub-keyword 2. The main keyword is the fourth keyword in the target statement that has a different category from the other two keywords.

[0019] In the first text information, the first co-occurrence frequency of the main word 1 and the sub-word 1, and the second co-occurrence frequency of the main word 1 and the sub-word 2 are counted. The first co-occurrence frequency and the second co-occurrence frequency are compared, and the two fourth keywords with the highest co-occurrence frequency are set as related words. The fourth keywords that are not related words are defined as discriminant words. A first statement is generated that infers the discriminant word from the related words, and a second statement is generated that infers the related words from the discriminant word.

[0020] Furthermore, retrieving the template text corresponding to the basic statement from the topic database includes the following steps:

[0021] The template text is obtained from the topic database. The template text includes at least one of the same third keywords as the first statement and the second statement. The similarity between each template text and the base statement is calculated. If the similarity between a template text and the first statement or the second statement is greater than a fourth threshold, the template text is set as the topic tag. Otherwise, the topic tag is set as a blank tag.

[0022] Furthermore, generating second text information based on the video in the second sub-segment includes the following steps:

[0023] The second sub-segment is split into multiple static images. The static images are first identified to obtain the target included in the static image and the basic type of the target. If the basic type of the target is the first type, the outline of the target is identified and the representative name of the target is obtained based on the outline.

[0024] If the basic type of the target is the second type, then the target is identified in the second way, the skeletal nodes of the target are obtained, and the pose name of the target is obtained based on the skeletal nodes;

[0025] Set the representative name or the pose name of the target as the second text information of the second sub-segment.

[0026] This invention also provides an intelligent analysis system for audio and video electronic case file elements. This system is used to implement the aforementioned intelligent analysis method for audio and video electronic case file elements. The system includes:

[0027] The acquisition module is used to acquire a target file, which is an audio and video electronic file, and to parse the target file to divide the target file into at least one first segment and a second segment, wherein the first segment contains human voice and the second segment does not contain human voice;

[0028] The splitting module, based on the human voice in the first segment, splits the first segment into multiple first sub-segments, each of which includes only one timbre; and based on the video in the second segment, splits the second segment into multiple second sub-segments, each of which includes the same target object.

[0029] The text generation module generates first text information based on the human voice in the first sub-segment, generates second text information based on the video in the second sub-segment, and splits the first text information and the second text information into multiple independent sentences;

[0030] The extraction module is used to extract a first keyword from the independent statement, filter the first keyword, use the filtered first keyword to search in sequence to obtain multiple second keywords, combine the first keyword with the second keywords retrieved therefrom to form a keyword set, compare the keyword set belonging to the same first text information to extract a third keyword in the same first text information, and establish a topic database, the topic database including multiple template texts;

[0031] The generation module generates a basic statement based on the third keyword. If the template text corresponding to the basic statement is obtained from the topic database, the basic statement is used as the topic tag for the corresponding first sub-segment.

[0032] Compared with the prior art, the beneficial effects of the present invention are at least as follows:

[0033] This invention, through the aforementioned technical solution, achieves mode conversion of audio and video electronic case files. It divides the video into first and second segments based on the presence or absence of sound, and converts complex and lengthy audio and video electronic case files into text information. By extracting keywords from each independent statement in the text information and analyzing the co-occurrence probability between keywords, the basic statement corresponding to each independent statement in the text information is determined. The basic statement is then compared with template text in the established topic database to obtain the topic tags for the corresponding segments. This solves the problem in existing technologies where the complexity and length of audio and video electronic case files make it difficult to quickly locate the searched topic information. Furthermore, this solution divides the video into different segments based on human voices, facilitating the classification and viewing of audio and video electronic case files by relevant personnel, improving work efficiency, and saving significant labor costs in the review process. Attached Figure Description

[0034] Figure 1 This is a flowchart illustrating the steps of an intelligent analysis method for audio and video electronic case file elements according to the present invention.

[0035] Figure 2This is a schematic diagram of the structure of an intelligent analysis system for audio and video electronic case file elements according to the present invention. Detailed Implementation

[0036] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the invention.

[0037] It is understood that the terms "first," "second," etc., used in this application may be used herein to describe various elements, but unless otherwise specified, these elements are not limited by these terms. These terms are used only to distinguish one element from another. For example, without departing from the scope of this application, a first script may be referred to as a second script, and similarly, a second script may be referred to as a first script.

[0038] like Figure 1 As shown, an intelligent analysis method for audio and video electronic case file elements is proposed.

[0039] Step S1: Obtain the target file, which is an audio and video electronic file. Parse the target file to divide it into at least one first segment and one second segment. The first segment contains human voices, while the second segment does not contain human voices.

[0040] Specifically, the target file is the electronic audio and video case file. This file contains all electronic transcripts, audio-visual materials, and electronic evidence related to the court proceedings. A speech recognition model, such as an RNN neural network, can be used to parse the target file. Based on whether there is human voice in the video, the target file is divided into a first segment and a second segment. The first segment contains human voice, meaning it contains both images and corresponding human voice. The second segment does not contain human voice, meaning it only contains images and video information. For example, in the same video, the first segment 1 is 0-5 seconds long and does not contain human voice.

[0041] Step S2: Based on the human voice in the first segment, the first segment is divided into multiple first sub-segments, each of which includes only one timbre. Based on the video in the second segment, the second segment is divided into multiple second sub-segments, each of which includes the same target object.

[0042] After obtaining the first segment, the voice characteristics of different speakers are identified using the timbre detection function of speech recognition. Based on the timbre detection results, the first segment is divided into multiple first sub-segments, each containing only one timbre, meaning only one person's voice is present in the first sub-segment. After obtaining the second segment, based on the image information of the second segment, it is divided into multiple second sub-segments. There are many methods for splitting the second segment, such as using artificial intelligence techniques like face recognition, contour detection, and skeletal recognition. These are existing technologies and will not be elaborated on further. Each second sub-segment includes the same target object; for example, the video length of the second sub-segment is 10 seconds, and either person A or object B appears in all of them.

[0043] Step S3: Generate first text information based on the human voice in the first sub-segment, generate second text information based on the video in the second sub-segment, and split the first text information and the second text information into multiple independent sentences.

[0044] Based on the human voice in the first sub-segment, the first text information is generated from the human voice in the first sub-segment. The first text information is the human voice in the first sub-segment converted into text information through speech recognition. The first text information is the words spoken by the person in the video in the first sub-segment. After obtaining the video information of the second sub-segment, the second text information is generated from the video in the second sub-segment. The second text information is the representative name and posture name of the target object in the video. For example, the second text information is "Person A is walking". Then, the obtained first text information and second text information are split into multiple independent sentences by using sentence ending symbols such as "." or "!".

[0045] Step S4: Extract the first keyword from the independent statement, filter the first keyword, use the filtered first keyword to search in turn to obtain multiple second keywords, and combine the first keyword with the second keyword retrieved from it to form a keyword set.

[0046] After obtaining multiple independent sentences, a dictionary is built, which includes multiple basic keywords. Basic keywords are extracted from each independent sentence, and the basic keywords that appear more than 5 times are used as the first keyword.

[0047] The first keyword is searched online to obtain multiple second keywords. The second keywords are words that are related to the first keyword and appear more frequently than a second threshold. The first and second keywords are combined into a keyword set. For example, if the first keyword is "plaintiff", "property dispute", and "legal rights", then the first keyword "plaintiff", "property dispute", and "legal rights" are searched online. For example, if the search for "plaintiff" yields 30 search results, and the second keywords "provide evidence" and "legal consequences" appear more than 10 times, then the first keyword "plaintiff", "property dispute", and "legal rights" and the second keyword "provide evidence" and "legal consequences" are combined into a keyword set, namely "plaintiff", "property dispute", "legal rights", "provide evidence", and "legal consequences".

[0048] Step S5: Compare the keyword sets belonging to the same first text information to extract the third keywords in the same first text information, and establish a topic database, which includes multiple template texts.

[0049] After obtaining the keyword set, multiple keyword sets belonging to the first text information are compared to extract the third keyword. The third keyword is a word with similar semantics extracted after comparing multiple keyword sets. For example, if there are keyword sets 1 and 2 belonging to the first text information, and keyword set 1 contains "plaintiff", "property dispute", "legal rights", "defendant", and "legal consequences", while keyword set 2 contains "plaintiff", "defendant", "joint property", "litigation", and "provide evidence", then the third keyword is "plaintiff", "defendant", and "property dispute". After obtaining the third keyword, a topic database is established. The topic database includes multiple template texts, which are pre-entered texts and will be described in detail later.

[0050] Step S6: Generate a basic statement based on the third keyword. If a template text corresponding to the basic statement is obtained from the topic database, the basic statement is used as the topic tag for the corresponding first sub-segment.

[0051] After obtaining the third keyword, a plurality of basic sentences are generated based on the third keyword, wherein the basic sentences include a first sentence and a second sentence, the first sentence and the second sentence represent different semantic logics, and the generation method will be described in detail later; template texts are extracted from the topic database, the template texts and the basic sentences comprise the same third keyword, for example, the template text is "the plaintiff has a property dispute with the defendant", then the similarity between each template text and the first sentence and the second sentence in the basic sentences is calculated, for example, the similarity between the first sentence and the template text is 30%, and the similarity between the second sentence and the template text is 80%, then it is determined that there is a template text extremely similar to the basic sentence, therefore, the basic sentence corresponding to the template text is set as a topic tag.

[0052] In this embodiment, extracting the third keyword in the same first text information comprises the following steps:

[0053] Based on the occurrence order of independent sentences, each keyword set is numbered 1 to N, and each first keyword and second keyword in keyword set 1 and keyword set 2 are compared to obtain semantically similar keywords as alternative keyword 1;

[0054] For example, if the first text information comprises 5 sentences, the independent sentences are numbered 1 to 5, and the keyword set 1 is generated corresponding to independent sentence 1. First, keyword set 1 and keyword set 2 are compared, and semantically similar first keywords and second keywords are extracted therefrom, for example, the first keyword is "beautiful" and the second keyword is "wonderful", and alternative keyword 1 is "beautiful" and "wonderful"; semantic approximation can be obtained by the included angle between word vectors, and it should be understood by those skilled in the art that each word has its own word vector, word vectors are established through a dictionary, the dictionary of word vectors can be obtained from the Internet, and the smaller the included angle between the word vectors of two words, the closer the semantics between the two words are. Therefore, words with a word vector included angle less than 15 degrees are identified as approximate words.

[0055] Continue to extract first keywords and second keywords semantically similar to alternative keyword 1 from keyword set 3 to form alternative keyword 2, repeat this step until no alternative keyword can be extracted, count the number of all keyword sets that have been extracted, and define this number as a first quantity, for example, when alternative keyword 1 is compared with keyword set 3, no word similar to alternative keyword 1 can be obtained in keyword set 3, therefore, the first quantity is 2.

[0056] If the first quantity is less than a third threshold, the third threshold is set according to the total number of keyword sets, if there are 50 keyword sets, the third threshold is 4, if the first quantity is less than 4, it indicates that this part of keywords appears less frequently, so they are discarded. For example, the above alternative keyword 2 is discarded, then alternative keyword 1 is extracted from keyword set 3 and keyword set 4, and subsequent steps are continued to be executed.

[0057] If the first number is greater than or equal to the third threshold, then all candidate keywords in the extracted keyword set are defined as the third keyword. The second number of the remaining unextracted keyword set is counted. If the second number is less than the third threshold, then candidate keywords are extracted from the remaining unextracted keyword set.

[0058] For example, if the first text information has 100 keywords, keyword set 1 is compared with keyword set 2 to obtain candidate keyword 1. Similarly, the comparison is carried out according to the above steps until keyword set 31 is compared with candidate keyword 29. If no semantically similar words can be extracted as candidate keywords, the first number of extracted keyword sets is counted as 30. If the first number 30 is greater than the third threshold 20, then the currently extracted candidate keyword is retained as the third keyword. The second number of the remaining unextracted keyword sets is counted as 70. If the second number 70 is less than the third threshold 100, then candidate keywords are extracted from the remaining unextracted keyword sets. The above steps are repeated until all keyword sets have been processed.

[0059] Generating basic sentences based on third keywords includes the following steps:

[0060] After obtaining the third keyword, basic statements are generated from it. These basic statements include the first and second statements. First, the third keyword is divided into two types. The first type consists of words related to people in the audio / video electronic case files, such as "plaintiff," "defendant," "judge," "prosecutor," "witness," and "defense attorney." The second type consists of words related to the events caused by the people in the audio / video electronic case files and the actions taken in response to these events, such as "property dispute," "legal consequences," "economic loss," "providing evidence," "illegal behavior," "legitimate rights," and "litigation."

[0061] The independent statement, such as "The plaintiff believes that the defendant infringed upon his / her civil rights and caused economic losses, and therefore filed a lawsuit with the court," contains three third keywords: "plaintiff," "defendant," and "economic losses." These three words are among the previously obtained candidate words, so this independent statement is defined as the target statement. From the target statement, the three third keywords "plaintiff," "defendant," and "economic losses" are extracted and defined as fourth keywords. One of the fourth keywords from the target statement is selected as the subject 1, and the remaining fourth keywords are defined as sub-keywords 1 and 2. Among the three fourth keywords, the subject is defined as the fourth keyword with a different category from the other two keywords. Since "plaintiff" and "defendant" both belong to the first category, and "economic losses" belongs to the second category, "economic losses" is designated as subject 1, and "plaintiff" and "defendant" are designated as sub-keywords 1 and 2, respectively.

[0062] In the first text information, calculate the first co-occurrence frequency P(subject 1, subject 1) between the main term "economic loss" and the sub-term "plaintiff," and the second co-occurrence frequency P(subject 1, sub-term 2) between the main term "economic loss" and the sub-term "defendant." Compare the first and second co-occurrence frequencies. Co-occurrence frequency is the frequency of common occurrences. For example, if the main term 1 and sub-term 1 co-occur in the same independent sentence 10 times, the first co-occurrence frequency is 10, and the second co-occurrence frequency is similarly calculated. If P(subject 1, sub-term 1) is greater than P(…), then… If we have "Main word 1, Subword 2", it means that the probability of both "Main word 1" and "Subword 1" appearing simultaneously is relatively high. We set "Economic loss" (Main word 1) and "Plaintiff" (Subword 1) as related words, and "Defendant" (Subword 2) as the discriminant word. This generates the first logical relation from the related words to the discriminant word: (Main word 1 & Subword 1) → Subword 2, for example, ("Economic loss" & "Plaintiff" → "Defendant"). Then, we generate the first statement based on this first logical relation. During generation, a blank template text is pre-set; for example, the template for the first logical relation is "Due to **". The presence of ** leads to **, where ** represents a gap. During generation, the subject and sub-sub ... If the probability is relatively high, then the main term 1 "economic loss" and the sub-term 2 "defendant" are set as related terms, and the sub-term 1 "plaintiff" is set as the discriminant term. The first statement inferred from the related terms is generated: (main term 1 & sub-term 2) → sub-term 1, for example ("economic loss" & "defendant" → "plaintiff"). Alternatively, the second statement inferred from the discriminant term can be generated: sub-term 1 → (main term 1 & sub-term 2), for example ("plaintiff" → "economic loss" & "defendant"). The generation method of the above statements is the same, and will not be described in detail here.

[0063] Furthermore, retrieving the template text corresponding to the basic statement from the topic database, the basic statement as the topic tag for the corresponding first sub-segment includes the following steps:

[0064] Multiple template texts are pre-set in the topic database. Each template text includes multiple representative keywords. The template texts and the first and second statements include the same third keyword. The topic database contains the template text "Because the plaintiff believes that the defendant infringed upon his civil rights and caused economic losses, he / she requests compensation for mental distress." If the generated first statement is "Because the plaintiff believes that the defendant committed an infringement, he / she requests compensation for economic losses," then the template texts in the topic database corresponding to the third keyword are found based on the representative keywords. The template texts are compared with the first and second statements in the basic statements, and the similarity between each template text and the basic statement is calculated.

[0065] This embodiment calculates the similarity δ based on the following steps, the specific formula being: Where n is the number of identical characters between the first or second statement and the template text, and M is the total number of characters in the first or second statement. For example, if the first statement has 20 characters, and 15 of them are present in the template text, then the similarity is 15 / 20 * 100% = 75%. If the similarity is greater than 80%, then the first or second statement is considered correctly generated, and the text with the highest similarity is used as the topic tag for the corresponding segment. For example, if the first statement has a 30% similarity to the template text, and the second statement has an 80% similarity, then the logic of the second statement is considered correct, and therefore the second statement is used as the topic tag.

[0066] Furthermore, the generation of second text information from the video in the second sub-segment in step S3 includes the following steps:

[0067] The second sub-segment is split into multiple static images at 1-second intervals. Then, the static images are identified using a CNN neural network to determine whether the target is an object (first type) or a person. If it is an object, the object's outline is identified and compared with the outlines already stored in the database to obtain the object's name. For example, if the first outline in the database is close to the target object's outline and the first outline's label is "stick," then "stick" is set as the representative name of the object.

[0068] If the basic type of the target is the second type, which is defined as a target object with life characteristics, such as a person or an animal, then the target object is identified by the skeleton detection algorithm to obtain the target's skeleton nodes. Based on the skeleton nodes, the target's posture name is obtained, such as: giving a speech, pronouncing a judgment, arguing, etc. The posture name helps to express the state of the target person.

[0069] Set the target's representative name or pose name as the second text information of the second sub-fragment.

[0070] If the target is a person, the skeletal key points of the target are extracted based on skeletal detection. The positions of the skeletal key points are also compared with existing images in the database to determine the target's posture. Then, the face recognition algorithm is combined to obtain the specific identity of the target. Therefore, the database also pre-stores the face data of the target to be detected, as well as the name corresponding to the face. In this way, the face recognition results and posture are combined to generate text such as "Person A is walking".

[0071] like Figure 2 As shown, the present invention also provides an intelligent analysis system for audio and video electronic case file elements:

[0072] The acquisition module is used to acquire the target file, which is an audio and video electronic file. The target file is parsed to divide it into at least one first segment and one second segment, wherein the first segment contains human voice and the second segment does not contain human voice.

[0073] The splitting module, based on the human voice in the first segment, splits the first segment into multiple first sub-segments, each of which includes only one timbre; based on the video in the second segment, it splits the second segment into multiple second sub-segments, each of which includes the same target object.

[0074] The text generation module generates first text information based on the human voice in the first sub-segment and second text information based on the video in the second sub-segment, and splits the first and second text information into multiple independent sentences;

[0075] The extraction module is used to extract the first keyword from independent sentences, filter the first keyword, and use the filtered first keyword to search in turn to obtain multiple second keywords. The first keyword is combined with the second keywords retrieved from it to form a keyword set. The keyword set belonging to the same first text information is compared to extract the third keyword in the same first text information. A topic database is established, which includes multiple template texts.

[0076] The generation module generates a basic statement based on the third keyword. If a template text corresponding to the basic statement is obtained from the topic database, the basic statement is used as the topic tag for the corresponding first sub-segment.

[0077] It is worth noting that this invention obtains the target file of the audio and video electronic case file, parses the video in the target file to divide the target file into at least one first segment and a second segment, wherein the first segment contains audio and the second segment does not; based on the audio in the first segment, the first segment is split into multiple first sub-segments, each first sub-segment including only one timbre; based on the image in the second segment, the second segment is split into multiple second sub-segments, each second sub-segment including the same target object; first text information is generated based on the audio in the first sub-segments, and second text information is generated based on the video in the second sub-segments; the first text information and the second text information are split into multiple independent sentences; a first keyword is extracted from the independent sentences, the first keyword is filtered, and the filtered first keyword is used to search sequentially to obtain multiple second keywords, and the first keyword and the second keyword are combined into a keyword set; the keyword sets belonging to the same first text information are compared to extract a third keyword in the same first text information, and a topic database is established, the topic database including multiple template texts, each template text including topic tags; based on the third keyword, a basic sentence is generated from the third keyword, and the template text corresponding to the basic sentence is obtained from the topic database, the topic tags of the template text are used as the topic tags of the corresponding first sub-segment.

[0078] This invention, through the aforementioned technical solution, achieves mode conversion of audio and video electronic case files. It divides the video into first and second segments based on the presence or absence of sound, and converts complex and lengthy audio and video electronic case files into text information. By extracting keywords from each independent statement in the text information and analyzing the co-occurrence probability between keywords, the basic statement corresponding to each independent statement in the text information is determined. The basic statement is then compared with template text in the established topic database to obtain the topic tags for the corresponding segments. This solves the problem in existing technologies where the complexity and length of audio and video electronic case files make it difficult to quickly locate the searched topic information. Furthermore, this solution divides the video into different segments based on human voices, facilitating the classification and viewing of audio and video electronic case files by relevant personnel, improving work efficiency, and saving significant labor costs in the review process.

[0079] It should be understood that although the steps in the flowcharts of the various embodiments of the present invention are shown sequentially according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the various embodiments may include multiple sub-steps or multiple stages. These sub-steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these sub-steps or stages is not necessarily sequential, but can be performed alternately or in turn with other steps or at least a portion of the sub-steps or stages of other steps.

[0080] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The program can be stored in a non-volatile computer-readable storage medium, and when executed, it can include the processes of the embodiments of the above methods. Any references to memory, storage, databases, or other media used in the embodiments provided in this application can include non-volatile and / or volatile memory. Non-volatile memory can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), or flash memory. Volatile memory can include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in various forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), dual data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link DRAM (SLDRAM), RAMbus direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and RAMbus dynamic RAM (RDRAM), etc.

[0081] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0082] The above embodiments merely illustrate several implementation methods of the present invention, and their descriptions are relatively specific and detailed, but they should not be construed as limiting the scope of the present invention. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of the present invention, and these all fall within the protection scope of the present invention. Therefore, the protection scope of this patent should be determined by the appended claims.

[0083] The above are merely preferred embodiments of the present invention and are not intended to limit the present invention. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A method for intelligent analysis of audio and video electronic case file elements, characterized in that, include: Obtain the target file, which is an audio and video electronic file; parse the target file to divide it into at least one first segment and one second segment, wherein the first segment contains human voice and the second segment does not contain human voice; Based on the human voice in the first segment, the first segment is divided into multiple first sub-segments, each of which includes only one timbre; based on the video in the second segment, the second segment is divided into multiple second sub-segments, each of which includes the same target object. First text information is generated based on the human voice in the first sub-segment, and second text information is generated based on the video in the second sub-segment. The first text information and the second text information are then split into multiple independent statements. Extract the first keyword from the independent statement, filter the first keyword, use the filtered first keyword to search in sequence to obtain multiple second keywords, and combine the first keyword with the second keywords retrieved from it to form a keyword set; The keyword sets belonging to the same first text information are compared to extract the third keyword in the same first text information, and a topic database is established, which includes multiple template texts; Based on the third keyword, a basic statement is generated. If the template text corresponding to the basic statement is obtained from the topic database, the basic statement is used as the topic tag corresponding to the first sub-segment. The process of generating a basic statement based on the third keyword includes the following steps: The basic statement includes a first statement and a second statement. The third keyword is divided into a first type and a second type. The independent statement that simultaneously includes three of the third keywords is located and defined as the target statement. The third keyword included in the target statement is defined as the fourth keyword. One of the fourth keywords is selected from the target statement and defined as the main keyword 1. The remaining fourth keywords are defined as sub-keyword 1 and sub-keyword 2. The main keyword 1 is the fourth keyword in the target statement that has a different category from the other two keywords. In the first text information, the first co-occurrence frequency of the main word 1 and the sub-word 1, and the second co-occurrence frequency of the main word 1 and the sub-word 2 are counted. The first co-occurrence frequency and the second co-occurrence frequency are compared, and the two fourth keywords with the highest co-occurrence frequency are set as related words. The fourth keywords that are not related words are defined as discriminant words. A first statement is generated that infers the discriminant word from the related words, and a second statement is generated that infers the related words from the discriminant word.

2. The intelligent analysis method for audio and video electronic case file elements according to claim 1, characterized in that, Extracting the third keyword from the same first text information includes the following steps: Based on the order of occurrence of the independent statements, each keyword set is numbered from 1 to N. The first keyword and the second keyword in keyword set 1 and keyword set 2 are compared to obtain semantically similar keywords, which are used as candidate keyword 1. Extract the first keyword and the second keyword that are semantically similar to the candidate keyword 1 from the keyword set 3, and use them as candidate keyword 2. Continue to extract the first keyword and the second keyword that are semantically similar to the candidate keyword 2 from the keyword set 4. Repeat this step until no candidate keywords can be extracted. Count the first number of the extracted keyword sets. If the first number is less than the second threshold, locate the last extracted keyword set and define it as keyword set n. Compare each of the first keyword and the second keyword in keyword set n and keyword set n+1 to obtain candidate keyword n. Repeat this step until no candidate keywords can be extracted again. Continue to obtain the first number and compare it with the second threshold. If the first number is greater than or equal to the second threshold, then all the extracted candidate keywords are defined as the third keyword, and the second number of the remaining unextracted keyword set is counted. If the second number is less than the third threshold, then the candidate keywords are extracted from the remaining unextracted keyword set.

3. The intelligent analysis method for audio and video electronic case file elements according to claim 1, characterized in that, Retrieving the template text corresponding to the basic statement from the topic database includes the following steps: The template text is obtained from the topic database. The template text includes at least one of the same third keywords as the first statement and the second statement. The similarity between each template text and the base statement is calculated. If the similarity between a template text and the first statement or the second statement is greater than a fourth threshold, the template text is set as the topic tag. Otherwise, the topic tag is set as a blank tag.

4. The intelligent analysis method for audio and video electronic case file elements according to claim 1, characterized in that, Generating second text information based on the video in the second sub-segment includes the following steps: The second sub-segment is split into multiple static images. The static images are first identified to obtain the target included in the static image and the basic type of the target. If the basic type of the target is the first type, the outline of the target is identified and the representative name of the target is obtained based on the outline. If the basic type of the target is the second type, then the target is identified in the second way, the skeletal nodes of the target are obtained, and the pose name of the target is obtained based on the skeletal nodes; Set the representative name or the pose name of the target as the second text information of the second sub-segment.

5. An intelligent analysis system for audio and video electronic case file elements, used to implement the intelligent analysis method for audio and video electronic case file elements as described in any one of claims 1-4, characterized in that, include: The acquisition module is used to acquire a target file, which is an audio and video electronic file, and to parse the target file to divide the target file into at least one first segment and a second segment, wherein the first segment contains human voice and the second segment does not contain human voice; The splitting module, based on the human voice in the first segment, splits the first segment into multiple first sub-segments, each of which includes only one timbre; and based on the video in the second segment, splits the second segment into multiple second sub-segments, each of which includes the same target object. The text generation module generates first text information based on the human voice in the first sub-segment, generates second text information based on the video in the second sub-segment, and splits the first text information and the second text information into multiple independent sentences; The extraction module is used to extract a first keyword from the independent statement, filter the first keyword, use the filtered first keyword to search in sequence to obtain multiple second keywords, combine the first keyword with the second keywords retrieved therefrom to form a keyword set, compare the keyword set belonging to the same first text information to extract a third keyword in the same first text information, and establish a topic database, the topic database including multiple template texts; The generation module generates basic statements based on the third keyword. If the template text corresponding to the basic statement is obtained from the topic database, the basic statement is used as the topic tag for the corresponding first sub-segment. The generation of basic statements based on the third keyword includes the following steps: the basic statement includes a first statement and a second statement; the third keyword is divided into a first type and a second type; the independent statement that simultaneously includes three of the third keywords is located and defined as the target statement; the third keyword included in the target statement is defined as the fourth keyword; one of the fourth keywords from the target statement is selected as the main keyword 1, and the remaining fourth keywords are defined as sub-word 1 and sub-word 2, wherein the main keyword 1 is the fourth keyword in the target statement that has a different category from the other two keywords; the first co-occurrence frequency of the main keyword 1 and sub-word 1 and the second co-occurrence frequency of the main keyword 1 and sub-word 2 are counted in the first text information; the first co-occurrence frequency and the second co-occurrence frequency are compared, and the two fourth keywords with the highest co-occurrence frequency are set as related words; the fourth keywords that are not related words are defined as discriminant words; a first statement inferring the discriminant word from the related words and a second statement inferring the related words from the discriminant words are generated.

Images