A sound image file intelligent classification identification and archiving method and system
By constructing a business rules system and a deep learning model, the problems of classification, identification, and cataloging in audio-visual archives management have been solved, achieving efficient and accurate archives management and ensuring the standardization and consistency of archives.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- STATE GRID SHANGHAI MUNICIPAL ELECTRIC POWER CO
- Filing Date
- 2026-02-26
- Publication Date
- 2026-05-29
AI Technical Summary
Traditional audiovisual archive management methods are difficult to achieve efficient, standardized, and intelligent management. In particular, they suffer from pain points such as low information retrieval efficiency, strong subjectivity, inconsistent archiving standards, and difficulty in subsequent retrieval and utilization in areas such as detailed classification of archive content, automatic determination of retention period, rapid extraction of personal identity information, and cross-departmental archiving collaboration.
We construct a business rules system, perform high-definition data preprocessing of audio-visual archives and build a face recognition database, train an intelligent classification and identification model for audio-visual archive files, and build a neural network model using a deep learning framework to achieve automatic classification, identification, and cataloging of files.
It enables accurate classification, identification, and cataloging of audio-visual archives. Through a quantitative identification index system and intelligent model analysis, it ensures the consistency and accuracy of identification results, avoiding the chaos in archive management caused by differences in human judgment.
Smart Images
Figure CN122111956A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of data management, and in particular to a method and system for intelligent classification, identification and archiving of audio-visual archives. Background Technology
[0002] As enterprises expand and their information management capabilities improve, audiovisual archives, as important information assets, are widely used in various fields such as business activity recording, corporate culture building, employee training, and project management. In recent years, with the explosive growth of high-definition video, audio, and multimedia data, corporate audiovisual archives have become characterized by large quantities, diverse types, complex content, and high management difficulty. Traditional archive management methods relying on manual identification and classification are no longer sufficient to meet the actual needs of enterprises for efficient, standardized, and intelligent management of audiovisual archives.
[0003] Existing methods for managing audiovisual archives mainly focus on basic file storage, manual coding, and cataloging, lacking intelligent data analysis and automated processing capabilities. This prevents in-depth understanding, accurate classification, and dynamic identification of the archive content. In particular, traditional methods suffer from numerous drawbacks, including low information retrieval efficiency, high subjectivity, inconsistent archiving standards, and difficulties in subsequent retrieval and utilization, particularly in areas involving detailed classification of archive content, automatic determination of retention periods, rapid extraction of personal identification information, and cross-departmental archiving collaboration. Summary of the Invention
[0004] To address the aforementioned problems, the present invention aims to provide an intelligent classification, identification, and archiving method and system for audio-visual archives, which can accurately classify, identify, and catalog audio-visual archives.
[0005] To achieve the above objectives, the present invention adopts the following technical solution: A method for intelligent classification, identification, and archiving of audio-visual archives includes the following steps: S1: Construct a business rule system based on the company's business rules for document classification, retention period determination, document title, archiving department, and document cataloging of audio-visual archives; S2: Acquire the original high-definition data of the audio-visual archives, preprocess it, and build a face recognition database; S3: Construct an intelligent classification and identification model for audio-visual archives; S4: Based on the preprocessed audio-visual data and the face recognition database, train the intelligent classification and identification model for audio-visual archives to obtain the optimized intelligent classification and identification model for audio-visual archives. S5: Based on the optimized intelligent classification and identification model for audio-visual archives, each collected document in the pre-collection stage is intelligently allocated, and verified based on the business rule system, it assists archivists in carrying out auxiliary classification, identification, organization, cataloging and archiving work for audio-visual archives.
[0006] Furthermore, the raw high-definition data of the audio-visual archives is acquired and preprocessed, as follows: First, a high-definition bitrate to low-bitrate conversion service is performed to obtain the original high-definition data of the audio-visual archives. Then, through a video encoding conversion algorithm, the high-definition bitrate is converted into a low-bitrate format suitable for storage and transmission. After the conversion is completed, the low-bitrate audio-visual files are saved to the specified storage location. Next, the low bitrate audio-visual file is sliced according to playback frames. One frame of image is extracted according to the set slicing interval, and the image data of each frame is obtained through video decoding technology and saved as an independent image file.
[0007] Furthermore, a facial recognition database is constructed, as detailed below: A face recognition database is constructed using image data obtained from image slices. Face detection is performed on each slice image using a Haar cascade detector to identify face regions in the image. For detected faces, the LBPH feature extraction method is used to extract their feature vectors. The extracted feature vectors are compared with the existing face database. If it is a new face, its feature vector and related information are added to the face recognition database. Simultaneously, speech recognition and text transcription are performed. The audio portion of the audio-visual archive is processed, and the speech is converted into text through a speech recognition engine. First, the audio is denoised, and then the audio is divided into several segments. Speech recognition is performed on each segment, and the recognition results are integrated and corrected to finally obtain the complete text transcription content.
[0008] Furthermore, based on the business rule system, an intelligent classification and identification model for audio-visual archives is constructed, specifically as follows: An intelligent classification and identification model is constructed based on the business rule system and the document titles, keywords, document summaries, and archiving department information of the audio-visual archives; a deep learning framework is used to build a neural network model; the model includes an input layer, an intermediate layer, and an output layer; the input layer receives preprocessed audio-visual archive text information and image feature information; the intermediate layer performs feature extraction and transformation through multiple fully connected layers or convolutional layers; and the output layer outputs the prediction result using the Softmax activation function according to the classification and identification objectives.
[0009] Furthermore, based on the preprocessed audio-visual data and face recognition database, an intelligent classification and identification model for audio-visual archives is trained to obtain an optimized intelligent classification and identification model for audio-visual archives. Specifically, the preprocessed audio-visual data and face recognition database are divided into training, validation, and test sets. During training, data from the training set is input into the model, and the model parameters are adjusted using the backpropagation algorithm to minimize the loss function between the model's predictions and the true labels. The cross-entropy loss function is used to measure the difference between the predictions and the true classification labels, and the model parameters are updated using optimization algorithms such as stochastic gradient descent. After each round of training, the model is evaluated using the validation set, and the changes in the model's accuracy and recall are observed. After several rounds of training and adjustment, the model is finally evaluated using the test set to ensure that the model's generalization ability and accuracy meet the requirements.
[0010] Furthermore, based on the optimized intelligent classification and identification model for audio-visual archives, each collected document in the pre-collection stage is intelligently assigned a document classification number, retention period, data verification, and cataloging and archiving tasks, as detailed below: When audio-visual archives enter the pre-collection stage, the collected documents undergo preliminary processing, extracting document titles, keywords, and abstract text information. Simultaneously, image and voice analysis is performed on the audio-visual content to obtain facial recognition results and text transcription. This information is then input into an optimized intelligent classification and identification model for audio-visual archives. The model outputs identification results such as document classification number and retention period. Based on the identification results, data verification rules are automatically assigned to the documents according to a business rule system to verify their completeness and accuracy. If the verification passes, the documents are cataloged and archived according to the identification results, stored in the designated location of the corresponding archiving department, and relevant information is recorded in the archive management system.
[0011] An intelligent classification, identification, and archiving system for audio-visual archives includes a processor, a memory, and a computer program stored in the memory. When the processor executes the computer program, it specifically performs the steps in the integrated real-time coordination and risk scheduling method for main and microgrids as described above.
[0012] The present invention has the following beneficial effects: This invention can accurately classify, identify, and catalog audio-visual archives. In terms of retention period identification, the consistency and accuracy of the identification results are guaranteed through a quantitative identification index system and intelligent model analysis, avoiding the chaos in archive management caused by differences in human judgment. Attached Figure Description
[0013] Figure 1 This is a flowchart of the method of the present invention. Detailed Implementation
[0014] The present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments: refer to Figure 1 In this embodiment, an intelligent classification, identification, and archiving method for audio-visual archives is provided, characterized by the following steps: S1: Construct a business rule system based on the company's business rules for document classification, retention period determination, document title, archiving department, and document cataloging of audio-visual archives; S2: Acquire the original high-definition data of the audio-visual archives, preprocess it, and build a face recognition database; S3: Construct an intelligent classification and identification model for audio-visual archives; S4: Based on the preprocessed audio-visual data and the face recognition database, train the intelligent classification and identification model for audio-visual archives to obtain the optimized intelligent classification and identification model for audio-visual archives. S5: Based on the optimized intelligent classification and identification model for audio-visual archives, each collected document in the pre-collection stage is intelligently allocated, and verified based on the business rule system, it assists archivists in carrying out auxiliary classification, identification, organization, cataloging and archiving work for audio-visual archives.
[0015] In this embodiment, the business rule system is built around five core components: "classification, identification, title, archiving, and cataloging," as detailed below: Document classification rules: Based on the nature of the content, it is divided into 3 major categories and 12 subcategories, such as business activities (including product launches, customer negotiations, and project inspections), corporate culture (including anniversary celebrations, employee commendations, and brand promotion), and employee training (including new employee onboarding, skills enhancement, and safety training). The criteria for judgment are clearly defined. For example, audio-visual files containing the keywords "new product launch + product model" are classified as "business activities - product launches". Title writing rules: Use the format "[Time + Activity Type + Core Subject] Audio-visual Archive", such as "20250618 Company Anniversary Celebration Executive Speech Video Archive" and "20250701 New Employee Onboarding Safety Training Audio Archive"; Filing Department Rules: Files are matched according to business affiliation. For example, sales-related files belong to the sales department, training-related files to the human resources department, and brand promotion files to the marketing department. For document classification, audio-visual archives are meticulously categorized based on their content nature, such as business activities, corporate culture, and employee training, with clear classification standards established. For retention period assessment, a quantitative assessment indicator system is established, comprehensively considering factors such as the importance of the archives, their timeliness, and their ongoing impact on related business. --Quantitative Indicator System for Retention Period Assessment Quantitative Indicators: Three core indicators are retained: Importance (I), Timeliness (T), and Ongoing Business Impact (B), each using a 1-5 point scale (1 being the lowest and 5 the highest). The criteria for these indicator values are as follows: Preserving Importance (I): First, extract keywords from the text of audio-visual archives and compare them with the importance keyword database. The more matches, the higher the score.
[0016] Timeliness (T): The interval between the file timestamp and the current time is calculated; the shorter the interval, the more timely the data.
[0017] Business Continuing Impact (B): Statistical profiles are cited more frequently on the enterprise platform; higher frequency results in higher scores.
[0018] The specific calculation formula is: Storage period score S = 0.4I + 0.3T + 0.3B.
[0019] Term mapping: S < 4 → short-term, 4 ≤ S < 7 → medium-term, S ≥ 7 → long-term. For example, for a file with I = 3, T = 3, B = 4, substituting these values gives S = 3.3, which is considered short-term.
[0020] In this embodiment, the original high-definition data of the audio-visual archives is acquired and preprocessed, as follows: First, a high-definition bitrate to low-bitrate conversion service is performed to obtain the original high-definition data of the audio-visual archives. Then, through a video encoding conversion algorithm, the high-definition bitrate is converted into a low-bitrate format suitable for storage and transmission. After the conversion is completed, the low-bitrate audio-visual files are saved to the specified storage location. Next, the low bitrate audio-visual file is sliced according to playback frames. One frame of image is extracted according to the set slicing interval, and the image data of each frame is obtained through video decoding technology and saved as an independent image file.
[0021] In this embodiment, a face recognition database is constructed as follows: A face recognition database is constructed using image data obtained from image slices. Face detection is performed on each slice image using a Haar cascade detector to identify face regions in the image. For detected faces, the LBPH feature extraction method is used to extract their feature vectors. The extracted feature vectors are compared with the existing face database. If it is a new face, its feature vector and related information (such as the audio-visual file number, possible identity of the person, etc.) are added to the face recognition database. Preferably, the construction of a face recognition database from sliced data adopts a five-step process: "slicing - detection - feature extraction - database entry - comparison," as detailed below: 1) Frame slicing: Use a slicing tool to slice one frame every 10 frames, generating 180 slices for a 1-minute video, named in the format "file number_frame number.jpg"; 2) Face detection: The MTCNN (Multi-task Cascaded Convolutional Neural Network) algorithm is used to locate face regions in slices and remove blurry / faceless slices; 3) Feature extraction: The FaceNet model is used to transform the face region into a 128-dimensional feature vector; 4) Library construction: Associate the feature vectors with the archive metadata (number, shooting time, preliminary identity tags, such as "suspected executive Zhang XX"), store them in the MySQL database, and set the similarity threshold to 0.85; 5) Identity matching: After the new file slice is detected, it is compared with the feature vector in the database. If the similarity is ≥0.85, the identity is determined and used for classification (e.g., those containing senior executives are classified as "important business activities").
[0022] Simultaneously, speech recognition and text transcription are performed. The audio portion of the audio-visual archive is processed, and the speech is converted into text through a speech recognition engine. First, the audio is denoised, and then the audio is divided into several segments. Speech recognition is performed on each segment, and the recognition results are integrated and corrected to finally obtain the complete text transcription content.
[0023] In this embodiment, an intelligent classification and identification model for audio-visual archives is constructed based on the business rule system, as follows: An intelligent classification and identification model is constructed based on the business rule system and the document titles, keywords, document summaries, and archiving department information of the audio-visual archives; a deep learning framework is used to build a neural network model; the model includes an input layer, an intermediate layer, and an output layer; the input layer receives preprocessed audio-visual archive text information (titles, keywords, summaries, etc.) and image feature information (face recognition results, etc.); the intermediate layer performs feature extraction and transformation through multiple fully connected layers or convolutional layers; and the output layer outputs prediction results based on the classification and identification objectives, such as document classification category and retention period, using the Softmax activation function.
[0024] In this embodiment, an intelligent classification and identification model for audio-visual archives is trained based on preprocessed audio-visual data and a face recognition database to obtain an optimized intelligent classification and identification model for audio-visual archives. Specifically, the preprocessed audio-visual data and face recognition database are divided into a training set, a validation set, and a test set in a ratio of 7:2:1. During training, data from the training set is input into the model, and the model parameters are continuously adjusted using the backpropagation algorithm to minimize the loss function between the model's predictions and the true labels. For example, the cross-entropy loss function is used to measure the difference between the predictions and the true classification labels, and the model parameters are updated using optimization algorithms such as stochastic gradient descent. After each round of training, the model is evaluated using the validation set, and changes in metrics such as accuracy and recall are observed. If the model's performance on the validation set no longer improves, it indicates a possible overfitting or underfitting problem, requiring adjustments to the model structure, optimization of hyperparameters, or data augmentation. After multiple rounds of training and adjustment, the model is finally evaluated using the test set to ensure that the model's generalization ability and accuracy meet the requirements. By utilizing machine learning and deep learning algorithms, the model is trained on massive amounts of historical archival data, enabling it to automatically learn the intrinsic relationships between various elements of audiovisual archives and their classification and identification results. For example, by learning from a large amount of audiovisual archive data with accurate classification labels, the model can accurately determine the category and retention period of new archives based on information such as their titles and keywords. During the model building process, the algorithm and parameters are continuously optimized to improve the model's accuracy and generalization ability.
[0025] In this embodiment, based on the optimized intelligent classification and identification model for audio-visual archives, each collected document in the pre-collection stage is intelligently assigned a document classification number, retention period, data verification, and cataloging and archiving tasks, as detailed below: When audiovisual archives enter the preliminary collection stage, the collected documents undergo initial processing, extracting document titles, keywords, and abstract text information. Simultaneously, image and voice analysis is performed on the audiovisual content to obtain facial recognition results and text transcription. This information is then input into an optimized intelligent classification and identification model for audiovisual archives. The model outputs identification results such as document classification number and retention period. Based on the identification results, data verification rules are automatically assigned to the documents according to a business rule system to verify their completeness and accuracy. For example, the system checks whether the document's encoding format is correct and whether the image and audio data are complete. If the verification passes, the documents are cataloged and archived according to the identification results, stored in the designated location of the corresponding archiving department, and relevant information is recorded in the archive management system, including document classification number, retention period, archiving time, and document title, for easy subsequent retrieval and management. Grassroots archivists can view the intelligent classification, identification, and archiving results through the system interface, and manually intervene and adjust any questionable parts, achieving efficient and accurate audiovisual archive organization and archiving.
[0026] An intelligent classification, identification, and archiving system for audio-visual archives includes a processor, a memory, and a computer program stored in the memory. When the processor executes the computer program, it specifically performs the steps in the integrated real-time coordination and risk scheduling method for main and microgrids as described above.
[0027] Those skilled in the art will understand that embodiments of the present invention can be provided as methods, systems, or computer program products. Therefore, the present invention can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention can take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.
[0028] This invention is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart illustrations and / or block diagrams. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.
[0029] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.
[0030] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.
[0031] The above description is merely a preferred embodiment of the present invention and is not intended to limit the present invention in any other way. Any person skilled in the art may make changes or modifications to the above-disclosed technical content to create equivalent embodiments. However, any simple modifications, equivalent changes, and modifications made to the above embodiments based on the technical essence of the present invention without departing from the scope of the present invention shall still fall within the protection scope of the present invention.
Claims
1. A method for intelligent classification, identification, and archiving of audio-visual archives, characterized in that, Includes the following steps: S1: Construct a business rule system based on the company's business rules for document classification, retention period determination, document title, archiving department, and document cataloging of audio-visual archives; S2: Acquire the original high-definition data of the audio-visual archives, preprocess it, and build a face recognition database; S3: Construct an intelligent classification and identification model for audio-visual archives; S4: Based on the preprocessed audio-visual data and the face recognition database, train the intelligent classification and identification model for audio-visual archives to obtain the optimized intelligent classification and identification model for audio-visual archives. S5: Based on the optimized intelligent classification and identification model for audio-visual archives, each collected document in the pre-collection stage is intelligently allocated, and verified based on the business rule system, it assists archivists in carrying out auxiliary classification, identification, organization, cataloging and archiving work for audio-visual archives.
2. The method for intelligent classification, identification, and archiving of audio-visual archives according to claim 1, characterized in that, The original high-definition data of the acquired audio-visual archives is preprocessed as follows: First, a high-definition bitrate to low-bitrate conversion service is performed to obtain the original high-definition data of the audio-visual archives. Then, through a video encoding conversion algorithm, the high-definition bitrate is converted into a low-bitrate format suitable for storage and transmission. After the conversion is completed, the low-bitrate audio-visual files are saved to the specified storage location. Next, the low bitrate audio-visual file is sliced according to playback frames. One frame of image is extracted according to the set slicing interval, and the image data of each frame is obtained through video decoding technology and saved as an independent image file.
3. The method for intelligent classification, identification, and archiving of audio-visual archives according to claim 2, characterized in that, The construction of the face recognition database is detailed as follows: A face recognition database was constructed using the image data obtained from the slices. Face detection was performed on each slice image using a Haar cascade detector to identify face regions in the image. For the detected faces, the LBPH feature extraction method was used to extract their feature vectors. The extracted feature vectors are compared with the existing face database. If it is a new face, its feature vectors and related information are added to the face recognition database. Simultaneously, speech recognition and text transcription are performed. The audio portion of the audio-visual archive is processed, and the speech is converted into text through a speech recognition engine. First, the audio is denoised, and then the audio is divided into several segments. Speech recognition is performed on each segment, and the recognition results are integrated and corrected to finally obtain the complete text transcription content.
4. The method for intelligent classification, identification, and archiving of audio-visual archives according to claim 1, characterized in that, The intelligent classification and identification model for audio-visual archives, based on the business rule system, is constructed as follows: An intelligent classification and identification model is built based on the business rule system and the document titles, keywords, document summaries, and archiving department information of the audio-visual archives; a deep learning framework is used to build a neural network model; the model includes an input layer, an intermediate layer, and an output layer; the input layer receives preprocessed audio-visual archive text information and image feature information; the intermediate layer performs feature extraction and transformation through multiple fully connected layers or convolutional layers; and the output layer outputs the prediction result based on the classification and identification objectives using the Softmax activation function.
5. The method for intelligent classification, identification, and archiving of audio-visual archives according to claim 4, characterized in that, The process involves training an intelligent classification and identification model for audio-visual archives based on preprocessed audio-visual data and a face recognition database, resulting in an optimized model. Specifically, the preprocessed audio-visual data and face recognition database are divided into training, validation, and test sets. During training, data from the training set is input into the model, and the model parameters are adjusted using backpropagation to minimize the loss function between the model's predictions and the true labels. Cross-entropy loss is used to measure the difference between the predictions and the true classification labels, and stochastic gradient descent and other optimization algorithms are used to update the model parameters. After each round of training, the model is evaluated using the validation set, and changes in accuracy and recall are observed. After several rounds of training and adjustment, the model is finally evaluated using the test set to ensure that its generalization ability and accuracy meet the requirements.
6. The method for intelligent classification, identification, and archiving of audio-visual archives according to claim 4, characterized in that, The optimized intelligent classification and identification model for audio-visual archives intelligently assigns a file classification number, retention period, data verification, and cataloging and archiving tasks to each collected file in the pre-collection stage, as detailed below: When audio-visual archives enter the pre-collection stage, the collected files are first preliminarily processed to extract file titles, keywords, and abstract text information. At the same time, image and voice analysis is performed on the audio-visual content to obtain facial recognition results and text transcription. This information is then input into an optimized intelligent classification and identification model for audio-visual archive files. The model outputs identification results such as file classification number and retention period. Based on the identification results, data verification rules are automatically assigned to the files according to the business rule system to verify the integrity and accuracy of the files. If the verification is successful, the documents will be cataloged and archived according to the verification results, stored in the designated location of the corresponding archiving department, and the relevant information will be recorded in the document management system.
7. A smart classification, identification, and archiving system for audio-visual archives, characterized in that, It includes a processor, a memory, and a computer program stored in the memory. When the processor executes the computer program, it specifically performs the steps in the integrated real-time coordination and risk scheduling method for main and distribution microgrids as described in any one of claims 1-6.