A data processing method, apparatus and equipment

Abstract

This application relates to the field of data processing, and discloses a data processing method, apparatus, and device. The data processing method of this application includes: acquiring data to be processed, the data to be processed including at least two categories of data: image data, voice data, text data, and video data; decomposing the data to be processed according to categories to obtain at least two categories of sub-data; extracting semantic features for each sub-data in the at least two categories according to feature extraction rules matching the category of the corresponding sub-data, obtaining at least two semantic features; and obtaining a result of the data to be processed based on the at least two semantic features. In this way, the processing object is transformed from a multi-type mixed data structure to a single-type data structure, which can reduce the difficulty of data processing and thus improve the accuracy of the data processing results.

Description

Technical Field

[0001] The present invention relates to the field of data processing, and in particular to a data processing method, apparatus and device. Background Technology

[0002] With the development of information technology, people's daily lives and work are filled with a large amount of diverse data, including unstructured data such as text, images, audio, and video. Unstructured data refers to data with irregular or incomplete data structures, lacking a predefined data model, and inconvenient to represent using the two-dimensional logic of a database. Because unstructured data inherently possesses semantically representative features with weak interrelationships, extracting semantic features from it is more difficult than from structured information. Often, multiple types of unstructured data are mixed; for example, an unstructured data document containing image information further complicates the extraction of core content from unstructured data.

[0003] Currently, semantic feature extraction from unstructured data typically relies on trained data processing models. These models perform semantic feature extraction by first identifying the unstructured data as a whole, and then extracting features from it.

[0004] However, because current data processing models are relatively limited in their scope of data processing, they cannot identify and process all types of unstructured data when faced with a mixture of different types. This can lead to inaccuracies in the semantic feature extraction results, resulting in a significant deviation between the final semantic feature extraction results and the actual data. Summary of the Invention

[0005] This application provides a data processing method, apparatus, and device to solve the problem of inaccurate results in existing data processing for handling multi-type mixed unstructured data.

[0006] In a first aspect, embodiments of this application provide a data processing method, the method comprising:

[0007] Acquire data to be processed, which includes at least two of the following categories: image data, voice data, text data, and video data;

[0008] The data to be processed is broken down according to categories to obtain sub-data in at least two categories;

[0009] For each sub-data in the at least two categories, semantic features are extracted according to the feature extraction rules that match the category of the corresponding sub-data, resulting in at least two semantic features;

[0010] The result of the data to be processed is obtained based on the at least two semantic features.

[0011] Optionally, the data to be processed is obtained from a preset unstructured database, and the data stored in the unstructured database is updated in real time according to data processing requirements. This ensures the sustainability of the data processing process.

[0012] Optionally, after acquiring the data to be processed and before breaking down the data to be processed according to category, the method further includes:

[0013] The acquired data undergoes preprocessing, including data integration, deduplication, and cataloging. This improves the accuracy of subsequent data breakdown.

[0014] In one possible implementation, prior to disassembling the unstructured data to be processed according to categories, the method further includes:

[0015] A preset data parsing model is invoked to perform content recognition on the data to be processed, thereby obtaining the data categories included in the data. In this way, by obtaining mixed data categories from the data to be processed, the category corresponding to each sub-data obtained in subsequent decomposition is more accurate.

[0016] In one possible implementation, the extraction of semantic features according to feature extraction rules matching the category of the corresponding sub-data is achieved by calling at least two preset models trained by the feature extraction rules. These preset models include: an image processing model, a speech processing model, a text processing model, and a video processing model. This allows for the processing of multi-type mixed unstructured data using various data processing models.

[0017] Optionally, the image processing model, speech processing model, text processing model, and video processing model are all independent and parallel. This ensures that their processes do not affect each other during the processing of multiple categories of unstructured data.

[0018] In one possible implementation, the text processing model extracts semantic features according to feature extraction rules that match the category of the corresponding sub-data, including:

[0019] Obtain the target text, which is derived from sub-data of the corresponding text category:

[0020] The target text is subjected to a first key feature and the corresponding metadata extraction to obtain a first extraction result. The first key feature includes: event text information.

[0021] A preset semantic analysis model is invoked to perform semantic analysis on the first extraction result, thereby obtaining the semantic analysis result of the first key feature;

[0022] The semantic analysis results of the first key feature are compared for similarity according to the first preset rule to obtain the first comparison result;

[0023] If the first comparison result is greater than or equal to a first preset threshold, the key feature semantic analysis result corresponding to the first comparison result is used to characterize the semantic features of the target text. This enables semantic extraction of text-type data, processing of unstructured text-type data, and separate extraction of metadata corresponding to the target document, which can provide data support for subsequent retrieval operations.

[0024] Optionally, before acquiring the target text, the method further includes constructing semantic extraction rules for the target text based on processing requirements. This ensures that the subsequent semantic analysis results of the first key feature meet actual needs, thereby improving the accuracy of the semantic extraction results of the target text.

[0025] In one possible implementation, the speech processing model is invoked to extract semantic features according to feature extraction rules that match the category of the corresponding sub-data, including:

[0026] The target speech is obtained from sub-data of the corresponding speech category;

[0027] The target speech is converted into corresponding text data by calling a preset prosody model and a preset speech-to-text model;

[0028] The speech is subjected to a second key feature and corresponding metadata extraction to obtain a second extraction result. The second key feature includes: speech frequency data, speech amplitude data and speech waveform data.

[0029] A preset semantic analysis model is invoked to perform semantic analysis on the second extraction result, thereby obtaining the semantic analysis result of the second key feature;

[0030] The semantic analysis results of the second key feature are compared for similarity according to the second preset rule to obtain the second comparison result;

[0031] If the second comparison result is greater than or equal to the second preset threshold, the key feature semantic analysis result corresponding to the second comparison result is used to characterize the semantic features of the target speech. In this way, semantic extraction of speech category data can be achieved through speech-to-text conversion, enabling the processing of unstructured speech data. Furthermore, the separate extraction of metadata corresponding to the target speech can provide data support for subsequent retrieval operations.

[0032] Optionally, the target speech includes: a recorded audio file and a real-time audio file.

[0033] In one possible implementation, the image processing model is invoked to extract semantic features according to feature extraction rules that match the category of the corresponding sub-data, including:

[0034] Acquire the target image, which is obtained from sub-data of the corresponding image category;

[0035] The target image is subjected to a third key feature and corresponding metadata extraction to obtain a third extraction result. The third key feature includes: text feature data, target feature data and face feature data.

[0036] A preset semantic analysis model is invoked to perform semantic analysis on the third extraction result, thereby obtaining the semantic analysis result of the third key feature;

[0037] The semantic analysis results of the third key feature are compared for similarity according to the third preset rule to obtain the third comparison result;

[0038] If the third comparison result is greater than or equal to a third preset threshold, the key feature semantic analysis result corresponding to the third comparison result is used to characterize the semantic features of the target image. This enables semantic extraction of image category data, processing of unstructured image data, and separate extraction of metadata corresponding to the target image, which can provide data support for subsequent retrieval operations.

[0039] Optionally, the target image format includes: JPEG, TIFF, RAW, BMP, GIF, and PNG.

[0040] Optionally, the image metadata includes: EXIF, IPTC, XMP, image time data, image pixel data, and image location data.

[0041] Optionally, the text feature data includes: text font data and text color data in the image, etc.

[0042] Optionally, the target feature data includes: target features identified in the image and target features not identified in the image.

[0043] Optionally, the facial feature data includes: facial detection data in the image.

[0044] In one possible implementation, the video processing model is invoked to extract semantic features according to feature extraction rules that match the category of the corresponding sub-data, including:

[0045] Obtain the target video, which is derived from sub-data of the corresponding video category;

[0046] The target video is segmented by features to obtain video stream information and audio stream information;

[0047] The video stream information is subjected to a fourth key feature and corresponding metadata extraction to obtain a fourth extraction result. The fourth key feature includes: video metadata, time data, scene data, shot data, representative frame data, and catalog data.

[0048] A preset semantic analysis model is invoked to perform semantic analysis on the fourth extraction result, thereby obtaining the semantic analysis result of the fourth key feature;

[0049] The semantic analysis results of the fourth key feature are compared with the similarity results according to the fourth preset rule to obtain the fourth comparison result;

[0050] If the fourth comparison result is greater than or equal to the fourth preset threshold, the key feature semantic analysis result corresponding to the fourth comparison result is used to characterize the semantic features of the target video. This enables semantic extraction of video category data, processing of unstructured video data, and separate extraction of metadata corresponding to the target video, which can provide data support for subsequent retrieval operations.

[0051] Optionally, the target video may be in the following formats: AVI, RMVB, FLV, and MP4.

[0052] Optionally, the target video is segmented using features, and the audio stream information is processed using a preset speech recognition model to obtain corresponding semantic features.

[0053] Secondly, embodiments of this application provide a data processing apparatus, comprising:

[0054] The acquisition module is used to acquire data to be processed, which includes at least two of the following categories of data: image data, voice data, text data, and video data;

[0055] The decomposition module is used to decompose the data to be processed according to categories to obtain sub-data of at least two categories;

[0056] The feature extraction module is used to extract semantic features for each sub-data in the at least two categories of sub-data according to the feature extraction rules that match the category of the corresponding sub-data, so as to obtain at least two semantic features;

[0057] The processing module is used to obtain the result of the data to be processed based on the at least two semantic features.

[0058] Thirdly, embodiments of this application also provide an electronic device, which includes: a processor, a memory, a communication interface, and a communication bus, wherein the processor, the memory, and the communication interface communicate with each other through the communication bus;

[0059] The memory is used to store executable instructions, which, when executed, cause the processor to perform a data processing method in either the first or second possible implementation.

[0060] Fourthly, embodiments of this application also provide a computer-readable storage medium storing executable instructions, which, when executed, cause a computing device to perform a data processing method in any possible implementation of the first or second aspect.

[0061] This application provides a data processing method. First, data to be processed is acquired, including at least two categories: image data, audio data, text data, and video data. Then, the data to be processed is decomposed according to category, resulting in at least two categories of sub-data. For each sub-data in the at least two categories, semantic features are extracted according to feature extraction rules matching the category of the corresponding sub-data, resulting in at least two semantic features. Finally, the result of the data to be processed is obtained based on the at least two semantic features. It can be seen that by decomposing data containing multiple categories to be processed, and obtaining the decomposed sub-data, since the decomposed sub-data are all independent categories, they are input into preset data processing models corresponding to the corresponding categories, and finally the data processing results are summarized. In this way, the processing object is transformed from a multi-type mixed data structure to a single-type data structure, which can reduce the difficulty of data processing and thus improve the accuracy of the data processing results. Attached Figure Description

[0062] Figure 1 This is a schematic flowchart of the data processing method provided in the embodiments of this application;

[0063] Figure 2a This is a schematic diagram of the text sub-data processing method provided in the embodiments of this application;

[0064] Figure 2b This is a schematic diagram of the voice sub-data processing method provided in the embodiments of this application;

[0065] Figure 2c This is a schematic diagram of the image sub-data processing method provided in the embodiments of this application;

[0066] Figure 2d This is a schematic diagram of the video sub-data processing method provided in the embodiments of this application;

[0067] Figure 3 This is a schematic diagram of the data processing apparatus provided in an embodiment of this application;

[0068] Figure 4 This is a schematic diagram of a data processing electronic device provided in an embodiment of this application. Detailed Implementation

[0069] The terminology used in the following embodiments of this application is for the purpose of describing alternative implementations and is not intended to be limiting of this application. As used in the specification and appended claims of this application, the singular expressions "a," "an," "the," "the," "the," and "this" are intended to include the plural expressions as well. It should also be understood that although the terms first, second, etc., may be used in the following embodiments to describe a class of objects, the objects are not limited to these terms. These terms are used to distinguish specific objects of that class of objects. For example, other classes of objects that may be described using the terms first, second, etc. in the following embodiments are similarly described and will not be repeated here.

[0070] This application provides an embodiment of an unstructured data processing method. In this method, firstly, data to be processed is acquired, including at least two categories of data: image data, audio data, text data, and video data. Then, the data to be processed is decomposed according to category, resulting in at least two categories of sub-data. For each sub-data in the at least two categories, semantic features are extracted according to feature extraction rules matching the category of the corresponding sub-data, resulting in at least two semantic features. Finally, the result of the data to be processed is obtained based on the at least two semantic features. It can be seen that by decomposing data containing multiple categories to be processed, and obtaining the decomposed sub-data, since the decomposed sub-data are all independent categories, they are respectively input into a preset data processing model corresponding to the corresponding category, and finally the data processing results are summarized. In this way, the processing object is transformed from a multi-type mixed data structure to a single-type data structure, which can reduce the difficulty of data processing and thus improve the accuracy of the data processing results.

[0071] The data processing method provided in this application can be executed by a single electronic device or by a computer cluster. The computer cluster includes at least two electronic devices that support the policy model training method described in this application. Any one of these electronic devices can implement the policy model training function described in this application through the policy model training method.

[0072] Any electronic device designed in the embodiments of this application can be an electronic device such as a mobile phone, tablet computer, wearable device (e.g., smartwatch, smart bracelet, etc.), laptop computer, desktop computer, and in-vehicle device. The electronic device is pre-installed with a policy model training application. It is understood that the embodiments of this application do not impose any limitations on the specific type of electronic device.

[0073] With the development of information technology, people's daily lives and work are filled with a large amount of diverse data, including unstructured data such as text, images, audio, and video. Unstructured data refers to data with irregular or incomplete data structures, lacking a predefined data model, and inconvenient to represent using the two-dimensional logic of a database. Because unstructured data inherently possesses semantically representative features with weak interrelationships, extracting semantic features from it is more difficult than from structured information. Often, multiple types of unstructured data are mixed; for example, an unstructured data document containing image information further complicates the extraction of core content from unstructured data.

[0074] Currently, semantic feature extraction from unstructured data typically relies on trained data processing models. These models perform semantic feature extraction by first identifying the unstructured data as a whole, and then extracting features from it.

[0075] However, because current data processing models are relatively limited in their scope of data processing, they cannot identify and process all types of unstructured data when faced with a mixture of different types. This can lead to inaccuracies in the semantic feature extraction results, resulting in a significant deviation between the final semantic feature extraction results and the actual data.

[0076] The following is a description of several exemplary embodiments, illustrating the technical solutions of the embodiments of this application and the technical effects produced by the technical solutions of this application.

[0077] For example, this application will illustrate the application of the data processing method in the scenario of unstructured data retrieval through the following embodiments.

[0078] Understandably, in order to ensure the accuracy of retrieval results for unstructured data in retrieval scenarios, it is necessary to process the data before retrieval (specifically, this includes extracting semantic features and classifying tags based on semantic features).

[0079] See Figure 1 , Figure 1 This is a schematic flowchart of a data processing method provided in an embodiment of this application, including the following steps:

[0080] Acquire data to be processed, which includes at least two of the following categories: image data, voice data, text data, and video data;

[0081] The data to be processed is broken down according to categories to obtain sub-data in at least two categories;

[0082] For each sub-data in the at least two categories, semantic features are extracted according to the feature extraction rules that match the category of the corresponding sub-data, resulting in at least two semantic features;

[0083] The result of the data to be processed is obtained based on the at least two semantic features.

[0084] It is understood that the data processing typically targets unstructured data. Processing unstructured data usually includes extracting key and hidden information from it. For example, it may involve acquiring the overall thematic semantics (i.e., semantic features) of a document or the thematic semantics of each paragraph.

[0085] Optionally, the data to be processed is obtained from a preset unstructured database, and the data stored in the unstructured database is updated in real time according to the data processing requirements.

[0086] For example, the unstructured database can be a database storing existing data to be processed, or it can be updated by importing external unstructured data into the unstructured database through data import.

[0087] Optionally, after acquiring the data to be processed and before breaking down the data to be processed according to category, the method further includes:

[0088] The acquired data is preprocessed, including data integration, data deduplication, data cataloging, data extraction, and format conversion.

[0089] For example, in order to fully utilize the data to be processed in the unstructured database, it is necessary to manage the unstructured data in a unified manner. Specifically, data cataloging includes:

[0090] Acquire the data to be managed, which is obtained from a preset unstructured database;

[0091] Based on the requirements, the data to be managed is hierarchically cataloged to obtain a file directory tree, which is used to present the hierarchical relationship of the data to be managed according to the requirements.

[0092] For example, after obtaining the file directory tree and before obtaining the data to be processed, the process further includes adding, deleting, modifying, querying, and deduplicating the data to be managed and the corresponding files to obtain the data to be processed. For example, the unstructured data can be hierarchically cataloged based on an organizational structure tree model to obtain a file directory tree, and then a file management model can be used to perform file addition, deletion, modification, querying, file preview, and file statistical analysis management.

[0093] In one possible implementation, prior to disassembling the unstructured data to be processed according to categories, the method further includes:

[0094] A preset data parsing model is invoked to perform content recognition on the data to be processed, so as to obtain the data categories included in the data to be processed.

[0095] For example, the data categories included in the data to be processed can be obtained by identifying tags in the content of the data to be processed. For instance, if the obtained data to be processed is a document containing image information (i.e., image data and text data coexist), then by obtaining and identifying the tags corresponding to the image data and the tags corresponding to the text data respectively, it can be finally confirmed that the data to be processed contains both image data and text data.

[0096] In one possible implementation, the extraction of semantic features according to feature extraction rules that match the category of the corresponding sub-data is achieved by calling at least two preset models trained by the feature extraction rules. The preset models include: an image processing model, a speech processing model, a text processing model, and a video processing model.

[0097] Optionally, the preset model is obtained by training the initial model using feature extraction rules that match the categories of the sub-data.

[0098] Optionally, the image processing model, speech processing model, text processing model, and video processing model are all independent and parallel.

[0099] It is understood that the preset models include, but are not limited to, the categories mentioned above, and can be supplemented according to actual needs.

[0100] In one possible implementation, the text processing model extracts semantic features according to feature extraction rules that match the category of the corresponding sub-data, including:

[0101] Obtain the target text, which is derived from sub-data of the corresponding text category:

[0102] The target text is subjected to a first key feature and the corresponding metadata extraction to obtain a first extraction result. The first key feature includes: event text information.

[0103] A preset semantic analysis model is invoked to perform semantic analysis on the first extraction result, thereby obtaining the semantic analysis result of the first key feature;

[0104] The semantic analysis results of the first key feature are compared for similarity according to the first preset rule to obtain the first comparison result;

[0105] If the first comparison result is greater than or equal to the first preset threshold, the key feature semantic analysis result corresponding to the first comparison result is used to characterize the semantic features of the target text.

[0106] It is understood that the metadata describes information resources or data objects, and here the metadata is used to describe other constituent data of the target text. The extraction of metadata is to support subsequent data retrieval in unstructured data scenarios.

[0107] Optionally, before obtaining the target text, the method further includes constructing semantic extraction rules for the target text based on processing requirements.

[0108] For example, such as Figure 2a The diagram illustrates the data processing procedure when the acquired sub-data is text data, including:

[0109] Obtain the sub-data of the text category (i.e., the target text);

[0110] The target text is preprocessed, including text deduplication and key feature extraction (i.e., metadata and event text information extraction in the first key feature extraction).

[0111] Semantic analysis is performed on the key feature extraction results to obtain the semantic analysis results (i.e., the semantic analysis results of the first key feature);

[0112] For the semantic analysis results, a similarity calculation model is used to compare the similarity with the standard results. If the comparison result is greater than or equal to a preset threshold (i.e., the first preset threshold), the semantics are visualized (i.e., the key feature semantic analysis result corresponding to the first comparison result is used to characterize the semantic features of the target text).

[0113] It is understandable that for the same text sub-data, different extraction requirements may lead to different extraction results. Therefore, during model training, the model can also be trained using the preset rules. That is, a parent rule is constructed, and unsupervised training iterations are performed on the parent rule to achieve the preset rules.

[0114] In one possible implementation, the speech processing model is invoked to extract semantic features according to feature extraction rules that match the category of the corresponding sub-data, including:

[0115] The target speech is obtained from sub-data of the corresponding speech category;

[0116] The target speech is converted into corresponding text data by calling a preset prosody model and a preset speech-to-text model;

[0117] The speech is subjected to a second key feature and corresponding metadata extraction to obtain a second extraction result. The second key feature includes: speech frequency data, speech amplitude data and speech waveform data.

[0118] A preset semantic analysis model is invoked to perform semantic analysis on the second extraction result, thereby obtaining the semantic analysis result of the second key feature;

[0119] The semantic analysis results of the second key feature are compared for similarity according to the second preset rule to obtain the second comparison result;

[0120] If the second comparison result is greater than or equal to the second preset threshold, the key feature semantic analysis result corresponding to the second comparison result is used to characterize the semantic features of the target speech.

[0121] Optionally, the target speech includes: a recorded audio file and a real-time audio file.

[0122] For example, such as Figure 2b The diagram illustrates the data processing procedure when the acquired sub-data is voice data. Voice sub-data is categorized according to the receiving method, typically into real-time voice files (e.g., real-time call voice) and recorded voice files. Before analysis and processing, preprocessing (e.g., noise reduction, deduplication) is performed after acquiring the voice sub-data.

[0123] After preprocessing, the speech sub-data is subjected to key feature extraction (i.e., second key feature extraction) and similarity result comparison through a preset speech data analysis and processing model to obtain the semantic features corresponding to the key features.

[0124] Finally, for the speech sub-files after key feature extraction, text transcription is performed (by a pre-set speech-to-text model). Then, a pre-set text analysis model is called to analyze and process the text (e.g., semantic feature extraction). Finally, based on the classification results, a speech tagging classification model is used to classify and display the results.

[0125] If the obtained audio file is in document format, then the document-type audio file can first be converted into audio data to form speech data (i.e., speech sub-data). Then, semantic features can be extracted and the final classification and display can be achieved according to the above method.

[0126] In one possible implementation, the image processing model is invoked to extract semantic features according to feature extraction rules that match the category of the corresponding sub-data, including:

[0127] Acquire the target image, which is obtained from sub-data of the corresponding image category;

[0128] The target image is subjected to a third key feature and corresponding metadata extraction to obtain a third extraction result. The third key feature includes: text feature data, target feature data and face feature data.

[0129] A preset semantic analysis model is invoked to perform semantic analysis on the third extraction result, thereby obtaining the semantic analysis result of the third key feature;

[0130] The semantic analysis results of the third key feature are compared for similarity according to the third preset rule to obtain the third comparison result;

[0131] If the third comparison result is greater than or equal to the third preset threshold, the key feature semantic analysis result corresponding to the third comparison result is used to characterize the semantic features of the target image.

[0132] Optionally, the target image format includes: JPEG, TIFF, RAW, BMP, GIF, and PNG.

[0133] Optionally, the image metadata includes: EXIF, IPTC, XMP, image time data, image pixel data, and image location data.

[0134] Optionally, the text feature data includes: text font data and text color data in the image, etc.

[0135] Optionally, the target feature data includes: target features identified in the image and target features not identified in the image.

[0136] Optionally, the facial feature data includes: facial detection data in the image.

[0137] For example, such as Figure 2c As shown, the data processing procedure is described when the acquired sub-data is image data. After acquiring the image sub-data (i.e. the target image), the target image is first preprocessed (denoising, deduplication, etc.).

[0138] Then, through the preset metadata parsing model, feature extraction is performed on the metadata, and image detection is performed simultaneously on the preset image recognition model to achieve feature extraction of the image, including: text feature data, target feature data and face feature data in the image (i.e., the third key feature);

[0139] The semantic analysis results of the third key feature are compared for similarity to obtain their respective recognition results (target recognition result, person recognition result, and text recognition result, etc.). Finally, the similarity recognition results and the extracted metadata results are combined to achieve image labeling and classification.

[0140] Optionally, for text recognition (or character features distinct from the image) in an image, an image text recognition model based on DBNET and CRNN algorithms can be constructed. Through character detection and character recognition technologies, text or other characters in the image can be detected, including attributes such as font and character color.

[0141] Optionally, for target recognition, image classification and recognition can be performed by constructing a OneStage target detection model based on deep learning and a deep learning model based on ResNet and MobileNet, extracting key feature information, and realizing the cropping and labeling of targets.

[0142] Optionally, for people recognition in images, a people recognition model based on deep learning feature extraction can be constructed, combined with CascadeResNet technology, to achieve face detection, key point detection, image preprocessing, feature extraction, and comparison recognition.

[0143] In one possible implementation, the video processing model is invoked to extract semantic features according to feature extraction rules that match the category of the corresponding sub-data, including:

[0144] Obtain the target video, which is derived from sub-data of the corresponding video category;

[0145] The target video is segmented by features to obtain video stream information and audio stream information;

[0146] The video stream information is subjected to a fourth key feature and corresponding metadata extraction to obtain a fourth extraction result. The fourth key feature includes: video metadata, time data, scene data, shot data, representative frame data, and catalog data.

[0147] A preset semantic analysis model is invoked to perform semantic analysis on the fourth extraction result, thereby obtaining the semantic analysis result of the fourth key feature;

[0148] The semantic analysis results of the fourth key feature are compared with the similarity results according to the fourth preset rule to obtain the fourth comparison result;

[0149] If the fourth comparison result is greater than or equal to the fourth preset threshold, then the key feature semantic analysis result corresponding to the fourth comparison result is used to characterize the semantic features of the target video.

[0150] For example, such as Figure 2d As shown, the data processing procedure is described when the acquired sub-data is video data. After acquiring the video-type sub-data (i.e., the target video), the metadata of the target video is extracted. The metadata is extracted through a preset metadata parsing model and a metadata extraction model, and the metadata is managed in a unified manner for easy use later.

[0151] Then, the video is split into audio stream and video stream, dividing the original target video into separate video data and separate audio data.

[0152] For audio data, a speech recognition analysis model is constructed to transcribe speech into text, and the corresponding text semantic recognition model is combined to obtain the semantic features corresponding to the speech (as described in the above embodiments, and will not be repeated here).

[0153] For video data, frame segmentation technology is used to divide the video into specific image frames, and key features of the image frames are identified and key frames are extracted. Specifically, key frames are obtained through scene analysis and video motion analysis technology to assist users in video cataloging.

[0154] Finally, a video algorithm recognition model is constructed, and the extracted videos are processed uniformly (specifically, a comprehensive evaluation of the comparison results of the semantic features of the video stream, the semantic features of the audio stream, and the combination of the two) to obtain the video recognition results.

[0155] This application provides a data processing method. First, data to be processed is acquired, including at least two categories: image data, audio data, text data, and video data. Then, the data to be processed is decomposed according to category, resulting in at least two categories of sub-data. For each sub-data in the at least two categories, semantic features are extracted according to feature extraction rules matching the category of the corresponding sub-data, resulting in at least two semantic features. Finally, the result of the data to be processed is obtained based on the at least two semantic features. It can be seen that by decomposing data containing multiple categories to be processed, and obtaining the decomposed sub-data, since the decomposed sub-data are all independent categories, they are input into preset data processing models corresponding to the corresponding categories, and finally the data processing results are summarized. In this way, the processing object is transformed from a multi-type mixed data structure to a single-type data structure, which can reduce the difficulty of data processing and thus improve the accuracy of the data processing results.

[0156] The above embodiments describe various implementations of the data processing method provided in this application from the perspective of the action logic and learning algorithm processing performed by the electronic device, including the acquisition method of the data to be processed, the breakdown of the data to be processed, the extraction of semantic features from the sub-data, and the obtaining of data processing results based on the semantic features. It should be understood that the processing steps, such as the acquisition method of the data to be processed, the breakdown of the data to be processed, the extraction of semantic features from the sub-data, and the obtaining of data processing results based on the semantic features, can be implemented in hardware or a combination of hardware and computer software in this application embodiment. Whether a certain function is executed by hardware or by computer software driving hardware depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0157] For example, if the above implementation steps are implemented through software modules to achieve the corresponding functions, such as... Figure 3 As shown, the data processing device may include:

[0158] The acquisition module is used to acquire data to be processed, which includes at least two of the following categories of data: image data, voice data, text data, and video data;

[0159] The decomposition module is used to decompose the data to be processed according to categories to obtain sub-data of at least two categories;

[0160] The feature extraction module is used to extract semantic features for each sub-data in the at least two categories of sub-data according to the feature extraction rules that match the category of the corresponding sub-data, so as to obtain at least two semantic features;

[0161] The processing module is used to obtain the result of the data to be processed based on the at least two semantic features.

[0162] It is understandable that the above division of modules / units is merely a logical functional division. In actual implementation, the functions of each module can be integrated into hardware entities. For example, the acquisition module, processing module, and comparison module can be integrated into the processor. The programs and instructions that implement the functions of each module can be maintained in memory. For example, Figure 4 An electronic device is provided, comprising a processor, a transceiver, and a memory. The transceiver is used for acquiring data to be processed. The memory can store semantic features corresponding to sub-data, or it can store code for execution by the processor. When the processor executes the code stored in the memory, the electronic device performs some or all of the operations of the policy model training method described above.

[0163] For details on the specific implementation process, please refer to the embodiments illustrated in the above method diagram, which will not be described in detail here.

[0164] In a specific implementation, corresponding to the aforementioned electronic device, this application embodiment also provides a computer storage medium. The computer storage medium disposed in the electronic device can store a program, which, when executed, can implement some or all of the steps in various embodiments of the training method for the strategy model. The storage medium can be a magnetic disk, optical disk, read-only memory (ROM), or random access memory (RAM), etc.

[0165] One or more of the above modules or units can be implemented by software, hardware, or a combination of both. When any of the above modules or units is implemented by software, the software exists as computer program instructions and is stored in memory. The processor can be used to execute the program instructions and implement the above method flow. The processor can include, but is not limited to, at least one of the following: a central processing unit (CPU), a microprocessor, a digital signal processor (DSP), a microcontroller unit (MCU), or an artificial intelligence processor, etc., and various computing devices that run software. Each computing device may include one or more cores for executing software instructions to perform calculations or processing. The processor can be built into a SoC (System-on-a-Chip) or an application-specific integrated circuit (ASIC), or it can be a separate semiconductor chip. In addition to the cores within the processor for executing software instructions to perform calculations or processing, it may further include necessary hardware accelerators, such as field-programmable gate arrays (FPGAs), PLDs (programmable logic devices), or logic circuits that implement dedicated logic operations.

[0166] When the above modules or units are implemented in hardware, the hardware can be any one or any combination of CPU, microprocessor, DSP, MCU, artificial intelligence processor, ASIC, SoC, FPGA, PLD, special purpose digital circuit, hardware accelerator or non-integrated discrete device, which can run the necessary software or perform the above method flow independently of software.

[0167] Furthermore, Figure 4 The system may also include a bus interface, which can include any number of interconnected buses and bridges, specifically linking various circuits of one or more processors (represented by the processor) and memory (represented by the memory). The bus interface can also link various other circuits such as peripheral devices, voltage regulators, and power management circuits, which are well known in the art and therefore will not be described further herein. The bus interface provides the interface. The transceiver provides the unit for communicating with various other devices over the transmission medium. The processor is responsible for managing the bus architecture and general processing, and the memory can store the data used by the processor during operation.

[0168] When the above modules or units are implemented using software, they can be implemented in whole or in part as a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, all or part of the processes or functions described in the embodiments of the present invention are generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired (e.g., coaxial cable, fiber optic, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that integrates one or more available media. The available medium can be a magnetic medium (e.g., floppy disk, hard disk, magnetic tape), an optical medium (e.g., DVD), or a semiconductor medium (e.g., solid-state disk (SSD)).

[0169] It should be understood that in the various embodiments of this application, the sequence number of each process does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments.

[0170] The various parts of this specification are described in a progressive manner. Similar or identical parts between the different embodiments can be referred to interchangeably. Each embodiment focuses on its differences from other embodiments. In particular, the device and system embodiments are basically similar to the method embodiments, so the descriptions are relatively simple; relevant details can be found in the description of the method embodiments.

[0171] Although alternative embodiments of this application have been described, those skilled in the art, upon learning the basic inventive concept, can make further changes and modifications to these embodiments. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments as well as all changes and modifications falling within the scope of this application.

[0172] The specific embodiments described above further illustrate the purpose, technical solution, and beneficial effects of this application. It should be understood that the above description is only a specific embodiment of this application and is not intended to limit the scope of protection of this application. Any modifications, equivalent substitutions, improvements, etc., made on the basis of the technical solution of this application should be included within the scope of protection of this invention.

Claims

1. A data processing method, characterized in that, The data processing method includes: Acquire data to be processed, which includes at least two of the following categories: image data, voice data, text data, and video data; The data to be processed is broken down according to categories to obtain sub-data in at least two categories; For each sub-data in the at least two categories, semantic features are extracted according to the feature extraction rules that match the category of the corresponding sub-data, resulting in at least two semantic features; The result of obtaining the data to be processed is obtained based on the at least two semantic features. The text processing model is invoked to extract semantic features according to feature extraction rules that match the category of the corresponding sub-data, including: Obtain the target text, which is derived from sub-data of the corresponding text category; The target text is subjected to a first key feature and the corresponding metadata extraction to obtain a first extraction result. The first key feature includes: event text information. A preset semantic analysis model is invoked to perform semantic analysis on the first extraction result, thereby obtaining the semantic analysis result of the first key feature; The semantic analysis results of the first key feature are compared for similarity according to the first preset rule to obtain the first comparison result; If the first comparison result is greater than or equal to the first preset threshold, the key feature semantic analysis result corresponding to the first comparison result is used to characterize the semantic features of the target text.

2. The data processing method as described in claim 1, characterized in that, Before the data to be processed is broken down according to category, the method further includes: A preset data parsing model is invoked to perform content recognition on the data to be processed, so as to obtain the data categories included in the data to be processed.

3. The data processing method as described in claim 1, characterized in that, The semantic features are extracted according to the feature extraction rules that match the category of the corresponding sub-data. This is achieved by calling at least two preset models trained by the feature extraction rules. The preset models include: an image processing model, a speech processing model, a text processing model, and a video processing model.

4. The data processing method as described in claim 1, characterized in that, The method further includes invoking the speech processing model to extract semantic features according to feature extraction rules that match the category of the corresponding sub-data, and also includes: The target speech is obtained from sub-data of the corresponding speech category; The target speech is converted into corresponding text data by calling a preset prosody model and a preset speech-to-text model; The speech is subjected to a second key feature and corresponding metadata extraction to obtain a second extraction result. The second key feature includes: speech frequency data, speech amplitude data and speech waveform data. A preset semantic analysis model is invoked to perform semantic analysis on the second extraction result, thereby obtaining the semantic analysis result of the second key feature; The semantic analysis results of the second key feature are compared for similarity according to the second preset rule to obtain the second comparison result; If the second comparison result is greater than or equal to the second preset threshold, the key feature semantic analysis result corresponding to the second comparison result is used to characterize the semantic features of the target speech.

5. The data processing method as described in claim 1, characterized in that, The method further includes invoking the image processing model to extract semantic features according to feature extraction rules that match the category of the corresponding sub-data, and also includes: Acquire the target image, which is obtained from sub-data of the corresponding image category; The target image is subjected to a third key feature and corresponding metadata extraction to obtain a third extraction result. The third key feature includes: text feature data, target feature data and face feature data. A preset semantic analysis model is invoked to perform semantic analysis on the third extraction result, thereby obtaining the semantic analysis result of the third key feature; The semantic analysis results of the third key feature are compared for similarity according to the third preset rule to obtain the third comparison result; If the third comparison result is greater than or equal to the third preset threshold, the key feature semantic analysis result corresponding to the third comparison result is used to characterize the semantic features of the target image.

6. The data processing method as described in claim 1, characterized in that, The method further includes invoking the video processing model to extract semantic features according to feature extraction rules that match the category of the corresponding sub-data, and also includes: Obtain the target video, which is derived from sub-data of the corresponding video category; The target video is segmented by features to obtain video stream information and audio stream information; The video stream information is subjected to a fourth key feature and corresponding metadata extraction to obtain a fourth extraction result. The fourth key feature includes: video metadata, time data, scene data, shot data, representative frame data, and catalog data. A preset semantic analysis model is invoked to perform semantic analysis on the fourth extraction result, thereby obtaining the semantic analysis result of the fourth key feature; The semantic analysis results of the fourth key feature are compared with the similarity results according to the fourth preset rule to obtain the fourth comparison result; If the fourth comparison result is greater than or equal to the fourth preset threshold, then the key feature semantic analysis result corresponding to the fourth comparison result is used to characterize the semantic features of the target video.

7. A data processing apparatus, characterized in that, The data processing device includes: The acquisition module is used to acquire data to be processed, which includes at least two of the following categories of data: image data, voice data, text data, and video data; The decomposition module is used to decompose the data to be processed according to categories to obtain sub-data of at least two categories; The feature extraction module is used to extract semantic features for each sub-data in the at least two categories of sub-data according to the feature extraction rules that match the category of the corresponding sub-data, so as to obtain at least two semantic features; The processing module is used to obtain the result of the data to be processed based on the at least two semantic features. The text processing model is invoked to extract semantic features according to feature extraction rules that match the category of the corresponding sub-data, including: Obtain the target text, which is derived from sub-data of the corresponding text category: The target text is subjected to a first key feature and the corresponding metadata extraction to obtain a first extraction result. The first key feature includes: event text information. A preset semantic analysis model is invoked to perform semantic analysis on the first extraction result, thereby obtaining the semantic analysis result of the first key feature; The semantic analysis results of the first key feature are compared for similarity according to the first preset rule to obtain the first comparison result; If the first comparison result is greater than or equal to the first preset threshold, the key feature semantic analysis result corresponding to the first comparison result is used to characterize the semantic features of the target text.

8. A computing device, characterized in that, include: The processor, memory, communication interface, and communication bus are provided, wherein the processor, memory, and communication interface communicate with each other via the communication bus. The memory is used to store executable instructions, which, when executed, cause the processor to perform the data processing method as described in any one of claims 1-6.

9. A computer storage medium, characterized in that, The storage medium stores executable instructions, which, when executed, cause the computing device to perform the data processing method as described in any one of claims 1-6.

Images