Data processing method, device, equipment and storage medium
By calculating the similarity value based on the sample set in big data, the problems of low detection efficiency and poor effect of abnormal data in the prior art are solved, and rapid and accurate abnormal data identification and processing are achieved, and data processing efficiency is improved. stability.
Patent Information
- Application Number
- CN202210299138.0
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2022-03-25
- Publication Date
- 2025-01-07
- Estimated Expiration
- 2042-03-25
AI Technical Summary
The prior art has problems such as low efficiency, long time consumption and poor monitoring effect in abnormal data detection. Especially in big data, abnormal data has diverse forms and changes in distribution and shape over time, resulting in increased detection challenges.
By obtaining the pending data, the similarity value between the pending data and the sample set is calculated based on the preset sample set, the target matching data is quickly and accurately matched, and the attribute information of the pending data is determined based on its annotation information, so as to directly maintain the original data Features, avoid data processing.
It improves data processing efficiency and processing effect, accurately identify abnormal data, enhances the stability and accuracy of data recognition, and overcomes the problem of unified sample feature dimensions and long monitoring periods in the prior art.
Smart Images

Figure CN114676774B_ABST
Abstract
Description
Technical Field
[0001] The present disclosure relates to the technical fields of big data, information flow, intelligent search, and deep learning in data processing, and in particular to a data processing method, device, equipment, and storage medium. Background Art
[0002] With the development of network technology, big data is spreading across various fields. Big data contains rich information, but at the same time, abnormal data inevitably exists, which brings inconvenience to subsequent use. How to accurately identify abnormal data in big data is very important.
[0003] In practical applications, since abnormal data comes in various forms and changes with the times, an abnormal data identification method is needed to improve the abnormal data identification effect in big data and improve the stability of data. Summary of the invention
[0004] The present disclosure provides a data processing method, apparatus, device and storage medium.
[0005] According to a first aspect of the present disclosure, there is provided a data processing method, comprising:
[0006] Get the data to be processed;
[0007] Processing the data to be processed based on a preset sample set, and determining at least one similarity value between the data to be processed and at least one sample data included in the sample set;
[0008] Based on the at least one similarity value, determining target matching data in the at least one sample data;
[0009] According to the labeling information of the target matching data, attribute information of the data to be processed is determined.
[0010] According to a second aspect of the present disclosure, there is provided a data processing device, comprising:
[0011] An acquisition unit, used for acquiring data to be processed;
[0012] a processing unit, configured to process the data to be processed based on a preset sample set, and determine at least one similarity value between the data to be processed and at least one sample data included in the sample set;
[0013] a matching determination unit, configured to determine target matching data in the at least one sample data based on the at least one similarity value;
[0014] The attribute determination unit is used to determine the attribute information of the data to be processed according to the label information of the target matching data.
[0015] According to a third aspect of the present disclosure, there is provided an electronic device, including:
[0016] at least one processor; and
[0017] a memory communicatively connected to the at least one processor; wherein,
[0018] The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the method described in the first aspect.
[0019] According to a fourth aspect of the present disclosure, a non-transitory computer-readable storage medium storing computer instructions is provided, wherein the computer instructions are used to cause the computer to execute the method described in the first aspect.
[0020] According to a fifth aspect of the present disclosure, a computer program product is provided, comprising: a computer program, wherein the computer program is stored in a readable storage medium, at least one processor of an electronic device can read the computer program from the readable storage medium, and the at least one processor executes the computer program so that the electronic device executes the method described in the first aspect.
[0021] According to the technical solution disclosed in the present invention, the attribute information of the data can be accurately determined, thereby improving the data processing efficiency and processing effect.
[0022] It should be understood that the content described in this section is not intended to identify the key or important features of the embodiments of the present disclosure, nor is it intended to limit the scope of the present disclosure. Other features of the present disclosure will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS
[0023] The accompanying drawings are used to better understand the present solution and do not constitute a limitation of the present disclosure.
[0024] Figure 1 It is a schematic diagram of an application scenario applicable to the embodiment of the present disclosure;
[0025] Figure 2 is a flowchart of a data processing method provided by the first embodiment of the present disclosure;
[0026] Figure 3 is a flowchart of a data processing method provided by the second embodiment of the present disclosure;
[0027] Figure 4 is a flowchart of a data processing method provided by the third embodiment of the present disclosure;
[0028] Figure 5is a flowchart of a data processing method provided by the fourth embodiment of the present disclosure;
[0029] Figure 6 is a structural schematic diagram of a data processing device provided by an embodiment of the present disclosure;
[0030] Figure 7 is a schematic block diagram of an example electronic device for implementing an embodiment of the present disclosure. DETAILED DESCRIPTION
[0031] The following is a description of exemplary embodiments of the present disclosure in conjunction with the accompanying drawings, including various details of the embodiments of the present disclosure to facilitate understanding, which should be considered as merely exemplary. Therefore, it should be recognized by those of ordinary skill in the art that various changes and modifications may be made to the embodiments described herein without departing from the scope and spirit of the present disclosure. Similarly, for the sake of clarity and conciseness, descriptions of well-known functions and structures are omitted in the following description.
[0032] Data processing refers to the process of using electronic computers to input, edit, summarize, calculate, analyze, predict, store and manage a large amount of raw data or information. Data can be in the form of numbers, text, images or sounds. The basic purpose of data processing is to extract relatively valuable and meaningful data from a large amount of chaotic and difficult to understand data. Data processing runs through all areas of social production and social life.
[0033] Optionally, big data is an object of data processing. It is a data collection whose scale far exceeds the capabilities of traditional database software tools in terms of acquisition, storage, management, and analysis. It has four major characteristics: massive data scale, fast data flow, diverse data types, and low value density.
[0034] At this stage, big data is well-known and spreads across all fields. Big data contains rich information, and at the same time, it inevitably contains abnormal data, which brings troubles to subsequent analysis and analysis. Abnormal data exists in various forms and changes with the times. In one way, a supervised machine learning model can be used for abnormal monitoring, but this method is more strict for the production of samples. For example, the characteristic dimensions of the samples need to be unified, the production cycle is long, and the balance of positive and negative samples needs to be considered. There are problems such as low efficiency and long time consumption for abnormal monitoring. In another way, an unsupervised machine learning model can be used to supervise the specific features of the data, but this method has poor monitoring effect on random time series in the data. Therefore, in order to overcome the problems of low efficiency, long time consumption and poor monitoring effect of abnormal monitoring, a monitoring method for abnormal data stream is urgently needed to determine abnormal data in big data.
[0035] In the related technology, the common abnormal data detection methods generally focus on data monitoring in low-dimensional space, mainly including statistical methods and unsupervised machine learning methods. Statistical methods include interquartile range (IQR) and zero-mean (Z-score) standardization methods; unsupervised machine learning methods mainly include density-based spatial clustering of applications with noise (DBSCAN) method, isolation forest and other methods. Among them, the IQR method determines whether there are outliers in the data by calculating the quartiles. The object of determination is based on the numerical type. This method is simple and easy to implement but prone to misjudgment; the premise of the Z-score standardization method is to assume that the data obeys the normal distribution. However, in actual applications, most of the data do not obey the normal distribution; the DBSCAN clustering method is only applicable to low-dimensional spatial data, and it needs to adjust the parameters repeatedly to achieve better results when clustering data; the isolation forest method is applicable to continuous data, not to particularly high-dimensional data, and has certain limitations in actual applications.
[0036] From the above analysis, it can be seen that the use of abnormal data detection methods in related technologies has certain limitations and poor detection effects. In addition, the distribution and form of data will continue to update over time, which brings greater challenges to the detection of abnormal data.
[0037] In response to the above technical problems, the technical conception process of the embodiment of the present disclosure is as follows: In the process of studying the existing anomaly detection, the inventors found that samples can be directly collected based on time series without processing, and all the characteristics of the original data can be maintained, so as to quickly create a sample set. Furthermore, when monitoring outliers, it is only necessary to input the original sequence of the data to be processed, and by calculating the similarity value between the data to be processed and the sample data in the sample set, the relevant samples can be quickly and accurately matched. Based on the annotation information of the relevant samples, the attribute information of the data to be processed is given, such as abnormal conditions, category information, etc.
[0038] Based on the above technical conception process, the embodiment of the present disclosure provides a data processing method, which obtains the data to be processed, processes the data to be processed based on a preset sample set, determines at least one similarity value between the data to be processed and at least one sample data included in the sample set, and then determines the target matching data in at least one sample data based on the at least one similarity value, and finally determines the attribute information of the data to be processed according to the annotation information of the target matching data. This technical solution can accurately determine the target matching data, lays a foundation for the subsequent determination of the attribute information of the data to be processed, thereby improving the data processing efficiency and data processing effect.
[0039] The present disclosure provides a data processing method, device, equipment and storage medium, which are applied to the fields of big data, information flow, intelligent search and deep learning technology in data processing, and can accurately determine the attribute information of data, thereby improving data processing efficiency and processing effect.
[0040] It should be noted that the data to be processed in this embodiment is not data in a specific scenario and cannot reflect the information of a specific scenario. Moreover, the data to be processed and the sample set in this embodiment are all from public data sets.
[0041] In the technical solution of the present disclosure, the collection, storage, use, processing, transmission, provision and disclosure of user personal information involved are in compliance with the provisions of relevant laws and regulations and do not violate public order and good morals.
[0042] It is understandable that the embodiments of the present disclosure are mainly explained and described in the context of big data, and are mainly used to detect abnormal data in big data. Moreover, the data in this embodiment can be high-dimensional time series data, and the dimension of the data is not limited. Optionally, the technical solution of the embodiments of the present disclosure can also be applied to other scenarios, such as user behavior recognition, advertising recommendation, and time series prediction, and this embodiment does not limit the application scenario.
[0043] For example, Figure 1 Schematic diagram of an application scenario applicable to the embodiment of the present disclosure. Figure 1 As shown, the application scenario diagram may include: a terminal device 11, a network 12, a server 13 and a processing device 14.
[0044] Optionally, the terminal device 11 can communicate with the server 13 through the network 12, so that the server 13 can obtain the user's data processing instruction, thereby determining the data to be processed based on the data processing instruction, and transmitting it to the processing device 14. Accordingly, the processing device 14 can receive the data to be processed from the server 13 and execute the technical solution of the embodiment of the present disclosure.
[0045] Optionally, the processing device 14 may also directly receive data processing instructions sent by the processing personnel through the terminal device 11, and obtain the data to be processed from its own database or other devices based on the data processing instructions, and then execute the technical solution of the embodiment of the present disclosure.
[0046] It is understandable that the embodiments of the present disclosure do not limit the specific manner in which the processing device 14 obtains the data to be processed, which can be determined according to the actual scenario and will not be elaborated here.
[0047] In this embodiment, the processing device 14 can execute the program code of the data processing method provided in this application based on the acquired data to be processed to determine the attribute information of the data to be processed.
[0048] Optional, Figure 1 The application scenario shown may also include a data storage device 15 , which may be connected to the server 13 or the processing device 14 , and is used to store data output by the server 13 and / or data processing results output by the processing device 14 .
[0049] It should be noted that the attached Figure 1 This is only a schematic diagram of an application scenario provided by the embodiment of the present disclosure. Figure 1 does not limit the equipment included in Figure 1 The positional relationship between the devices in the Figure 1 In the embodiment, the data storage device 15 can be an external memory relative to the server 13 or the processing device 14. In other cases, the data storage device 15 can also be placed in the server 13 or the processing device 14. The processing device 14 can be a device that exists independently of the server 13 or a component integrated in the server 13. The embodiments of the present application do not limit this.
[0050] It should also be noted that the processing device 14 in the disclosed embodiment can be a terminal device, a server or a virtual machine, etc., or a distributed computer system composed of one or more servers and / or computers, etc. Among them, the terminal device includes but is not limited to: smart phones, laptops, desktop computers, platform computers, vehicle-mounted devices, smart wearable devices, etc., which are not limited in the disclosed embodiment. The server can be an ordinary server or a cloud server. The cloud server is also called a cloud computing server or a cloud host, which is a host product in the cloud computing service system. The server can also be a server of a distributed system, or a server combined with a blockchain.
[0051] It is worth noting that the product implementation form of the present disclosure is a program code contained in the platform software and deployed on a processing device (which may also be hardware with computing capabilities such as a computing cloud or a mobile terminal). Figure 1 In the system structure diagram shown, the program code of the present disclosure can be stored inside the processing device. When running, the program code runs in the host memory and / or GPU memory of the processing device.
[0052] In the embodiments of the present disclosure, "multiple" refers to two or more. "And / or" describes the association relationship of associated objects, indicating that three relationships may exist. For example, A and / or B can represent: A exists alone, A and B exist at the same time, and B exists alone. The character " / " generally indicates that the associated objects before and after are in an "or" relationship.
[0053] Next, combined with the above Figure 1 The application scenarios shown are used to describe the technical solutions of the present disclosure in detail through specific embodiments. It should be noted that the following specific embodiments can be combined with each other, and the same or similar concepts or processes may not be described in detail in some embodiments.
[0054] For example, Figure 2 is a flow chart of a data processing method provided by the first embodiment of the present disclosure. Figure 1 The processing device in the example is used as the execution subject for explanation. Figure 2 As shown, the data processing method may include the following steps:
[0055] S201, obtaining data to be processed.
[0056] In the embodiments of the present disclosure, the processing device may receive the data to be processed from other devices, for example, obtain the data to be processed from a server based on the acquired link or website, obtain the data to be processed from an associated data platform, etc., or read the data to be processed from a database stored in itself (in this case, a database is deployed in the processing device). The embodiments of the present disclosure do not limit the acquisition process of the data to be processed, which can be determined according to the actual scenario.
[0057] It can be understood that in the embodiments of the present disclosure, the data to be processed acquired by the processing device is usually unprocessed data, so that the data to be processed can retain all the characteristics of the data, thereby ensuring the accuracy of attribute information recognition and enhancing the stability of data recognition.
[0058] Optionally, this embodiment is explained by detecting abnormal data in big data. Therefore, in this embodiment, the data to be processed may be some test data in the big data.
[0059] S202: Process the data to be processed based on a preset sample set, and determine at least one similarity value between the data to be processed and at least one sample data included in the sample set.
[0060] Optionally, a sample set is preset in the processing device, and the sample set may include at least one sample data. When the sample set includes at least two sample data, the at least two sample data are different sample data.
[0061] In this embodiment, the sample data in the sample set are arranged in chronological order of collection. The length of each sample data depends on the respective access duration and retrieval content, and each sample data carries annotation information, which mainly includes categories and labels, which are manually annotated.
[0062] Among them, the category is mainly used to distinguish which field the sample data belongs to; the label is mainly used to identify whether the sample data is normal data.
[0063] Exemplarily, Table 1 is the format of each sample data in the sample set. As shown in Table 1, the category and label of the sample data can be determined based on the source and / or content of the sample data. Optionally, the label can be represented by 0 and 1. If the label of the sample data is 1, it means that the sample data is normal; if the label of the sample data is 0, it means that the sample data is abnormal.
[0064] For example, the content of sample data P1 includes "math", "english", "physics", "chemistry", "exam", "Chinese", etc. At this time, the category is student. Since the spelling of each object in sample data P1 is correct, the label is 1; the content of sample data P2 includes "doc", "laa", "la", "al", "allal", "hhhhh", etc., and its corresponding category is hospital. Since there are misspelled objects in sample data P2, the label is 0; similarly, sample data P n The contents include "python", "c language", "java", "php", "matlab", and their corresponding category is the program "programmer". m If all objects in are spelled correctly, the label is 1.
[0065] Table 1 Format of sample data in the sample set
[0066]
[0067] Among them, P1 represents the first sample data, P2 represents the second sample data, and P m Represents the mth sample data, and so on.
[0068] Accordingly, in the embodiment of the present disclosure, each similarity value between the data to be processed and each sample data in the sample set may be calculated based on the content of the data to be processed and the content of each sample data in the sample set.
[0069] Exemplarily, the content of the data can be understood as the objects included in the data. Therefore, based on the objects included in the data to be processed and the objects included in each sample data in the sample set, the similarity values between the data to be processed and each sample data in the sample set can be calculated.
[0070] S203: Based on the at least one similarity value, determine target matching data in at least one sample data.
[0071] Optionally, after performing similarity analysis on the data to be processed and at least one sample data in the sample set to obtain at least one similarity value, the at least one similarity value may be analyzed according to a set rule to determine target matching data in at least one sample data corresponding to the at least one similarity value.
[0072] Exemplarily, the setting rule may be to determine the maximum similarity value from the at least one similarity value, and then determine the sample data corresponding to the maximum similarity value as the target matching data. The larger the similarity value, the more similar the features of the two data are.
[0073] S204: Determine attribute information of the data to be processed according to the labeling information of the target matching data.
[0074] In this embodiment, since the target matching data is sample data screened and processed from the sample set, it also carries annotation information, and therefore, the attribute information of the data to be processed can be determined based on the annotation information of the target data. For example, the same category and label as the target matching data can be given to the data to be processed.
[0075] Exemplarily, the category of the data to be processed is determined according to the category of the target matching data; and whether the data to be processed is abnormal is determined according to whether the target matching data is abnormal.
[0076] It can be understood that the technical solution of the present invention can collect samples based on time series to obtain sample sets and data to be processed without any processing, and all the characteristics of the original data are directly maintained. In subsequent data processing, for example, when monitoring outliers, it is only necessary to input the original access sequence of the data to be processed, and by calculating the similarity value between the data to be processed and each sample data in the sample set, the relevant samples can be quickly and accurately matched, thereby determining the category of the data to be processed and attribute information such as the anomaly monitoring results.
[0077] In the embodiment of the present disclosure, by acquiring the data to be processed, the data to be processed is processed based on the preset sample set, at least one similarity value between the data to be processed and at least one sample data included in the sample set is determined, and then based on the at least one similarity value, the target matching data is determined in the at least one sample data, and finally the attribute information of the data to be processed is determined according to the annotation information of the target matching data. This technical solution can accurately determine the target matching data, lay a foundation for the subsequent determination of the attribute information of the data to be processed, thereby improving the data processing efficiency and data processing effect.
[0078] In order to enable readers to more deeply understand the implementation principle of the present disclosure, the following Figures 3 to 5 right Figure 2 The illustrated embodiment is further refined.
[0079] Optional, in the above Figure 2 Based on the embodiment shown, Figure 3 FIG. 1 is a flow chart of a data processing method provided by the second embodiment of the present disclosure. Figure 3 As shown, in an embodiment of the present disclosure, the above S202 may include the following steps:
[0080] S301 , numerically process the data to be processed and at least one sample data included in the sample set to obtain a sequence of numerical values to be processed corresponding to the data to be processed and at least one sample numerical sequence corresponding to the at least one sample data.
[0081] In this embodiment, in order to quantify the relationship between the data to be processed and each sample data in the sample set, and considering the time series characteristics, the data to be processed and at least one sample data included in the sample set need to be digitized respectively, wherein the time series characteristics can reflect the importance of the object in each sample data.
[0082] Optionally, in the embodiments of the present disclosure, a numerical conversion tool may be used to perform numerical processing, for example, an open source string (string) to integer (int) tool may be used to perform numerical processing.
[0083] For example, assume that the data to be processed is represented by T = {w1, w2, ..., w n}, the sample set is represented by P = {P1, P2, ..., P m}, then the data to be processed T = {w1, w2, ..., w n The result of the digitization is a test value sequence. Sample set P = {P1, P2, ..., P m The result of digitizing each sample data in} is at least one sample value sequence
[0084] It is understandable that, in practical applications, the number of objects included in the data to be processed and the number of objects included in each sample data in the sample set may be the same as or different from each other, and this embodiment does not limit this.
[0085] S302: Calculate at least one distance between the to-be-processed numerical sequence and at least one sample numerical sequence.
[0086] Exemplarily, in this embodiment, in order to compare the similarity between the data to be processed and each sample data in the above sample set, the distance between them may be calculated, and the smaller the distance, the higher the similarity.
[0087] Optional, assuming is the i-th sample value sequence in at least one sample value sequence, and correspondingly, the value sequence to be processed and The distance function between The distance function depends on the actual distance metric used. In the dynamic time warping (DTW) algorithm, Euclidean distance is usually used.
[0088] For example, assuming that the DTW algorithm is used to calculate and The similarity value of each sample numerical sequence in can obtain at least one Euclidean distance, for example, using d = {d1, d2, ..., d m}express.
[0089] S303: Determine at least one similarity value between the data to be processed and at least one sample data according to the at least one distance.
[0090] In this embodiment, the similarity value between the data to be processed and at least one sample data is characterized by at least one distance between the to-be-processed numerical sequence and at least one sample numerical sequence, but the smaller the distance, the higher the similarity.
[0091] It can be understood that after determining the above-mentioned at least one distance, on the one hand, the relationship between the distance and the similarity can be used to characterize the magnitude of the similarity; on the other hand, at least one similarity value between the data to be processed and at least one sample data can be determined by a certain rule transformation. For example, the relationship between the similarity Sim and the distance d can be expressed by the following formula: Sim = 1 / d. Therefore, the above-mentioned at least one distance d = {d1, d2, ..., d m} corresponds to at least one similarity Sim = {1 / d1, 1 / d2, ..., 1 / d m}.
[0092] Optionally, in the embodiments of the present disclosure, refer to Figure 3 As shown, the above S203 may include the following steps:
[0093] S304: Determine a maximum similarity value among at least one similarity value.
[0094] Exemplarily, after determining at least one similarity value, a maximum similarity value may be determined based on a set rule.
[0095] For example, the setting rule may be to first sort the at least one similarity value in order of magnitude, and then determine the maximum similarity value therein. For another example, the setting rule may also be to compare two adjacent similarity values in order according to the determination order of the at least one similarity value, and after all similarity values are compared, determine the maximum similarity value therefrom.
[0096] It is understandable that the embodiments of the present disclosure do not limit the specific method of determining the maximum similarity, which can be set according to actual needs and will not be described in detail here.
[0097] S305 , determining whether the maximum similarity value is greater than or equal to a preset similarity threshold; if so, executing S306 ; if not, executing S307 to S309 .
[0098] In this embodiment, a similarity threshold is preset in the processing device. In order to determine the target matching data that meets the requirements, the determined maximum similarity can be first compared with the preset similarity threshold to determine whether the maximum similarity value is greater than or equal to the preset similarity threshold, and then subsequent operations are performed based on the judgment result.
[0099] S306: Determine the sample data corresponding to the maximum similarity value as the target matching data.
[0100] As an example, in response to the maximum similarity value being greater than or equal to the similarity threshold, the sample data corresponding to the maximum similarity value is determined to be the target matching data.
[0101] For example, when distance is used to represent the similarity value, it is assumed that, given a minimum distance threshold ε, if min(d)=min{d1, d2, ..., d m} = d p ≤ε, that is, if the minimum distance d among at least one of the above distances p If the distance is less than or equal to the set minimum distance threshold ε, the feature of the data to be processed is determined to be the minimum distance d p The characteristics of the corresponding sample data are similar, and the minimum distance d p The corresponding sample data is the sample data corresponding to the maximum similarity, so the minimum distance d p The corresponding sample data is determined as the target matching data.
[0102] For example, assuming a maximum similarity threshold δ is given, if That is, if the maximum similarity 1 / d among the above at least one similarity value p If it is greater than or equal to the set maximum similarity threshold δ, then the feature of the data to be processed is determined to have the maximum similarity 1 / d p The features of the corresponding sample data are similar, so the maximum similarity can be 1 / d p The corresponding sample data is determined as the target matching data.
[0103] In this step, by comparing the similarity between the data to be processed and each sample data in the sample set, the target matching data whose similarity meets the requirements can be accurately determined, so that the data processing process can be automatically completed, thereby improving the data processing efficiency and data processing accuracy.
[0104] S307: Output processing prompt.
[0105] As another example, in response to the maximum similarity value being less than a similarity threshold, a processing prompt is output.
[0106] Exemplarily, when the maximum similarity between each sample data in the sample set and the data to be processed is less than a set similarity threshold, a processing prompt can be output at this time. For example, when the processing device has a human-computer interaction interface, the processing prompt is output through the human-computer interaction interface so that the above-mentioned data to be processed can be manually processed.
[0107] It is understandable that when the maximum similarity value is less than the similarity threshold, if the processing device has a voice prompt function, a processing prompt can also be output through voice, so that the processing personnel can be informed of the processing task in time.
[0108] S308: Obtain the processing result and annotation information made by the processing personnel in response to the processing prompt.
[0109] Exemplarily, after receiving the processing instruction, the processing personnel can manually analyze the attribute information to be processed and provide the processing result.
[0110] Optionally, in order to improve the diversity of sample data in the sample set, the processing personnel may also mark the data to be processed after making processing results on the data to be processed, for example, marking the category and / or label of the data to be processed so that the data to be processed carries the marking information.
[0111] S309: Add the data to be processed that carries the annotation information to the sample set.
[0112] Optionally, after obtaining the data to be processed carrying the labeled information, in order to enable the processing device to complete the data processing process for more data, the data to be processed carrying the labeled information may be added to the sample set to enrich the sample data in the sample set.
[0113] In the embodiments of the present disclosure, for the unprocessed data that does not meet the similarity value conditions, it is added to the sample set after manual processing, information labeling and other operations, which can improve the sample types in the sample set and lay the foundation for subsequent closed-loop automatic processing.
[0114] Based on the above embodiments, Figure 4 FIG. 1 is a flow chart of a data processing method provided by the third embodiment of the present disclosure. Figure 4 As shown, in an embodiment of the present disclosure, before the above S202, the data processing method may further include the following steps:
[0115] S401 : Determine the matching degree between the data to be processed and each data based on the objects included in the data to be processed and the objects included in each data in the preset database.
[0116] Exemplarily, there may be a preset database in the processing device, in which there are multiple pieces of data, each piece of data having labeling information.
[0117] For example, suppose the pre-built database S consists of different data streams S i , i = 1, 2, 3, ..., n, expressed as: S = {S1, S2, ..., S n The data of the preset database is collected in chronological order. The different data lengths depend on the respective access durations and retrieval contents. The collected data sequences are manually labeled. The labeled contents mainly include categories and labels. Categories are mainly used to distinguish which field the data belongs to. Labels are used to identify whether the data is normal. 1 indicates that the data is normal, and 0 indicates that it is abnormal.
[0118] Generally, for a piece of data to be processed, the preset database can be searched for target matching data similar to the data to be processed. However, when the amount of data in the preset database is large, directly searching in the preset database may result in a long query time. Therefore, in order to shorten the query time and reduce the amount of subsequent calculations, in this embodiment, the matching degree between the data to be processed and each data can be calculated based on the objects included in the data to be processed and the objects included in each data in the preset database, and then a sample set with a higher matching degree can be determined.
[0119] In the embodiments of the present disclosure, the objects included in the data can be interpreted as the values of the data in various dimensions. For example, the data to be processed T = {w1, w2, ..., w n The objects included are w1, w2, ..., w n ; Similarly, the objects included in each data in the preset database can be interpreted in a manner that is not described here.
[0120] In a possible implementation of this embodiment, step S401 may be implemented by the following steps:
[0121] A1. Based on the objects included in the data to be processed and the objects included in each data in the preset database, determine the number of the same objects that each data has with the data to be processed.
[0122] A2. Determine the ratios between the number of identical objects and the total number of objects in the data to be processed;
[0123] A3. Based on each ratio, determine the matching degree between the data to be processed and each data.
[0124] In this possible implementation, the data to be processed T = {w1, w2, ..., w n} and the preset database S = {S1, S2, ..., S n}The i-th data S i The matching degree calculation formula is shown in formula (1):
[0125]
[0126] Among them, ρ(T, S i ) represents the data to be processed T and data S i The matching degree, C(·) represents the count, T∩S i Indicates T and S i The intersection of T and S i The same object in C(T∩S i ) represents the data to be processed T and data S i The number of identical objects C(T) has represents the total number of objects included in the data T to be processed.
[0127] Similarly, based on the above formula (1), the matching degree ρ between the data to be processed T and each data in the preset database S can be calculated as follows: n}.
[0128] In another possible implementation of this embodiment, step S401 may be implemented by the following steps:
[0129] B1. Based on the objects included in the data to be processed and the objects included in each data in the preset database, determine the number of objects that are the same between each data and the data to be processed;
[0130] B2. Determine the matching degree between the data to be processed and each data based on the number of identical objects between each data and the data to be processed.
[0131] In this possible implementation, the data to be processed T = {w1, w2, ..., w n} and the preset database S = {S1, S2, ..., S n}The i-th data S i The matching degree calculation formula is shown in formula (2):
[0132] ρ(T,S i )=C(T∩S i ) Formula (2)
[0133] Among them, ρ(T, S i ) represents the data to be processed T and data S i The matching degree, C(·) represents the count, T∩S i Indicates T and S i The intersection of T and S i The same object in C(T∩S i ) represents the data to be processed T and data S i The number of identical objects.
[0134] Similarly, based on the above formula (2), the matching degree ρ between the data to be processed T and each data in the preset database S can be calculated as follows: n}.
[0135] It is understandable that the embodiments of the present disclosure do not limit the actual calculation method of the matching degree. The matching degree method can be designed according to the actual application situation, which will not be elaborated here.
[0136] S402: Determine a sample set from a preset database, wherein the matching degree between each sample data included in the sample set and the data to be processed is greater than or equal to a matching degree threshold.
[0137] In this step, the matching degree between the data to be processed and each data in the preset database can be compared with a preset matching degree threshold, and at least one data with a matching degree greater than or equal to the matching degree threshold can be screened out from the preset database to form the above-mentioned sample set.
[0138] Exemplarily, the matching degree ρ between the data to be processed T and each data in the preset database S is obtained = {ρ1, ρ2, ..., ρ n}, we can find the matching degree set whose value in ρ is greater than the matching degree threshold θ, for example, {ρ l , ρ k , ..., ρ m}, and then find the data set corresponding to the matching set according to the subscript index of the matching set {S l , S k , ..., S m}, thus forming a sample set And S l , S k , ..., S m exist They are also arranged in chronological order.
[0139] It can be understood that the sample set in this embodiment In fact, the above Figure 2 and Figure 3 In the embodiment shown, the sample set P = {P1, P2, ..., P m}, that is, S l For P1, S k For P2, S m P m wait.
[0140] It is worth noting that in this step, for the matching degree determined based on the above formula (1), the preset matching degree threshold θ∈[0,1], the larger the θ value, the greater the matching possibility between the data to be processed and the data, and the specific value of θ can be determined according to the total amount of data in the preset database, for example, θ=0.5, which is not limited here.
[0141] In this step, for the matching degree determined based on the above formula (2), the preset matching degree threshold θ is an integer greater than 1. The larger the value of θ, the greater the matching possibility between the data to be processed and the data. The specific value of θ can also be determined according to the total amount of data in the preset database, which is not limited here.
[0142] In the embodiment of the present disclosure, based on the objects included in the data to be processed and the objects included in each data in the preset database, the matching degree between the data to be processed and each data is determined, and a sample set is determined from the preset database, and the matching degree between each sample data included in the sample set and the data to be processed is greater than or equal to the matching degree threshold. This technical solution can reduce the amount of data to be queried, shorten the query time, reduce the subsequent calculation amount, and improve the processing efficiency while ensuring the processing accuracy.
[0143] Based on the technical solutions of the above embodiments, the following Figure 5 The illustrated embodiment provides an embodiment for simply introducing the complete implementation scheme of the present disclosure. Figure 5FIG. 4 is a flow chart of a data processing method provided by the fourth embodiment of the present disclosure. Figure 5 As shown, the data processing method may include the following steps:
[0144] S501. Generate a sample database using data collected in chronological order.
[0145] Optionally, the sample database is the above Figure 4 The preset database in the illustrated embodiment.
[0146] S502: Acquire a test data stream.
[0147] Optionally, the test data stream is the data to be processed in the above embodiments.
[0148] S503: Perform matching calculation on the test data and each data in the sample database, and select a minimum matching sample set from the sample database.
[0149] S504 , performing numerical conversion on the minimum matching sample set in S503 and the data to be processed in S502 respectively, to obtain a minimum matching sample numerical set and a test numerical sequence.
[0150] S505: Calculate similarity for the minimum matching sample value set and the test value sequence.
[0151] S506: Calculate the maximum similarity value.
[0152] Optionally, the maximum similarity value is calculated based on the DTW algorithm.
[0153] S507, based on the maximum similarity value and the similarity threshold, determine whether there is target matching data in the minimum matching sample data set; if so, go to S508; if not, execute S509;
[0154] S508, outputting a processing result based on the target matching data;
[0155] S509 , obtaining the test data stream after manual annotation, and adding it to the sample database of S501 .
[0156] Exemplarily, the scheme of the embodiment of the present disclosure involves the creation of a sample database, performing matching calculations on the test data and the sample database to obtain a minimum sample set, and then, through numerical conversion, converting the test data and the minimum sample set into numerical types respectively, calculating the similarity values between each sub-sample sequence in the sample set and the test numerical sequence respectively, and finally finding the maximum value of all similarity values. Through the maximum value, it is possible to find out which sample data or sample data in the sample database match the test data, thereby inferring the attribute information of the test data. If the best match is not found (the similarity value does not meet the preset threshold), the test data stream can be added to the sample database as a new sample type for subsequent use.
[0157] It can be understood that the specific implementation of each step in this embodiment can refer to the records in the above embodiments and will not be repeated here.
[0158] Based on the above embodiments, it can be seen that the embodiments of the present disclosure propose a method for processing high-dimensional data, which overcomes the limitation of requiring uniform sample feature dimensions, solves the problem of long sample database production cycle and difficulty in manually extracting features, and maintains all the features of the original data, with the characteristics of high recognition accuracy and strong stability. In addition, in this embodiment, when constructing a sample database, it is only necessary to collect samples according to the time series. In order to speed up the calculation efficiency, a matching degree calculation method is also provided, which can quickly match similar subsets. Furthermore, by using dynamic regularization values for calculation, the similarity values of two sequences of different lengths can be effectively calculated. As the test continues, the sample database will continue to improve, forming a closed-loop abnormality monitoring, laying the foundation for obtaining accurate processing results in the future, and improving the degree of automation.
[0159] Figure 6 Schematic diagram of a data processing device provided by an embodiment of the present disclosure. The data processing device provided by this embodiment may be an electronic device or a device in an electronic device. Figure 6 As shown, the data processing device 600 provided in the embodiment of the present disclosure may include:
[0160] An acquisition unit 601 is used to acquire data to be processed;
[0161] The processing unit 602 is used to process the data to be processed based on a preset sample set, and determine at least one similarity value between the data to be processed and at least one sample data included in the sample set;
[0162] A matching determination unit 603, configured to determine target matching data in the at least one sample data based on the at least one similarity value;
[0163] The attribute determination unit 604 is used to determine the attribute information of the data to be processed according to the label information of the target matching data.
[0164] In a possible implementation of the embodiment of the present disclosure, the processing unit 602 includes:
[0165] A first processing module, configured to perform numerical processing on the data to be processed and at least one sample data included in the sample set, to obtain a sequence of numerical values to be processed corresponding to the data to be processed and at least one sample numerical sequence corresponding to the at least one sample data;
[0166] A second processing module, configured to calculate at least one distance between the to-be-processed numerical sequence and the at least one sample numerical sequence;
[0167] The third processing module is used to determine at least one similarity value between the data to be processed and the at least one sample data according to the at least one distance.
[0168] In a possible implementation of the embodiment of the present disclosure, the matching determination unit 603 includes:
[0169] A first determining module, configured to determine a maximum similarity value among the at least one similarity value;
[0170] A determination module, used to determine whether the maximum similarity value is greater than or equal to a preset similarity threshold;
[0171] The second determination module is configured to determine, in response to the maximum similarity value being greater than or equal to the similarity threshold, that the sample data corresponding to the maximum similarity value is the target matching data.
[0172] In this possible implementation, the data processing device further includes:
[0173] an output unit (not shown), configured to output a processing prompt in response to the maximum similarity value being less than the similarity threshold;
[0174] A capture unit (not shown), used to obtain the processing results and annotation information made by the processing personnel in response to the processing prompt;
[0175] An adding unit (not shown) is used to add the to-be-processed data carrying the annotation information to the sample set.
[0176] In a possible implementation of the embodiment of the present disclosure, the device further includes:
[0177] A matching degree determining unit (not shown), configured to determine a matching degree between the data to be processed and each data based on an object included in the data to be processed and an object included in each data in a preset database;
[0178] The sample set determination unit (not shown) is used to determine the sample set from the preset database, wherein the matching degree between each sample data included in the sample set and the data to be processed is greater than or equal to the matching degree threshold.
[0179] In a possible implementation of the embodiment of the present disclosure, the matching degree determination unit includes:
[0180] A first quantity counting module, configured to determine the number of identical objects between each data and the data to be processed based on the objects included in the data to be processed and the objects included in each data in the preset database;
[0181] A ratio determination module, used to determine each ratio between the number of each identical object and the total number of objects included in the data to be processed;
[0182] The first matching degree determination module is used to determine the matching degree between the data to be processed and each data based on each ratio.
[0183] In a possible implementation of the embodiment of the present disclosure, the matching degree determination unit includes:
[0184] A second quantity counting module, used for determining the number of identical objects between each data and the data to be processed based on the objects included in the data to be processed and the objects included in each data in the preset database;
[0185] The second matching degree determination module is used to determine the matching degree between the data to be processed and each data based on the number of identical objects between each data and the data to be processed.
[0186] The data processing device provided in this embodiment can be used to execute the data processing method in any of the above method embodiments. Its implementation principle and technical effects are similar and will not be elaborated here.
[0187] According to an embodiment of the present disclosure, the present disclosure also provides an electronic device, a readable storage medium and a computer program product.
[0188] According to an embodiment of the present disclosure, the present disclosure also provides a computer program product, which includes: a computer program, the computer program is stored in a readable storage medium, at least one processor of an electronic device can read the computer program from the readable storage medium, and at least one processor executes the computer program so that the electronic device executes the solution provided by any of the above embodiments.
[0189] Figure 7is a schematic block diagram of an example electronic device used to implement an embodiment of the present disclosure. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device may also represent various forms of mobile devices, such as personal digital processing, cellular phones, smart phones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present disclosure described and / or claimed herein.
[0190] like Figure 7 As shown, the device 700 includes a computing unit 701, which can perform various appropriate actions and processes according to a computer program stored in a read-only memory (ROM) 702 or a computer program loaded from a storage unit 708 into a random access memory (RAM) 703. In the RAM 703, various programs and data required for the operation of the device 700 can also be stored. The computing unit 701, the ROM 702, and the RAM 703 are connected to each other via a bus 704. An input / output (I / O) interface 705 is also connected to the bus 704.
[0191] A number of components in the device 700 are connected to the I / O interface 705, including: an input unit 706, such as a keyboard, a mouse, etc.; an output unit 707, such as various types of displays, speakers, etc.; a storage unit 708, such as a disk, an optical disk, etc.; and a communication unit 709, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 709 allows the device 700 to exchange information / data with other devices through a computer network such as the Internet and / or various telecommunication networks.
[0192] The computing unit 701 may be a variety of general and / or special processing components with processing and computing capabilities. Some examples of the computing unit 701 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various dedicated artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, digital signal processors (DSPs), and any appropriate processors, controllers, microcontrollers, etc. The computing unit 701 performs the various methods and processes described above, for example, data processing methods. For example, in some embodiments, the data processing method may be implemented as a computer software program, which is tangibly contained in a machine-readable medium, such as a storage unit 708. In some embodiments, part or all of the computer program may be loaded and / or installed on the device 700 via ROM 702 and / or communication unit 709. When the computer program is loaded into RAM 703 and executed by the computing unit 701, one or more steps of the data processing method described above may be performed. Alternatively, in other embodiments, the computing unit 701 may be configured to perform the data processing method in any other appropriate manner (e.g., by means of firmware).
[0193] Various implementations of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), complex programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various implementations can include: being implemented in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which can be a special purpose or general purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.
[0194] The program code for implementing the method of the present disclosure may be written in any combination of one or more programming languages. These program codes may be provided to a processor or controller of a general-purpose computer, a special-purpose computer, or other programmable data processing device, so that the program code, when executed by the processor or controller, enables the functions / operations specified in the flow chart and / or block diagram to be implemented. The program code may be executed entirely on the machine, partially on the machine, partially on the machine and partially on a remote machine as a stand-alone software package, or entirely on a remote machine or server.
[0195] In the context of the present disclosure, a machine-readable medium may be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, device, or equipment. A machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or equipment, or any suitable combination of the foregoing. A more specific example of a machine-readable storage medium may include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.
[0196] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user can provide input to the computer. Other types of devices can also be used to provide interaction with the user; for example, the feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including acoustic input, voice input, or tactile input).
[0197] The systems and techniques described herein may be implemented in a computing system that includes back-end components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes front-end components (e.g., a user computer with a graphical user interface or a web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such back-end components, middleware components, or front-end components. The components of the system may be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: a local area network (LAN), a wide area network (WAN), and the Internet.
[0198] A computer system may include a client and a server. The client and the server are generally remote from each other and usually interact through a communication network. The relationship between the client and the server is generated by computer programs that run on the corresponding computers and have a client-server relationship with each other. The server may be a cloud server, also known as a cloud computing server or a cloud host, which is a host product in the cloud computing service system to solve the defects of difficult management and weak business scalability in traditional physical hosts and VPS services ("Virtual Private Server", or "VPS" for short). The server may also be a server for a distributed system, or a server combined with a blockchain.
[0199] It should be understood that the various forms of processes shown above can be used to reorder, add or delete steps. For example, the steps recorded in this disclosure can be executed in parallel, sequentially or in different orders, as long as the desired results of the technical solutions disclosed in this disclosure can be achieved, and this document does not limit this.
[0200] The above specific implementations do not constitute a limitation on the protection scope of the present disclosure. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions can be made according to design requirements and other factors. Any modification, equivalent substitution and improvement made within the spirit and principle of the present disclosure shall be included in the protection scope of the present disclosure.
Claims
1. A data processing method, comprising: Get the data to be processed; Processing the data to be processed based on a preset sample set, determining at least one similarity value between the data to be processed and at least one sample data included in the sample set; each sample data in the sample set has labeling information, and the labeling information includes a category and a label; Based on the at least one similarity value, determining target matching data in the at least one sample data; Determining attribute information of the data to be processed according to the labeling information of the target matching data; The processing of the data to be processed based on a preset sample set to determine at least one similarity value between the data to be processed and at least one sample data included in the sample set includes: Using a numerical conversion tool to perform numerical processing on the data to be processed and at least one sample data included in the sample set, to obtain a sequence of numerical values to be processed corresponding to the data to be processed and at least one sample numerical sequence corresponding to the at least one sample data; Calculating at least one distance between the to-be-processed numerical sequence and the at least one sample numerical sequence; At least one similarity value between the data to be processed and the at least one sample data is determined according to the at least one distance.
2. The method according to claim 1, wherein: The determining target matching data in the at least one sample data based on the at least one similarity value includes: Determining a maximum similarity value among the at least one similarity value; Determining whether the maximum similarity value is greater than or equal to a preset similarity threshold; In response to the maximum similarity value being greater than or equal to the similarity threshold, the sample data corresponding to the maximum similarity value is determined as target matching data.
3. The method according to claim 2, further comprising: In response to the maximum similarity value being less than the similarity threshold, outputting a processing prompt; Obtaining the processing results and annotation information made by the processing personnel in response to the processing prompt; The data to be processed carrying the annotation information is added to the sample set.
4. The method according to any one of claims 1 to 3, further comprising: Determining the matching degree between the data to be processed and each data based on the objects included in the data to be processed and the objects included in each data in the preset database; The sample set is determined from the preset database, wherein the matching degree between each sample data included in the sample set and the data to be processed is greater than or equal to a matching degree threshold.
5. The method according to claim 4, wherein: The determining, based on the objects included in the data to be processed and the objects included in each data in the preset database, the matching degree between the data to be processed and each data includes: Based on the objects included in the data to be processed and the objects included in each data in the preset database, determining the number of the same objects between each data and the data to be processed; Determining each ratio between the number of each identical object and the total number of objects included in the data to be processed; Based on each ratio, a matching degree between the data to be processed and each data is determined.
6. The method according to claim 4, wherein: The determining, based on the objects included in the data to be processed and the objects included in each data in the preset database, the matching degree between the data to be processed and each data includes: Based on the objects included in the data to be processed and the objects included in each data in the preset database, determining the number of objects that are the same between each data and the data to be processed; Based on the number of identical objects between each data and the data to be processed, a matching degree between the data to be processed and each data is determined.
7. A data processing device, comprising: An acquisition unit, used for acquiring data to be processed; A processing unit, configured to process the data to be processed based on a preset sample set, and determine at least one similarity value between the data to be processed and at least one sample data included in the sample set; each sample data in the sample set has labeling information, and the labeling information includes a category and a label; a matching determination unit, configured to determine target matching data in the at least one sample data based on the at least one similarity value; An attribute determination unit, configured to determine attribute information of the data to be processed according to the label information of the target matching data; The processing unit comprises: A first processing module, configured to perform numerical processing on the data to be processed and at least one sample data included in the sample set using a numerical conversion tool to obtain a numerical sequence to be processed corresponding to the data to be processed and at least one sample numerical sequence corresponding to the at least one sample data; A second processing module, configured to calculate at least one distance between the to-be-processed numerical sequence and the at least one sample numerical sequence; The third processing module is used to determine at least one similarity value between the data to be processed and the at least one sample data according to the at least one distance.
8. The device according to claim 7, wherein: The matching determination unit includes: A first determining module, configured to determine a maximum similarity value among the at least one similarity value; A determination module, used to determine whether the maximum similarity value is greater than or equal to a preset similarity threshold; The second determination module is configured to determine, in response to the maximum similarity value being greater than or equal to the similarity threshold, that the sample data corresponding to the maximum similarity value is the target matching data.
9. The apparatus according to claim 8, further comprising: an output unit, configured to output a processing prompt in response to the maximum similarity value being less than the similarity threshold; A capture unit, used to obtain the processing results and annotation information made by the processing personnel in response to the processing prompt; An adding unit is used to add the data to be processed carrying the annotation information to the sample set.
10. The device according to any one of claims 7 to 9, further comprising: A matching degree determining unit, configured to determine a matching degree between the data to be processed and each data based on an object included in the data to be processed and an object included in each data in a preset database; The sample set determining unit is used to determine the sample set from the preset database, wherein the matching degree between each sample data included in the sample set and the data to be processed is greater than or equal to a matching degree threshold.
11. The device according to claim 10, wherein: The matching degree determination unit includes: A first quantity counting module, configured to determine the number of identical objects between each data and the data to be processed based on the objects included in the data to be processed and the objects included in each data in the preset database; A ratio determination module, used to determine each ratio between the number of each identical object and the total number of objects included in the data to be processed; The first matching degree determination module is used to determine the matching degree between the data to be processed and each data based on each ratio.
12. The device according to claim 10, wherein: The matching degree determination unit includes: A second quantity counting module, used for determining the number of identical objects between each data and the data to be processed based on the objects included in the data to be processed and the objects included in each data in the preset database; The second matching degree determination module is used to determine the matching degree between the data to be processed and each data based on the number of identical objects between each data and the data to be processed.
13. An electronic device, comprising: at least one processor; as well as a memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the method according to any one of claims 1 to 6.
14. A non-transitory computer-readable storage medium storing computer instructions, wherein: The computer instructions are used to cause the computer to execute the method according to any one of claims 1 to 6.
15. A computer program product, comprising a computer program, which implements the steps of the method according to any one of claims 1 to 6 when executed by a processor.
Citation Information
Patent Citations
Label determination method, device and system
CN112446399A
Lyric recognition method and device thereof, equipment, storage medium and product
CN113658594A