Confusion matrix scoring method and device based on entropy coding

Abstract

The invention provides a confusion matrix scoring method and device based on entropy coding, and relates to the technical field of machine learning. The method comprises the following steps: acquiring an original confusion matrix; setting an element in a main diagonal line of the original confusion matrix to be zero to obtain an error matrix; binary entropy coding is carried out on terms of the error matrix in an entropy coding mode, and a coding matrix is obtained; calculating the code length of each code in the coding matrix according to the coding matrix to obtain a code length matrix; multiplying each item of the code length matrix by each item of the error matrix to obtain an average code length matrix; summing each item of the average code length matrix to obtain a total average code length; and the value of the total average code length is used as a scoring index. According to the invention, the error rate and the dispersity of error conditions can be reflected.

Description

Technical Field

[0001] The present invention relates to the technical field of machine learning, and particularly to a confusion matrix scoring method and device based on entropy coding. Background Art

[0002] Situations where the obtained results are inconsistent with the correct answers occur in the fields of teaching, communication, and machine learning; according to the above situations, a confusion matrix is currently used to summarize various error situations. There are two dimensions in the confusion matrix; among them, the first dimension is the correct answer, which has multiple possible values; among them, the second dimension is the obtained result, which has multiple possible values. The confusion matrix is a matrix with many elements, and it is difficult to compare two different matrices, while comparison is required in many scenarios; for example, in a teaching scenario, to evaluate which of two students has better grades; in order to obtain an exact value for evaluation through the confusion matrix, concepts such as accuracy, precision, recall, and F1-Score are currently used for evaluation.

[0003] Accuracy, precision, recall, and F1-Score are all obtained by simply multiplying and dividing ratios after counting the number of correct and incorrect ones. However, in many cases, the error rate does not evaluate the performance situation well enough. For example, in the evaluation of Putonghua proficiency, some candidates in dialect areas have inherent pronunciation habits. According to the Putonghua standard, their error rate is very high, but in fact, it does not affect others' understanding of what they express. And the Putonghua error rate of foreigners may not be high, but the mistakes are very irregular and easy to cause misunderstandings. Therefore, simply summing up various correct rates on the confusion matrix cannot truly reflect the appearance between different matrices, and there should be more effective indicators. Summary of the Invention

[0004] In order to solve the technical problems that the existing technologies cannot reflect the error rate when using indicators such as accuracy, precision, recall, and F1-Score for evaluation, embodiments of the present invention provide a confusion matrix scoring method and device based on entropy coding. The technical solutions are as follows:

[0005] On the one hand, a confusion matrix scoring method based on entropy coding is provided. This method is implemented by a confusion matrix scoring device based on entropy coding, and the method includes:

[0006] S1. Obtain the original confusion matrix; set the elements in the main diagonal of the original confusion matrix to zero to obtain an error matrix;

[0007] S2. Perform binary entropy coding on the items of the error matrix by using an entropy coding method to obtain a coding matrix;

[0008] S3. Calculate the code length of each code in the coding matrix according to the coding matrix to obtain a code length matrix;

[0009] S4. Multiply each term of the code length matrix by each term of the error matrix to obtain an average code length matrix;

[0010] S5. Sum each term of the average code length matrix to obtain the total average code length; Use the value of the total average code length as the scoring index.

[0011] Optionally, before the step of performing binary entropy coding on the terms of the error matrix by using entropy coding in S2 to obtain a coding matrix, it further includes:

[0012] Adjust the terms of the error matrix by using the adjusted weight method to obtain a new error matrix.

[0013] Optionally, before the step of calculating the code length of each code in the coding matrix according to the coding matrix in S3 to obtain a code length matrix, it further includes:

[0014] Modify the terms of the original code length matrix by using the method of adjusting the proportion to obtain a new code length matrix.

[0015] Optionally, after the step of summing each term of the average code length matrix in S5 to obtain the total average code length, it further includes:

[0016] Transform the total average code length by using the method of transformation calculation, and use the value obtained by the transformation as the scoring index.

[0017] Optionally, the step of performing binary entropy coding on the terms of the error matrix by using entropy coding in S2 to obtain a coding matrix includes:

[0018] Perform binary entropy coding on the terms of the error matrix by using entropy coding, without deleting correct elements, and calculate according to the calculation method with the shortest code length for the correct part by default to obtain a coding matrix; or

[0019] By adjusting the calculation method of each proportion, by adjusting each ratio, calculate entropy coding to obtain a coding matrix; or

[0020] After calculating entropy coding by adjusting the calculation method of each proportion, obtain a coding matrix by adjusting the weights.

[0021] Optionally, in the process of obtaining the coding matrix, it further includes: performing coding in the manner of independently coding each column or each row of the error matrix to obtain a coding matrix.

[0022] Optionally, the step of performing binary entropy coding on the terms of the error matrix by using entropy coding in S2 to obtain a coding matrix includes:

[0023] Entropy encode the terms of the error matrix, with each term being a symbol participating in the encoding. The occurrence probability of the symbol is the value of the corresponding term in the error matrix. Use the entropy encoding method to perform binary encoding on the symbols to obtain an encoding matrix.

[0024] On the other hand, a confusion matrix scoring device based on entropy encoding is provided. This device is applied to the confusion matrix scoring method based on entropy encoding. The device includes:

[0025] A first acquisition unit for acquiring the original confusion matrix; setting the elements in the main diagonal of the original confusion matrix to zero to obtain an error matrix;

[0026] A second acquisition unit for performing binary entropy encoding on the terms of the error matrix using the entropy encoding method to obtain an encoding matrix;

[0027] A third acquisition unit for calculating the code length of each encoding in the encoding matrix according to the encoding matrix to obtain a code length matrix;

[0028] A fourth acquisition unit for multiplying each term of the code length matrix by each term of the error matrix to obtain an average code length matrix;

[0029] A fifth acquisition unit for summing each term of the average code length matrix to obtain the total average code length; using the value of the total average code length as the scoring index.

[0030] Optionally, before the step of performing binary entropy encoding on the terms of the error matrix using the entropy encoding method to obtain an encoding matrix, it further includes:

[0031] Adjust the terms of the error matrix using the adjusted weight method to obtain a new error matrix.

[0032] Optionally, before the step of calculating the code length of each encoding in the encoding matrix according to the encoding matrix to obtain a code length matrix, it further includes:

[0033] Modify the terms of the original code length matrix by adjusting the proportion to obtain a new code length matrix.

[0034] Optionally, after the step of summing each term of the average code length matrix to obtain the total average code length, it further includes:

[0035] Perform transformation on the total average code length using the transformation calculation method, and use the obtained value as the scoring index.

[0036] Optionally, the step of performing binary entropy encoding on the terms of the error matrix using the entropy encoding method to obtain an encoding matrix includes:

[0037] Perform binary entropy coding on the terms of the error matrix in an entropy coding manner without deleting correct elements, and calculate using the calculation method with the shortest code length for the correct part by default to obtain a coding matrix; or

[0038] By adjusting the calculation method of each proportion and adjusting each ratio, calculate entropy coding to obtain a coding matrix; or

[0039] After calculating entropy coding by adjusting the calculation method of each proportion, obtain a coding matrix by adjusting the weights.

[0040] Optionally, the process of obtaining the coding matrix further includes: performing coding in a manner of independently coding each column or each row of the error matrix to obtain a coding matrix.

[0041] Optionally, the second obtaining unit is used for:[[]]

[0042] Perform entropy coding on the terms of the error matrix, and use each term as a symbol participating in the coding. Among them, the occurrence probability of the symbol is the value of the corresponding term in the error matrix, and perform binary coding on the symbols using the entropy coding method to obtain a coding matrix.

[0043] On the other hand, a confusion matrix scoring device based on entropy coding is provided. The confusion matrix scoring device based on entropy coding includes: a processor; a memory, and a computer-readable instruction is stored on the memory. When the computer-readable instruction is executed by the processor, any one of the methods in the above-mentioned confusion matrix scoring method based on entropy coding is implemented.

[0044] On the other hand, a computer-readable storage medium is provided. At least one instruction is stored in the storage medium, and the at least one instruction is loaded and executed by a processor to implement any one of the methods in the above-mentioned confusion matrix scoring method based on entropy coding.

[0045] The beneficial effects brought by the technical solution provided by the embodiments of the present invention at least include:

[0046] In the embodiments of the present invention, first, obtain an original confusion matrix; set the elements in the main diagonal of the original confusion matrix to zero to obtain an error matrix; perform binary entropy coding on the terms of the error matrix using an entropy coding method to obtain a coding matrix; secondly, calculate the code length of each coding in the coding matrix according to the coding matrix to obtain a code length matrix; multiply each term of the code length matrix by each term of the error matrix to obtain an average code length matrix; finally, sum each term of the average code length matrix to obtain a total average code length; use the value of the total average code length as a scoring index.

[0047] Using the present invention can well reflect the error rate and at the same time reflect the dispersion degree of the error situation.

[0048] It can well reflect the amount of information carried by the annotation content in data annotation. The present invention has interpretability, that is, the average minimum coding amount required to mark error details. BRIEF DESCRIPTION OF THE DRAWINGS

[0049] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following will briefly introduce the drawings required for the description of the embodiments. Obviously, the drawings in the following description are only some embodiments of the present invention. For those of ordinary skill in the art, without creative efforts, other drawings can also be obtained based on these drawings.

[0050] Figure 1 It is a flowchart of a confusion matrix scoring method based on entropy coding provided by an embodiment of the present invention;

[0051] Figure 2 It is a block diagram of a confusion matrix scoring device based on entropy coding provided by an embodiment of the present invention;

[0052] Figure 3 It is a schematic structural diagram of a confusion matrix scoring device based on entropy coding provided by an embodiment of the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0053] The following will describe the technical solutions in the present invention with reference to the drawings.

[0054] In the embodiments of the present invention, words such as "exemplarily" and "for example" are used to represent examples, illustrations or explanations. Any embodiment or design solution described as "example" in the present invention should not be construed as being more preferred or having more advantages than other embodiments or design solutions. Exactly, the use of the word "example" is intended to present concepts in a specific manner. In addition, in the embodiments of the present invention, the meaning expressed by "and / or" can be both, or either one of the two.

[0055] In the embodiments of the present invention, "image" and "picture" can sometimes be used interchangeably. It should be noted that when the difference is not emphasized, the meanings they express are the same. "(of)", "corresponding", and "corresponding" can sometimes be used interchangeably. It should be noted that when the difference is not emphasized, the meanings they express are the same.

[0056] In the embodiments of the present invention, sometimes subscripts such as W 1 may be written in a non-subscript form such as W1. When the difference is not emphasized, the meanings they express are the same.

[0057] In order to make the technical problems to be solved, technical solutions and advantages of the present invention clearer, the following will be described in detail with reference to the drawings and specific embodiments.

[0058] An embodiment of the present invention provides a method for scoring a confusion matrix based on entropy coding. This method can be implemented by a device for scoring a confusion matrix based on entropy coding, and the device for scoring a confusion matrix based on entropy coding can be a terminal or a server. As Figure 1 shown in the flowchart of the method for scoring a confusion matrix based on entropy coding, the processing flow of this method may include the following steps:

[0059] S1. Obtain the original confusion matrix; set the elements in the main diagonal of the original confusion matrix to zero to obtain an error matrix.

[0060] Among them, the confusion matrix is used to summarize various error situations; the confusion matrix includes two dimensions. The first dimension is various correct answers, including possibilities in ; the second dimension is various occurrence results, including possibilities, denoted as , where, generally and contain the same elements; there are elements in the confusion matrix, and each element

[0061] , represents a ratio, and the meaning of the ratio is: among all data, when the correct answer is the th value, the actual result is the th value, accounting for the proportion of all situations, in the form of: .

[0062] Among them, for example, in a Putonghua pronunciation test, 10% of the characters have the final a, and half of the a's are mispronounced as e by the examinee. Then, in the confusion matrix of finals, the value of the element representing a mispronounced as e is: .

[0063] Among them, set the values of the elements representing the correct part in the original confusion matrix to 0. The correct elements are the main diagonal of the confusion matrix, that is, if .

[0064] S2. Perform binary entropy coding on the terms of the error matrix using an entropy coding method to obtain a coding matrix.

[0065] Optionally, performing binary entropy coding on the terms of the error matrix using an entropy coding method to obtain a coding matrix in S2 includes:

[0066] Entropy encode the terms of the error matrix, taking each term as a symbol participating in the encoding. Here, the occurrence probability of the symbol is the value of the corresponding term in the error matrix. Use the entropy encoding method to perform binary encoding on the symbols to obtain the encoding matrix.

[0067] Among them, the entropy encoding method can adopt Huffman coding, Shannon-Fano coding, arithmetic coding, etc., and the present invention does not make further limitations.

[0068] Optionally, before the step of using the entropy encoding method in S2 to perform binary entropy encoding on the terms of the error matrix to obtain the encoding matrix, it further includes:

[0069] Use the adjusted weight method to adjust the terms of the error matrix to obtain a new error matrix.

[0070] In a feasible implementation, double the column ratio of the correct answers in the original error matrix. Table 1 is the original error matrix.

[0071] Table 1

[0072]

[0073] Among them, as shown in Table 2 is the matrix after doubling the column ratio of the correct answers in the original error matrix.

[0074] Table 2

[0075]

[0076] Optionally, using the entropy encoding method in S2 to perform binary entropy encoding on the terms of the error matrix to obtain the encoding matrix includes:

[0077] Use the entropy encoding method to perform binary entropy encoding on the terms of the error matrix, without deleting the correct elements, and calculate using the calculation method with the shortest code length for the correct part by default to obtain the encoding matrix; or

[0078] By adjusting the calculation method of each item's ratio, by adjusting each ratio, calculate the entropy encoding to obtain the encoding matrix; or

[0079] After calculating the entropy encoding by adjusting the calculation method of each item's ratio, obtain the encoding matrix by adjusting the weights.

[0080] Optionally, in the process of obtaining the encoding matrix, it further includes: encoding by independently encoding each column or each row of the error matrix to obtain the encoding matrix.

[0081] Among them, when obtaining the encoding matrix, do not encode the overall, and encode by independently encoding each column or each row of the error matrix to obtain the encoding matrix

[0082] Among them, as shown in Table 3, the result of encoding by the independent encoding method for each row is presented.

[0083] Table 3

[0084]

[0085] S3. Calculate the code length of each code in the encoding matrix according to the encoding matrix to obtain a code length matrix.

[0086] Calculate the code length of each code in the encoding matrix by manual calculation to obtain a code length matrix.

[0087] Optionally, before S3 calculates the code length of each code in the encoding matrix according to the encoding matrix to obtain a code length matrix, it further includes:

[0088] Modify the items of the original code length matrix by adjusting the proportion to obtain a new code length matrix.

[0089] Among them, a new average code length matrix can be obtained by modifying the average code length matrix.

[0090] Among them, double the column code length of the light sound in the original code length matrix to obtain a new average code length matrix as shown in Table 4;

[0091] Table 4

[0092]

[0093] S4. Multiply the items of the code length matrix by the items of the error matrix to obtain an average code length matrix.

[0094] S5. Sum the items of the average code length matrix to obtain the total average code length; use the value of the total average code length as the scoring index.

[0095] Optionally, after S5 sums the items of the average code length matrix to obtain the total average code length, it further includes:

[0096] Perform a transformation on the total average code length by using a transformation calculation method, and use the obtained value as the scoring index.

[0097] Among them, the transformation calculation method includes square calculation, square root calculation, fraction calculation, etc., and the present invention does not make further limitations. For example, for the total average code length x, it can be transformed into and to be used as an index.

[0098] Among them, the total average code length can be comprehensively calculated with other indexes. For example, if the total average code length is x and the overall error rate is c, then can be used as an index.

[0099] Among them, other evaluation values can be obtained by performing a transformation with the average length and total length of entropy coding based on the confusion matrix as parameters.

[0100] Among them, a comprehensive evaluation value can be obtained by mixing the average length, total length, and other indicators of entropy coding based on the confusion matrix.

[0101] In a feasible implementation manner, the steps of evaluating the Chinese tone confusion matrix using the method of the present application include:

[0102] (1) Obtain the original confusion matrix, as shown in Table 5;

[0103] Table 5

[0104]

[0105] (2) Remove the parts where the output result is the same as the correct answer, that is, , to obtain an error matrix, as shown in Table 6;

[0106] Table 6

[0107]

[0108] (3) Construct a Huffman coding for the remaining partial matrix to obtain a coding matrix, as shown in Table 7;

[0109] Table 7

[0110]

[0111] (4) According to the coding matrix, obtain a code length matrix, as shown in Table 8;

[0112] Table 8

[0113]

[0114] (5) According to the code length matrix, multiply each code length by the probability corresponding to the error matrix to obtain an average code length matrix, as shown in Table 9;

[0115] Table 9

[0116]

[0117] (6) Sum up the items in the average code length matrix to obtain a total average code length of 2.646.

[0118] In a feasible implementation manner, the present invention can be used for ordinary examination standards; for example, organize the error situations of students into a confusion matrix, and use the total average code length obtained by the method proposed by the present invention as the deduction weight; compared with deducting points based on the error rate, it more reflects whether the types of errors are more.

[0119] In a feasible implementation, the present invention can be used for communication channel quality assessment. For example, in the communication process, noise can cause decoding errors, resulting in differences between the input content and the output content. The input-output situation is statistically formed into a confusion matrix, and the method proposed by the present invention is used to evaluate the communication quality.

[0120] In a feasible implementation, the present invention can be used for machine learning loss functions. For example, taking the correct label as the correct answer and the machine output label as the output result, a confusion matrix is established. The method proposed by the present invention is used to replace the method of using cross-entropy as the loss function in the field of machine learning.

[0121] In a feasible implementation, the present invention can be used for data annotation quality assessment. For example, a test set is used to test a classification model, the output result is compared with the answer label to check if they are consistent, a confusion matrix is statistically obtained, and the method proposed by the present invention is used to determine the accuracy of the model.

[0122] In a feasible implementation, the present invention can be used for second language learning outcome assessment. For example, in the standard of a Putonghua test, the error situations are statistically formed into a confusion matrix, and the method proposed by the present invention is used to evaluate the second language levels of individuals and groups.

[0123] In a feasible implementation, the present invention can be used for large language model output quality assessment. For example, the incorrect parts of the content output by the speech model are locally modified, the original text of the modified parts is statistically obtained, and the modified content forms a confusion matrix. The method proposed by the present invention is used for output assessment.

[0124] In the embodiment of the present invention, first, an original confusion matrix is obtained; the elements in the main diagonal of the original confusion matrix are set to zero to obtain an error matrix; the terms of the error matrix are subjected to binary entropy coding using an entropy coding method to obtain a coding matrix; secondly, the code length of each code in the coding matrix is calculated according to the coding matrix to obtain a code length matrix; the terms of the code length matrix are multiplied by the terms of the error matrix to obtain an average code length matrix; finally, the terms of the average code length matrix are summed to obtain the total average code length; the value of the total average code length is used as a scoring index.

[0125] Using the present invention can well reflect the error rate and at the same time reflect the dispersion of error situations.

[0126] It can well reflect the amount of information carried by the annotated content in data annotation. The present invention has interpretability, that is, the average minimum coding amount required to mark error details.

[0127] Figure 2 It is a block diagram of a confusion matrix scoring device based on entropy coding shown according to an exemplary embodiment. This device is used for the confusion matrix scoring method based on entropy coding. Refer to Figure 2, the device includes a first acquisition unit 310, a second acquisition unit 320, a third acquisition unit 330, a fourth acquisition unit 340, and a fifth acquisition unit 350. Among them:

[0128] The first acquisition unit 310 is used to acquire the original confusion matrix; set the elements in the main diagonal of the original confusion matrix to zero to obtain an error matrix;

[0129] The second acquisition unit 320 is used to perform binary entropy coding on the items of the error matrix by using entropy coding to obtain a coding matrix;

[0130] The third acquisition unit 330 is used to calculate the code length of each code in the coding matrix according to the coding matrix to obtain a code length matrix;

[0131] The fourth acquisition unit 340 is used to multiply the items of the code length matrix by the items of the error matrix to obtain an average code length matrix;

[0132] The fifth acquisition unit 350 is used to sum the items of the average code length matrix to obtain a total average code length; use the value of the total average code length as a scoring index.

[0133] Optionally, before the step of performing binary entropy coding on the items of the error matrix by using entropy coding to obtain a coding matrix, it further includes:

[0134] Adjust the items of the error matrix by using the adjusted weight method to obtain a new error matrix.

[0135] Optionally, before the step of calculating the code length of each code in the coding matrix according to the coding matrix to obtain a code length matrix, it further includes:

[0136] Modify the items of the original code length matrix by using the adjusted proportion method to obtain a new code length matrix.

[0137] Optionally, after the step of summing the items of the average code length matrix to obtain a total average code length, it further includes:

[0138] Perform transformation on the total average code length by using the transformation calculation method, and use the obtained value as a scoring index.

[0139] Optionally, the step of performing binary entropy coding on the items of the error matrix by using entropy coding to obtain a coding matrix includes:

[0140] Perform binary entropy coding on the items of the error matrix by using entropy coding, without deleting the correct elements, and calculate according to the calculation method with the shortest code length for the correct part by default to obtain a coding matrix; or

[0141] By adjusting the calculation method of each proportion, by adjusting each ratio, entropy coding is calculated to obtain a coding matrix; or

[0142] After calculating the entropy coding by adjusting the calculation method of each proportion, a coding matrix is obtained by adjusting the weights.

[0143] Optionally, the process of obtaining the coding matrix further includes: performing coding in a manner of independently coding each column or each row of the error matrix to obtain the coding matrix.

[0144] Optionally, the second obtaining unit 320 is configured to:

[0145] Perform entropy coding on the terms of the error matrix, and take each term as a symbol participating in the coding. Among them, the occurrence probability of the symbol is the value of the corresponding term in the error matrix, and binary coding is performed on the symbol by using the entropy coding method to obtain the coding matrix.

[0146] In the embodiment of the present invention, first, an original confusion matrix is obtained; the elements in the main diagonal of the original confusion matrix are set to zero to obtain an error matrix; binary entropy coding is performed on the terms of the error matrix by using the entropy coding method to obtain a coding matrix; secondly, the code length of each coding in the coding matrix is calculated according to the coding matrix to obtain a code length matrix; each term of the code length matrix is multiplied by each term of the error matrix to obtain an average code length matrix; finally, the terms of the average code length matrix are summed to obtain the total average code length; the value of the total average code length is used as the scoring index.

[0147] Adopting the present invention can well reflect the error rate and at the same time reflect the dispersion degree of the error situation.

[0148] It can well reflect the amount of information carried by the annotation content in data annotation. The present invention has interpretability, that is, the average minimum coding amount required to mark the error details.

[0149] Figure 3 It is a schematic structural diagram of a confusion matrix scoring device based on entropy coding provided by an embodiment of the present invention. As Figure 3 shown, the confusion matrix scoring device based on entropy coding may include the above-mentioned Figure 2 shown confusion matrix scoring device. Optionally, the confusion matrix scoring device 310 based on entropy coding may include a first processor 2001.

[0150] Optionally, the confusion matrix scoring device 310 based on entropy coding may further include a memory 2002 and a transceiver 2003.

[0151] Among them, the first processor 2001, the memory 2002, and the transceiver 2003 may be connected through a communication bus, for example.

[0152] The following combines Figure 3 to specifically introduce each component of the confusion matrix scoring device 310 based on entropy coding:

[0153] Among them, the first processor 2001 is the control center of the confusion matrix scoring device 310 based on entropy coding, which can be a single processor or a collective term for multiple processing elements. For example, the first processor 2001 is one or more central processing units (CPUs), or can be an application specific integrated circuit (ASIC), or one or more integrated circuits configured to implement the embodiments of the present invention, such as: one or more digital signal processors (DSPs), or one or more field programmable gate arrays (FPGAs).

[0154] Optionally, the first processor 2001 can execute various functions of the confusion matrix scoring device 310 based on entropy coding by running or executing software programs stored in the memory 2002 and calling data stored in the memory 2002.

[0155] In a specific implementation, as an embodiment, the first processor 2001 may include one or more CPUs, such as Figure 3 the CPU0 and CPU1 shown in

[0156] In a specific implementation, as an embodiment, the confusion matrix scoring device 310 based on entropy coding may also include multiple processors, such as Figure 3 the first processor 2001 and the second processor 2004 shown in

[0157] Each of these processors can be a single-core processor (single-CPU) or a multi-core processor (multi-CPU). Here, the processor can refer to one or more devices, circuits, and / or processing cores for processing data (such as computer program instructions).

[0158] Optionally, the memory 2002 may be a read-only memory (ROM) or other type of static storage device that can store static information and instructions, a random access memory (RAM) or other type of dynamic storage device that can store information and instructions, or may also be an electrically erasable programmable read-only memory (EEPROM), a compact disc read-only memory (CD-ROM) or other optical disc storage, optical disc storage (including compact discs, laser discs, optical discs, digital versatile discs, Blu-ray discs, etc.), magnetic disk storage media or other magnetic storage devices, or any other medium that can be used to carry or store the desired program code in the form of instructions or data structures and can be accessed by a computer, but is not limited thereto. The memory 2002 may be integrated with the first processor 2001 or may exist independently and be coupled to the first processor 2001 through an interface circuit ( Figure 3 not shown) of the entropy coding-based confusion matrix scoring device 310. The embodiments of the present invention do not make specific limitations thereto.

[0159] The transceiver 2003 is used to communicate with a network device or with a terminal device.

[0160] Optionally, the transceiver 2003 may include a receiver and a transmitter ( Figure 3 not shown separately). Among them, the receiver is used to implement the receiving function, and the transmitter is used to implement the sending function.

[0161] Optionally, the transceiver 2003 may be integrated with the first processor 2001 or may exist independently and be coupled to the first processor 2001 through an interface circuit ( Figure 3 not shown) of the entropy coding-based confusion matrix scoring device 310. The embodiments of the present invention do not make specific limitations thereto.

[0162] It should be noted that Figure 3 the structure of the entropy coding-based confusion matrix scoring device 310 shown does not constitute a limitation on the router. The actual knowledge structure recognition device may include more or fewer components than shown, or combine certain components, or have different component arrangements.

[0163] In addition, the technical effects of the entropy coding-based confusion matrix scoring device 310 may refer to the technical effects of the entropy coding-based confusion matrix scoring method described in the above method embodiments, and will not be elaborated here.

[0164] It should be understood that the first processor 2001 in the embodiments of the present invention may be a central processing unit (CPU), and the processor may also be other general-purpose processors, digital signal processors (DSPs), application specific integrated circuits (ASICs), field programmable gate arrays (FPGAs) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor may be a microprocessor or the processor may also be any conventional processor, etc.

[0165] It should also be understood that the memory in the embodiments of the present invention may be a volatile memory or a non-volatile memory, or may include both volatile and non-volatile memories. Among them, the non-volatile memory may be a read-only memory (ROM), a programmable ROM (PROM), an erasable programmable ROM (EPROM), an electrically erasable programmable ROM (EEPROM) or a flash memory. The volatile memory may be a random access memory (RAM), which is used as an external cache. By way of example but not limitation, many forms of random access memory (RAM) are available, such as static random access memory (SRAM), dynamic random access memory (DRAM), synchronous dynamic random access memory (SDRAM), double data rate synchronous dynamic random access memory (DDR SDRAM), enhanced synchronous dynamic random access memory (ESDRAM), synchlink DRAM (SLDRAM) and direct rambus RAM (DR RAM).

[0166] The above embodiments can be implemented in whole or in part by software, hardware (such as circuits), firmware, or any other combination. When implemented using software, the above embodiments can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions or computer programs. When the computer instructions or computer programs are loaded or executed on a computer, the processes or functions described in the embodiments of the present invention are generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable devices. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired (such as infrared, wireless, microwave, etc.) means. The computer-readable storage medium can be any available medium that can be accessed by a computer or a data storage device such as a server or a data center that contains one or more collections of available media. The available media can be magnetic media (such as floppy disks, hard disks, magnetic tapes), optical media (such as DVDs), or semiconductor media. The semiconductor media can be a solid-state drive.

[0167] It should be understood that the term "and / or" in this document is merely a description of the association relationship between associated objects, indicating that three relationships can exist. For example, A and / or B can represent three situations: A exists alone, A and B exist simultaneously, and B exists alone. Here, A and B can be singular or plural. Additionally, the character " / " in this document generally represents an "or" relationship between the associated objects before and after, but it may also represent an "and / or" relationship, which can be specifically understood by referring to the context before and after.

[0168] In the present invention, "at least one" means one or more, and "a plurality" means two or more. "At least one of the following" or its similar expressions refer to any combination of these items, including any combination of single items or plural items. For example, at least one of a, b, or c can represent: a, b, c, a - b, a - c, b - c, or a - b - c, where a, b, and c can be single or multiple.

[0169] It should be understood that in various embodiments of the present invention, the magnitudes of the sequence numbers of the above processes do not imply the order of execution. The order of execution of each process should be determined by its function and internal logic, and should not constitute any limitation to the implementation process of the embodiments of the present invention.

[0170] Those of ordinary skill in the art can realize that the units and algorithm steps of each example described in combination with the embodiments disclosed herein can be implemented by electronic hardware, or by a combination of computer software and electronic hardware. Whether these functions are executed in a hardware or software manner depends on the specific application and design constraints of the technical solution. Skilled professionals can use different methods for each specific application to implement the described functions, but such implementation should not be considered to exceed the scope of the present invention.

[0171] Those skilled in the art can clearly understand that for the convenience and brevity of description, the specific working processes of the above-described devices, apparatuses, and units can refer to the corresponding processes in the foregoing method embodiments and will not be elaborated herein.

[0172] In several embodiments provided by the present invention, it should be understood that the disclosed devices, apparatuses, and methods can be implemented in other ways. For example, the device embodiments described above are merely illustrative. For example, the division of the units is only a logical function division, and there can be other division methods in actual implementation. For example, multiple units or components can be combined or integrated into another device, or some features can be ignored or not executed. Another point is that the displayed or discussed couplings or direct couplings or communication connections to each other can be through some interfaces, and the indirect couplings or communication connections of the devices or units can be in electrical, mechanical, or other forms.

[0173] The units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they can be located in one place, or can be distributed to multiple network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0174] In addition, the functional units in each embodiment of the present invention can be integrated in a processing unit, or each unit can exist physically alone, or two or more units can be integrated in one unit.

[0175] When the above-described functions are implemented in the form of software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or a part of this technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions for causing a computer device (which may be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the methods described in various embodiments of the present invention. The foregoing storage medium includes: various media that can store program codes, such as USB flash drives, mobile hard disks, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical discs.

[0176] As described above, the above are only specific embodiments of the present invention, but the protection scope of the present invention is not limited thereto. Any person skilled in the art within the technical scope disclosed by the present invention can easily think of changes or substitutions, which should all be covered by the protection scope of the present invention. Therefore, the protection scope of the present invention shall be subject to the protection scope of the claims.

Claims

1. A confusion matrix scoring method based on entropy coding, characterized in that: The method comprises: S1. Obtain an original confusion matrix; set the elements in the main diagonal of the original confusion matrix to zero to obtain an error matrix; S2. Performing binary entropy coding on the items of the error matrix by adopting an entropy coding method to obtain a coding matrix; S3. Calculate the code length of each code in the coding matrix according to the coding matrix to obtain a code length matrix; S4, multiplying each item of the code length matrix with each item of the error matrix to obtain an average code length matrix; S5. Sum each item of the average code length matrix to obtain a total average code length; and use the value of the total average code length as a scoring index.

2. The confusion matrix scoring method based on entropy coding according to claim 1, characterized in that: Before the step of performing binary entropy coding on the items of the error matrix by entropy coding to obtain the coding matrix in S2, the step further includes: The items of the error matrix are adjusted by using an adjustment weight method to obtain a new error matrix.

3. The confusion matrix scoring method based on entropy coding according to claim 1, characterized in that: The step S3 further includes calculating the code length of each code in the coding matrix according to the coding matrix, before obtaining the code length matrix: The items of the original code length matrix are modified by adjusting the proportions to obtain a new code length matrix.

4. The confusion matrix scoring method based on entropy coding according to claim 1, characterized in that: After the step of summing up each item of the average code length matrix to obtain the total average code length in S5, the method further includes: The total average code length is transformed by transformation calculation, and the transformed value is used as the scoring indicator.

5. The confusion matrix scoring method based on entropy coding according to claim 1, characterized in that: The step S2 uses entropy coding to perform binary entropy coding on the items of the error matrix to obtain a coding matrix, including: Performing binary entropy coding on the items of the error matrix by entropy coding, without deleting the correct elements, and calculating by default using the shortest code length calculation method for the correct part to obtain a coding matrix; or By adjusting the calculation method of each weight, by adjusting each ratio, calculating the entropy coding, and obtaining the coding matrix; or After calculating the entropy coding by adjusting the calculation method of each weight, the coding matrix is ​​obtained by adjusting the weights.

6. The confusion matrix scoring method based on entropy coding according to claim 1, characterized in that: The process of obtaining the coding matrix further includes: encoding each column or each row of the error matrix independently to obtain the coding matrix.

7. The confusion matrix scoring method based on entropy coding according to claim 1, characterized in that: The step S2 uses entropy coding to perform binary entropy coding on the items of the error matrix to obtain a coding matrix, including: The items of the error matrix are entropy coded, and each item is used as a symbol involved in the coding, where the probability of occurrence of the symbol is the value of the corresponding item in the error matrix. The symbols are binary coded using entropy coding to obtain a coding matrix.

8. A confusion matrix scoring device based on entropy coding, wherein the confusion matrix scoring device based on entropy coding is used to implement the confusion matrix scoring method based on entropy coding as claimed in any one of claims 1 to 7, characterized in that: The device comprises: A first acquisition unit is used to acquire an original confusion matrix; set the elements in the main diagonal of the original confusion matrix to zero to obtain an error matrix; A second acquisition unit is used to perform binary entropy coding on the items of the error matrix by adopting an entropy coding method to obtain a coding matrix; A third acquisition unit, configured to calculate the code length of each code in the coding matrix according to the coding matrix, and obtain a code length matrix; a fourth acquisition unit, configured to multiply each item of the code length matrix by each item of the error matrix to obtain an average code length matrix; The fifth acquisition unit is used to sum up the items of the average code length matrix to obtain the total average code length; and use the value of the total average code length as a scoring indicator.

9. A confusion matrix scoring device based on entropy coding, characterized in that: The confusion matrix scoring device based on entropy coding includes: processor; A memory having computer-readable instructions stored thereon, wherein when the computer-readable instructions are executed by the processor, the method according to any one of claims 1 to 7 is implemented.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores program codes, which can be called by a processor to execute the method according to any one of claims 1 to 7.

Images