Image recognition model testing method, device, computer equipment and storage medium

Abstract

The present invention is applicable to the field of performance testing, and discloses an image recognition model testing method, device, computer equipment and storage medium, the method comprising: obtaining a test image sample set, the test image sample set comprising multiple test image samples and multiple real annotation information, one test image sample corresponding to one real annotation information; inputting multiple test image samples into the image recognition model to be tested, and obtaining the predicted recognition results of multiple test image samples; respectively calculating the matching degree of the predicted recognition result of each test image sample with the real annotation information, and obtaining the matching degree of multiple test image samples; constructing a confusion matrix according to the matching degree of multiple test image samples, and calculating the performance evaluation result of the image recognition model to be tested according to the confusion matrix. The testing method of the present invention can realize machine automated testing, and has high testing efficiency and good reliability.

Description

Technical Field

[0001] The present invention relates to the field of performance testing, and in particular to an image recognition model testing method, device, computer equipment and storage medium. Background Art

[0002] Artificial Intelligence (AI) is a branch of computer science. It is a method of simulating human intelligence through artificial objects and its implementation technology. Artificial intelligence involves big data and deep learning. The main mode of artificial intelligence in solving practical problems is to use big data and deep learning technology to train a model to solve a practical problem. The performance of the model is the key to its final use effect (solving practical problems).

[0003] At present, model performance testing is basically carried out by technicians with relevant professional skills through manual evaluation. Not only is the testing workload large, complex and inefficient, but it is also easy to cause errors in the test results due to human errors, thus affecting the reliability of the test results.

[0004] It can be seen that the current model testing method has the problems of low testing efficiency and poor reliability. Summary of the invention

[0005] Based on this, it is necessary to provide an image recognition model testing method, device, computer equipment and storage medium to address the above technical problems, so as to solve the problems of low testing efficiency and poor reliability of the current model testing method.

[0006] A method for testing an image recognition model, comprising:

[0007] Acquire a test image sample set, wherein the test image sample set includes multiple test image samples and multiple real annotation information, and one test image sample corresponds to one real annotation information;

[0008] Inputting the plurality of test image samples into the image recognition model to be tested to obtain predicted recognition results of the plurality of test image samples;

[0009] Calculating the matching degree between the predicted recognition result and the real annotation information of each test image sample respectively, and obtaining the matching degree of multiple test image samples;

[0010] A confusion matrix is ​​constructed according to the matching degrees of the multiple test image samples, and a performance evaluation result of the image recognition model to be tested is calculated according to the confusion matrix.

[0011] An image recognition model testing device, comprising:

[0012] A first acquisition module is used to acquire a test image sample set, wherein the test image sample set includes a plurality of test image samples and a plurality of real annotation information, and one test image sample corresponds to one real annotation information;

[0013] A second acquisition module, used for inputting the plurality of test image samples into the image recognition model to be tested, and obtaining the predicted recognition results of the plurality of test image samples;

[0014] A first calculation module is used to calculate the matching degree between the predicted recognition result and the real annotation information of each of the test image samples, so as to obtain the matching degree of multiple test image samples;

[0015] The second calculation module is used to construct a confusion matrix according to the matching degree of the multiple test image samples, and calculate the performance evaluation result of the image recognition model to be tested according to the confusion matrix.

[0016] A computer device comprises a memory, a processor and computer-readable instructions stored in the memory and executable on the processor, wherein the processor implements the above-mentioned image recognition model testing method when executing the computer-readable instructions.

[0017] One or more readable storage media storing computer-readable instructions, which, when executed by one or more processors, enable the one or more processors to execute the image recognition model testing method as described above.

[0018] The image recognition model testing method, device, computer equipment and storage medium described above, in the process of testing the model performance of the trained image recognition model, can input multiple test image samples in the acquired test image sample set into the image recognition model to be tested to obtain the predicted recognition results of the multiple test image samples, and establish a confusion matrix by respectively calculating the matching degree between the predicted recognition results of each test image sample and the real annotation information, and calculate the performance evaluation result of the image recognition model to be tested based on the established confusion matrix. The above process can realize machine automated testing with high testing efficiency and good reliability. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings required for use in the description of the embodiments of the present invention will be briefly introduced below. Obviously, the accompanying drawings in the following description are only some embodiments of the present invention. For ordinary technicians in this field, other accompanying drawings can be obtained based on these accompanying drawings without paying creative labor.

[0020] Figure 1 is a flowchart of an image recognition model testing method in a first embodiment of the present invention;

[0021] Figure 2 is a flow chart of an image recognition model testing method in a second embodiment of the present invention;

[0022] Figure 3 is a flow chart of an image recognition model testing method in a third embodiment of the present invention;

[0023] Figure 4 is a schematic flow chart of an image recognition model testing method in a fourth embodiment of the present invention;

[0024] Figure 5 is a structural schematic diagram of an image recognition model testing device in one embodiment of the present invention;

[0025] Figure 6 is a schematic diagram of the structure of the second acquisition module 12 in one embodiment of the present invention;

[0026] Figure 7 is a schematic diagram of the structure of the first calculation unit 13 in one embodiment of the present invention;

[0027] Figure 8 is a schematic structural diagram of the first calculation unit 13 in another embodiment of the present invention;

[0028] Fig. 9 is a schematic diagram of a computer device in one embodiment of the present invention. DETAILED DESCRIPTION

[0029] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0030] The image recognition model testing method provided by the embodiment of the present invention, during the process of performing model performance testing on the trained image recognition model, multiple test image samples with real annotation information are input into the image recognition model to be tested for testing, and predicted recognition results of the multiple test image samples are obtained, and a confusion matrix is ​​established by respectively calculating the matching degree between the predicted recognition result of each test image sample and the real annotation information, and the performance evaluation result of the image recognition model to be tested is calculated based on the established confusion matrix. The entire process can be completed by machine operation to complete the automatic testing task, and the testing efficiency is high and the reliability is good.

[0031] In the first embodiment, if Figure 1As shown, a flowchart of an implementation method of an image recognition model testing method is provided. For ease of description, only the part related to this embodiment is shown in the figure. The method includes the following steps:

[0032] Step S10, obtaining a test image sample set, wherein the test image sample set includes a plurality of test image samples and a plurality of real annotation information, and one test image sample corresponds to one real annotation information.

[0033] The test images in the test image sample set may be pictures or video data pre-stored in a computer memory. During the test, part or all of the pictures or video data may be directly retrieved from the memory.

[0034] If the video data retrieved is a 30-second video, then you can first use the preset capture rule to capture an image of a certain segment in the video as a test image sample. The capture rule here usually refers to the video at which second to capture. For example, if you capture the image at the 10th second of the video, then the test image sample is the image captured after the video is paused at the 10th second.

[0035] In the embodiment of the present invention, the real annotation information refers to the description information set according to the recognition object or recognition purpose of the image recognition model. For example, if the image recognition model to be tested is used to identify whether the employees shown in the test image sample are wearing work clothes and nameplates correctly, then the real annotation information here can be text annotation information or voice information such as "the employee wears the work clothes and nameplate correctly" or "the employee does not wear the work clothes and nameplate correctly".

[0036] It is understandable that the test image samples may also be pictures or video data collected from the Internet by connecting to the Internet when testing the image recognition model. The corresponding real annotation information can be generated by manually annotating the pictures or video data collected from the Internet.

[0037] Step S20: input the multiple test image samples into the image recognition model to be tested to obtain predicted recognition results of the multiple test image samples.

[0038] In combination with the above example, the image recognition model to be tested may be a model for identifying whether the employees shown in the test image samples are wearing work clothes and nameplates correctly. When the image recognition model to be tested is run to analyze and test each test image sample, each test image sample may be compared with a preset standard image of an employee wearing work clothes and nameplates correctly, thereby identifying whether the employees in each test image sample are wearing work clothes and nameplates correctly, that is, obtaining a predicted recognition result of whether the employees in each test image sample are wearing work clothes and nameplates correctly.

[0039] Step S30, respectively calculating the matching degree between the predicted recognition result of each of the test image samples and the real annotation information to obtain the matching degree of multiple test image samples.

[0040] As an exemplary embodiment of the present invention, it is assumed that there are 4 test image samples A, B, C, and D in the acquired test image sample set, and the number of positive samples and negative samples are half each, where the positive samples refer to image samples in which employees wear work clothes and nameplates correctly, and the negative samples are image samples in which employees do not wear work clothes and nameplates correctly. Each test image sample corresponds to a real annotation information, which are a, b, c, and d respectively. These 4 test image samples are input into the image recognition model to be tested, and after running the image recognition model to be tested to test these 4 test image samples, the predicted recognition results corresponding to the test image samples A, B, C, and D are output as a', b', c', and d'; then, the matching degrees of a and a', b and b', c and c', and d and d' are calculated respectively, and the matching degrees e, f, g, and h of the 4 test image samples are obtained.

[0041] Step S40, constructing a confusion matrix according to the matching degrees of the multiple test image samples, and calculating the performance evaluation result of the image recognition model to be tested according to the confusion matrix.

[0042] In combination with the above exemplary embodiments, in actual applications, the model prediction result can be pre-set to be that the employee wears the work clothes and nameplate correctly, and the actual annotation information is also that the employee wears the work clothes and nameplate correctly, with a matching degree of +100%; the model prediction result is that the employee does not wear the work clothes and nameplate correctly, and the actual annotation information is also that the employee does not wear the work clothes and nameplate correctly, with a matching degree of -100%; the model prediction result is that the employee wears the work clothes and nameplate correctly, and the actual annotation information is that the employee does not wear the work clothes and nameplate correctly, with a matching degree of +50%; the model prediction result is that the employee does not wear the work clothes and nameplate correctly, and the actual annotation information is also that the employee wears the work clothes and nameplate correctly, with a matching degree of -50%.

[0043] Assuming that the matching degrees e, f, g, and h of the four test image samples obtained are +100%, -100%, +50%, and -50% respectively, then the model prediction result of these four test image samples is that the employee wears the work clothes and nameplate correctly, and the actual annotation information is also that the employee wears the work clothes and nameplate correctly. The number of samples is 1; the model prediction result is that the employee does not wear the work clothes and nameplate correctly, and the actual annotation information is that the employee does not wear the work clothes and nameplate correctly. The number of samples is 1; the model prediction result is that the employee wears the work clothes and nameplate correctly, and the actual annotation information is that the employee does not wear the work clothes and nameplate correctly. The number of samples is 1, and the model prediction result is that the employee does not wear the work clothes and nameplate correctly, and the actual annotation information is that the employee wears the work clothes and nameplate correctly. The number of samples is 1, and the confusion matrix is ​​constructed based on the above statistical results. Among them, the first row and the first column in the matrix count the number of samples whose model prediction results are that the employees do not wear work clothes and nameplates correctly, and the actual annotation information is that the employees also do not wear work clothes and nameplates correctly; the first row and the second column count the number of samples whose model prediction results are that the employees wear work clothes and nameplates correctly, and the actual annotation information is that the employees do not wear work clothes and nameplates correctly; the second row and the first column count the number of samples whose model prediction results are that the employees do not wear work clothes and nameplates correctly, and the actual annotation information is that the employees wear work clothes and nameplates correctly; the second row and the second column count the number of samples whose model prediction results are that the employees wear work clothes and nameplates correctly, and the actual annotation information is that the employees wear work clothes and nameplates correctly, and then calculate the performance evaluation result of the image recognition model to be tested according to the fixed formula of the confusion matrix.

[0044] The evaluation indicators commonly used to evaluate the performance of the trained model in the field of machine learning are accuracy, recall rate, precision rate and F1 value. The calculation methods of these evaluation indicators can be calculated by the corresponding fixed formula of the confusion matrix, which is the existing technology in this field, so it will not be elaborated in detail here.

[0045] In the process of testing the model performance of the trained image recognition model, the present invention can input multiple test image samples in the acquired test image sample set into the image recognition model to be tested to obtain the predicted recognition results of the multiple test image samples, and establish a confusion matrix by respectively calculating the matching degree between the predicted recognition results of each test image sample and the real annotation information, and calculate the performance evaluation result of the image recognition model to be tested according to the established confusion matrix. The above process can realize machine automated testing with high testing efficiency and good reliability.

[0046] In the second embodiment, if Figure 2As shown, this embodiment is basically the same as the first embodiment, except that step S20 is replaced by step S201 and step S202; step S30 is replaced by step S301. For ease of description, only the parts related to this embodiment are shown in the figure, which are described in detail as follows:

[0047] Step S201: input the plurality of test image samples into the image recognition model to be tested, and extract image feature information of each of the test image samples, wherein the image feature information includes environmental feature information and predicted target object feature information.

[0048] Assume that the image recognition model to be tested is a recognition model for identifying whether the target object in a scene image is a football. The test image samples in the acquired test image sample set are images captured by a camera device (such as a camera or a camera installed on a robot) preset at a certain place in a football field. The captured images include people, footballs, and football field lawns that enter the shooting field of view.

[0049] When conducting the test, first, the image feature information in each test image sample can be extracted according to the preset color feature, and the extracted image feature information includes the environmental feature information of the image and the predicted target object feature information. For example, the preset color feature can be green and non-green, and the football turf and non-football field turf image areas in the test image sample can be distinguished according to the color feature. At this time, the predicted target object feature information, that is, the feature information of the football (such as round, black and white, etc.) can be further extracted from the non-football field turf image area.

[0050] Step S202: identifying the predicted target object corresponding to the test image sample according to the predicted target object feature information.

[0051] In combination with the above example, when the predicted target feature information extracted from the non-football field lawn image area is circular and black and white, the predicted target object corresponding to the current test image sample can be retrieved and determined to be a football according to the preset object feature-object mapping table. The preset object feature-object mapping table can be set in the form shown in Table 1 below.

[0052] Table 1 Preset object feature-object mapping table

[0053] Object characteristics Object Black and white, round football Yellow, round table tennis … …

[0054] Step S301 , respectively calculating the matching degree between the predicted target object and the real target object of each of the test image samples to obtain the matching degree of multiple test image samples.

[0055] In this embodiment, in combination with the above example, each test image sample may be labeled with information indicating that the image is actually “football” or “non-football”.

[0056] In one possible implementation, a first eigenvalue is constructed based on a predicted recognition result of a test image sample; a second eigenvalue is constructed based on real annotation information corresponding to the test image sample; and a matching degree feature vector of the test image sample is constructed based on the first eigenvalue and the second eigenvalue.

[0057] Specifically, a first eigenvalue is constructed according to the predicted recognition result (soccer or non-soccer) of the test image sample (if the predicted recognition result is soccer, it is recorded as 1, and if the predicted recognition result is non-soccer, it is recorded as 0).

[0058] Similarly, a second eigenvalue is constructed according to the true annotation information (soccer ball or non-soccer ball) corresponding to the test image sample (if the true annotation information is soccer ball, it is recorded as 1, and if the true annotation information is non-soccer ball, it is recorded as 0).

[0059] If the first eigenvalue of a test image sample is 1 and the second eigenvalue is 1, then the matching feature vector of the test image sample is (1,1); if the first eigenvalue of a test image sample is 1 and the second eigenvalue is 0, then the matching feature vector of the test image sample is (1,0); if the first eigenvalue of a test image sample is 0 and the second eigenvalue is 1, then the matching feature vector of the test image sample is (0,1); if the first eigenvalue of a test image sample is 0 and the second eigenvalue is 0, then the matching feature vector of the test image sample is (0,0).

[0060] In this embodiment, a confusion matrix is ​​constructed based on the matching feature vectors of each test image sample obtained. Among them, (1,1) means that the actual result is football and the predicted result is also football; (1,0) means that the actual result is not football and the predicted result is football; (0,1) means that the actual result is football and the predicted result is not football; (0,0) means that the actual result is not football and the predicted result is also not football.

[0061] After obtaining the matching feature vector of each test image sample, the image matching type of each test image sample can be determined, and the image matching types include true positive examples (matching feature vector is (1,1)), true negative examples (matching feature vector is (0,0)), false positive examples (matching feature vector is (1,0)) and false negative examples (matching feature vector is (0,1)). The number of each image matching type is counted, and a confusion matrix is ​​constructed according to the image matching types and their numbers.

[0062] In the third embodiment, if Figure 3As shown, this embodiment is basically the same as the first embodiment, except that the step S20 is replaced by step S203, step S204 and step S205.

[0063] Step S203: performing blur degree identification on each of the test image samples to obtain the blur degree of each of the test image samples.

[0064] The quality of an image is an important factor affecting the accuracy of image recognition, and the quality of an image is usually reflected in whether the image is clear. A clear image can reduce the recognition error rate, while a blurred image can easily increase the error rate of image recognition. In order to save manual selection of clear images for testing the performance of the image recognition model to be tested, it is preferred to identify the blurriness of the test image samples before using the image recognition model to be tested to test the input test image samples, so as to obtain the blurriness of each test image sample.

[0065] In one embodiment, a preset image blur algorithm can be called to identify the blur degree of the test image sample to obtain the blur degree of each test image sample. The image blur algorithm can adopt the Gaussian blur algorithm in the prior art, or the convolutional neural network CNN and other methods. The specific implementation process of the algorithm can refer to the prior art and will not be described one by one here.

[0066] In another embodiment, the image edge information of the test image sample may be obtained first, where the image edge information includes edge width, and then the blurriness of the test image sample may be determined according to the edge width.

[0067] Step S204 , according to the blurriness, blurriness correction is performed on each of the test image samples based on restoration filters with different blurriness parameters to obtain a plurality of corrected test image samples.

[0068] In this embodiment, the preset blurriness can be divided into three levels: no blurriness, slight blurriness, and severe blurriness. A predetermined point spread function can be used to generate a restoration filter with different blurriness parameters to perform blurriness correction on images with different blurriness to obtain a clear image, that is, a corrected test image sample.

[0069] Among them, the point spread function (PSF) is a function that describes the ability of an optical system to resolve a point source. This is because a point source will form an enlarged image point due to diffraction after passing through any optical system. The point spread function is the transformation output from the image corresponding to the point source input, and is usually used to transform a clear input into a blurred output. The point spread function, as an image blur model, includes parameters corresponding to the degree of blur, specifically, parameters corresponding to the three blur levels of no blur, slight blur, and severe blur.

[0070] Step S205: input the plurality of corrected test image samples into the image recognition model to be tested to obtain predicted recognition results of the plurality of test image samples.

[0071] In this embodiment, the specific implementation process of step S205 can refer to the implementation process of the relevant steps of the first and second embodiments above, and will not be described in detail here.

[0072] In the fourth embodiment, if Figure 4 As shown, this embodiment is basically the same as the first embodiment, except that step S30 is replaced by step S302, step S303, step S304 and step S305. For ease of description, only the parts related to this embodiment are shown in the figure, which are described in detail as follows:

[0073] Step S302: Acquire first text information of a predicted recognition result of the test image sample and second text information of real annotation information corresponding to the test image sample.

[0074] In an exemplary embodiment of the present invention, the predicted recognition result and the actual annotation information of the test image sample can be text description information of one or more paragraphs, one or more sentences, one or more words or voice information about the image content.

[0075] For example, the real annotation information is a text description information of "an exciting football match was held on the green grass stadium football field, the two teams were the Wolves wearing red uniforms and the Eagles wearing blue uniforms" (i.e. the first text information). The predicted recognition result is the text information of "green lawn, red and blue football" (i.e. the second text information).

[0076] It is understandable that when the predicted recognition result and / or the actual annotation information is voice information, the voice information can be converted into text information first.

[0077] Step S303: perform a word segmentation operation on the first text information, count the frequency of each word segmentation, and determine the first keyword of the first text information according to the statistical result.

[0078] From the perspective of part of speech, words can be divided into content words and function words; content words are words with actual meaning, including nouns, verbs, adjectives, numerals, and quantifiers; function words are words without actual meaning, including adverbs, prepositions, conjunctions, auxiliary words, interjections, and onomatopoeia.

[0079] In one embodiment, the first text information is segmented according to parts of speech, and the frequency of each segmented word is counted. Based on the statistical results, a word that appears repeatedly a preset number of times in the first text information is determined as a first keyword.

[0080] The preset number of times may be greater than or equal to 2, 3, 4, etc., which is not specifically limited here. For example, in "A wonderful football match was held on the green grass stadium football field, and the two teams were the Wolves wearing red uniforms and the Eagles wearing blue uniforms", the words "football", "match", and "uniforms" that appear at least twice are repeated. At this time, these words can be determined as the first keywords.

[0081] In practical applications, in order to more intuitively and accurately reflect the functions to be achieved by the image recognition model, the above-mentioned divided words can be screened according to the recognition object of the image recognition model, and finally a word that is most relevant to the function to be achieved by the model is determined, and the word is usually a noun. For example, if the image recognition model to be tested is used to recognize football images, then the word that is the same as "football" that appears in the first text information can be determined as the first keyword. If "Cuju" appears in the first text information, "Cuju", which is a synonym of "football" in the first text information, can also be determined as the first keyword. If "ball" appears in the first text information, "ball" which belongs to the same theme as "football" can also be determined as the first keyword.

[0082] Step S304: perform a word segmentation operation on the second text information, count the frequency of each word segmentation, and determine the second keyword of the second text information according to the statistical result.

[0083] In this embodiment, the word segmentation operation of the second text information and the method for determining the second keyword can refer to the above-mentioned method for determining the first text information and the first keyword.

[0084] Step S305: Calculate the similarity between the first keyword and the second keyword, and determine the matching degree of the multiple test image samples according to the similarity.

[0085] Specifically, in the embodiment of the present invention, the above step S305 includes:

[0086] It is determined whether the part of speech of the first keyword is the same as the part of speech of the second keyword.

[0087] If the part of speech of the first keyword is the same as the part of speech of the second keyword, the similarity of the meanings of the first keyword and the second keyword is determined.

[0088] The matching degree of the plurality of test image samples is determined according to the word meaning similarity.

[0089] In this embodiment, first determine whether the part of speech of the first keyword is the same as the part of speech of the second keyword. Generally, it is simpler and more accurate to compare words with the same part of speech. If the part of speech of the first keyword is the same as the part of speech of the second keyword, for example, both are nouns, then the semantic similarity between the two can be determined based on the preset association relationship between word A and word B. The association relationship generally means that word A is the same as word B, word A and word B are synonyms, and word A and word B have the same subject. A mapping table of the association relationship and similarity between word A and word B can also be preset here, as shown in Table 2 below.

[0090] Table 2 Mapping table of association and similarity between word A and word B

[0091] Word A and Word B Similarity same 100％ Synonyms 50％ Same topic 0％

[0092] In combination with the above example, if the first keyword of the test image sample is football and the second keyword is football, then the semantic similarity between the first keyword and the second keyword is 100%, and the matching degree between the predicted recognition result of the test image sample and the real annotation information is 100%. For another example, if the first keyword of the test image sample is football and the second keyword is Cuju, then the semantic similarity between the first keyword and the second keyword is 50%, and the matching degree between the predicted recognition result of the test image sample and the real annotation information is 50%.

[0093] It should be understood that the order of execution of the steps in the above embodiment does not necessarily mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiment of the present invention.

[0094] In one embodiment, an image recognition model testing device is provided, and the image recognition model testing device corresponds one-to-one to the image recognition model testing method in the above embodiment. Figure 5 As shown, the image recognition model testing device includes a first acquisition module 11, a second acquisition module 12, a first calculation module 13 and a second calculation module 14. The functional modules are described in detail as follows:

[0095] A first acquisition module 11 is used to acquire a test image sample set, wherein the test image sample set includes a plurality of test image samples and a plurality of real annotation information, and one test image sample corresponds to one real annotation information;

[0096] A second acquisition module 12, used for inputting the plurality of test image samples into the image recognition model to be tested, and obtaining the predicted recognition results of the plurality of test image samples;

[0097] A first calculation module 13 is used to calculate the matching degree between the predicted recognition result and the real annotation information of each of the test image samples, and obtain the matching degree of multiple test image samples;

[0098] The second calculation module 14 is used to construct a confusion matrix according to the matching degrees of the multiple test image samples, and calculate the performance evaluation result of the image recognition model to be tested according to the confusion matrix.

[0099] In one embodiment, the real annotation information includes a real target object.

[0100] The second acquisition module 12 is further used to input the multiple test image samples into the image recognition model to be tested, and extract image feature information of each of the test image samples, wherein the image feature information includes environmental feature information and predicted target object feature information.

[0101] According to the predicted target object feature information, the predicted target object corresponding to the test image sample is identified.

[0102] The first calculation module 13 is further used to respectively calculate the matching degree between the predicted target object and the real target object of each of the test image samples to obtain the matching degree of multiple test image samples.

[0103] In one embodiment, in combination Figure 6 The second acquisition module 12 includes a fuzziness recognition unit 121, a fuzziness correction unit 122 and a recognition unit 123. For ease of description, only the parts related to the embodiment of the present invention are shown in the figure, which are described in detail as follows:

[0104] The blur degree identification unit 121 is used to identify the blur degree of each of the test image samples to obtain the blur degree of each of the test image samples.

[0105] The blur correction unit 122 is configured to perform blur correction on each of the test image samples according to the blur and based on restoration filters with different blur parameters to obtain a plurality of corrected test image samples.

[0106] The recognition unit 123 is used to input the multiple corrected test image samples into the image recognition model to be tested to obtain the predicted recognition results of the multiple test image samples.

[0107] In one embodiment, if Figure 7As shown, the first calculation unit 13 includes a feature vector construction unit 131 and a matching feature vector construction unit 132. For ease of description, only the parts related to the embodiment of the present invention are shown in the figure, which are described in detail as follows:

[0108] The feature vector construction unit 131 is used to construct a first feature value according to the predicted recognition result of the test image sample, and to construct a second feature value according to the real annotation information corresponding to the test image sample.

[0109] The matching degree feature vector construction unit 132 is used to construct a matching degree feature vector of the test image sample according to the first feature value and the second feature value.

[0110] The second calculation unit 14 is further configured to construct a confusion matrix based on the plurality of matching feature vectors. Specifically, based on the matching feature vectors of the plurality of test image samples, determine the image matching types of the plurality of test image samples, the image matching types including true positive examples, true negative examples, false positive examples, and false negative examples, count the number of each image matching type, and construct a confusion matrix based on the image matching types and their numbers.

[0111] In another embodiment, if Figure 8 As shown, the first calculation unit 13 includes a text information acquisition unit 133, a first keyword determination unit 134, a second keyword determination unit 135, and a matching degree determination unit 136. For the convenience of description, only the parts related to the embodiment of the present invention are shown in the figure, which are described in detail as follows:

[0112] The text information acquisition unit 133 is used to acquire first text information of the predicted recognition result of the test image sample and second text information of the real annotation information corresponding to the test image sample.

[0113] The first keyword determination unit 134 is configured to perform a word segmentation operation on the first text information, count the frequency of each word segmentation, and determine the first keyword of the first text information according to the statistical result.

[0114] A second keyword determination unit 135 is used to perform a word segmentation operation on the second text information, count the frequency of each word segmentation, and determine the second keyword of the second text information according to the statistical result;

[0115] The matching degree determining unit 136 is configured to calculate the similarity between the first keyword and the second keyword, and determine the matching degrees of the plurality of test image samples according to the similarity.

[0116] The matching degree determination unit 136 is further configured to determine whether the part of speech of the first keyword is the same as the part of speech of the second keyword.

[0117] If the part of speech of the first keyword is the same as the part of speech of the second keyword, the similarity of the meanings of the first keyword and the second keyword is determined.

[0118] The matching degree of the plurality of test image samples is determined according to the word meaning similarity.

[0119] For the specific definition of the image recognition model testing device, please refer to the definition of the image recognition model testing method above, which will not be repeated here. Each module in the above-mentioned image recognition model testing device can be implemented in whole or in part by software, hardware and a combination thereof. The above-mentioned modules can be embedded in or independent of the processor in the computer device in the form of hardware, or can be stored in the memory of the computer device in the form of software, so that the processor can call and execute the operations corresponding to the above modules.

[0120] In one embodiment, a computer device is provided. The computer device may be a server, and its internal structure diagram may be as follows: Fig. 9 As shown. The computer device includes a processor, a memory, a network interface and a database connected via a system bus. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a readable storage medium and an internal memory. The readable storage medium stores an operating system, computer-readable instructions and a database. The internal memory provides an environment for the operation of the operating system and computer-readable instructions in the readable storage medium. The database of the computer device is used to store data involved in the image recognition model testing method. The network interface of the computer device is used to communicate with an external terminal via a network connection. When the computer-readable instructions are executed by the processor, an image recognition model testing method is implemented. The readable storage medium provided in this embodiment includes a non-volatile readable storage medium and a volatile readable storage medium.

[0121] In one embodiment, a computer device is provided, including a memory, a processor, and computer-readable instructions stored in the memory and executable on the processor, wherein when the processor executes the computer-readable instructions, the following steps are implemented:

[0122] A test image sample set is obtained, where the test image sample set includes a plurality of test image samples and a plurality of real annotation information, and one test image sample corresponds to one real annotation information.

[0123] The multiple test image samples are input into the image recognition model to be tested to obtain predicted recognition results of the multiple test image samples.

[0124] The matching degree between the predicted recognition result and the real annotation information of each test image sample is calculated respectively to obtain the matching degree of multiple test image samples.

[0125] A confusion matrix is ​​constructed according to the matching degrees of the multiple test image samples, and a performance evaluation result of the image recognition model to be tested is calculated according to the confusion matrix.

[0126] In one embodiment, one or more computer-readable storage media storing computer-readable instructions are provided. The readable storage media provided in this embodiment include non-volatile readable storage media and volatile readable storage media. The readable storage media store computer-readable instructions. When the computer-readable instructions are executed by one or more processors, the following steps are implemented:

[0127] A test image sample set is obtained, where the test image sample set includes a plurality of test image samples and a plurality of real annotation information, and one test image sample corresponds to one real annotation information.

[0128] The multiple test image samples are input into the image recognition model to be tested to obtain predicted recognition results of the multiple test image samples.

[0129] The matching degree between the predicted recognition result and the real annotation information of each test image sample is calculated respectively to obtain the matching degree of multiple test image samples.

[0130] A confusion matrix is ​​constructed according to the matching degrees of the multiple test image samples, and a performance evaluation result of the image recognition model to be tested is calculated according to the confusion matrix.

[0131] Those of ordinary skill in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be implemented by instructing related hardware through computer-readable instructions, and the computer-readable instructions can be stored in a non-volatile readable storage medium or a volatile readable storage medium. When the computer-readable instructions are executed, they may include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, storage, database or other media used in the embodiments provided in this application may include non-volatile and / or volatile memory. Non-volatile memory may include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM) or flash memory. Volatile memory may include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in many forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link (Synchlink) DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM), etc.

[0132] Those skilled in the art can clearly understand that for the convenience and simplicity of description, only the division of the above-mentioned functional units and modules is used as an example. In actual applications, the above-mentioned functions can be distributed and completed by different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above.

[0133] The embodiments described above are only used to illustrate the technical solutions of the present invention, rather than to limit the same. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that the technical solutions described in the aforementioned embodiments may still be modified, or some of the technical features may be replaced by equivalents. Such modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention, and should all be included in the protection scope of the present invention.

Claims

1. A method for testing an image recognition model, characterized in that: include: Acquire a test image sample set, wherein the test image sample set includes multiple test image samples and multiple real annotation information, and one test image sample corresponds to one real annotation information; Inputting the plurality of test image samples into the image recognition model to be tested to obtain predicted recognition results of the plurality of test image samples; Calculating the matching degree between the predicted recognition result and the real annotation information of each test image sample respectively, and obtaining the matching degree of multiple test image samples; Constructing a confusion matrix according to the matching degrees of the multiple test image samples, and calculating a performance evaluation result of the image recognition model to be tested according to the confusion matrix; The respectively calculating the matching degree between the predicted recognition result and the real annotation information of each of the test image samples to obtain the matching degree of multiple test image samples includes: Constructing a first eigenvalue according to the predicted recognition result of the test image sample, and constructing a second eigenvalue according to the real annotation information corresponding to the test image sample; Constructing a matching degree feature vector of the test image sample according to the first eigenvalue and the second eigenvalue; The step of constructing a confusion matrix according to the matching degrees of the plurality of test image samples comprises: A confusion matrix is ​​constructed according to the plurality of matching degree feature vectors.

2. The image recognition model testing method according to claim 1, characterized in that: The real annotation information includes the real target object; The step of inputting the plurality of test image samples into the image recognition model to be tested to obtain the predicted recognition results of the plurality of test image samples comprises: Inputting the plurality of test image samples into an image recognition model to be tested, and extracting image feature information of each of the test image samples, wherein the image feature information includes environmental feature information and predicted target object feature information; identifying the predicted target object corresponding to the test image sample according to the predicted target object feature information; The respectively calculating the matching degree between the predicted recognition result and the real annotation information of each of the test image samples to obtain the matching degree of multiple test image samples includes: The matching degree between the predicted target object and the real target object of each of the test image samples is calculated respectively to obtain the matching degree of multiple test image samples.

3. The image recognition model testing method according to claim 1, characterized in that: The step of inputting the plurality of test image samples into the image recognition model to be tested to obtain the predicted recognition results of the plurality of test image samples comprises: Performing blur degree identification on each of the test image samples to obtain blur degree of each of the test image samples; According to the blurriness, performing blurriness correction on each of the test image samples based on restoration filters with different blurriness parameters to obtain a plurality of corrected test image samples; The multiple corrected test image samples are input into the image recognition model to be tested to obtain predicted recognition results of the multiple test image samples.

4. The image recognition model testing method according to claim 1, characterized in that: The step of constructing a confusion matrix according to the matching feature vectors of the plurality of test image samples comprises: According to the matching degree feature vectors of the multiple test image samples, the image matching types of the multiple test image samples are determined, the image matching types include true positive examples, true negative examples, false positive examples and false negative examples, the number of each image matching type is counted, and a confusion matrix is ​​constructed according to the image matching types and their numbers.

5. The image recognition model testing method according to claim 1, characterized in that: The respectively calculating the matching degree between the predicted recognition result and the real annotation information of each of the test image samples to obtain the matching degree of multiple test image samples includes: Acquire first text information of a predicted recognition result of the test image sample and second text information of real annotation information corresponding to the test image sample; Performing a word segmentation operation on the first text information, counting the frequency of each word segmentation, and determining a first keyword of the first text information according to the statistical result; Performing a word segmentation operation on the second text information, counting the frequency of each word segmentation, and determining a second keyword of the second text information according to the statistical result; The similarity between the first keyword and the second keyword is calculated, and the matching degree of the plurality of test image samples is determined according to the similarity.

6. The image recognition model testing method according to claim 5, characterized in that: The calculating the similarity between the first keyword and the second keyword, and determining the matching degree of the plurality of test image samples according to the similarity, comprises: Determining whether the part of speech of the first keyword and the part of speech of the second keyword are the same; If the part of speech of the first keyword is the same as the part of speech of the second keyword, determining the similarity of the meanings of the first keyword and the second keyword; The matching degree of the plurality of test image samples is determined according to the word meaning similarity.

7. An image recognition model testing device, characterized in that: include: A first acquisition module is used to acquire a test image sample set, wherein the test image sample set includes a plurality of test image samples and a plurality of real annotation information, and one test image sample corresponds to one real annotation information; A second acquisition module, used for inputting the plurality of test image samples into the image recognition model to be tested, and obtaining the predicted recognition results of the plurality of test image samples; A first calculation module is used to calculate the matching degree between the predicted recognition result and the real annotation information of each of the test image samples, so as to obtain the matching degree of multiple test image samples; A second calculation module is used to construct a confusion matrix according to the matching degree of the multiple test image samples, and calculate the performance evaluation result of the image recognition model to be tested according to the confusion matrix; The first calculation module further includes: A feature vector construction unit, configured to construct a first feature value according to the predicted recognition result of the test image sample, and to construct a second feature value according to the real annotation information corresponding to the test image sample; A matching degree feature vector construction unit, configured to construct a matching degree feature vector of a test image sample according to the first feature value and the second feature value; The second computing module is further used for: A confusion matrix is ​​constructed according to the plurality of matching degree feature vectors.

8. A computer device comprising a memory, a processor, and computer-readable instructions stored in the memory and executable on the processor, characterized in that: When the processor executes the computer-readable instructions, the image recognition model testing method according to any one of claims 1 to 6 is implemented.

9. One or more readable storage media storing computer-readable instructions, wherein when the computer-readable instructions are executed by one or more processors, the one or more processors execute the image recognition model testing method as described in any one of claims 1 to 6.

Images