Model performance information generation method and apparatus, and electronic device

Abstract

The present disclosure provides a model performance information generation method and device and electronic equipment, relates to the field of artificial intelligence, and in particular to the fields of automatic driving, intelligent traffic, etc. The specific implementation scheme is: performing road disease identification on a test image based on a target model to obtain a road disease identification result of the test image; identifying road disease identification information of M image blocks based on a road disease image region, the test image being divided into the M image blocks, M being an integer greater than 1; performing difference comparison on the road disease identification information of the M image blocks and preset road disease marking information of the M image blocks respectively, to obtain difference information of the M image blocks; and generating model performance information of the target model based on the difference information of the M image blocks. The present disclosure can improve the reliability of the finally output model performance information.

Description

Technical Field

[0001] This disclosure relates to the fields of artificial intelligence technology, such as autonomous driving and intelligent transportation, and in particular to methods, devices and electronic devices for generating model performance information. Background Technology

[0002] Currently, in the process of calculating the performance of a disease identification model, related technologies establish a ground truth set of test images in the form of detection boxes, and perform intersection over union (IOU) processing on the ground truth set and the identification set obtained by the disease identification model from the test images. Finally, the performance of the disease identification model is calculated based on the IOU result.

[0003] During the IOU processing, a pre-set threshold is used for judgment. If the overlap between the box in the identification set and the box in the truth set is greater than the threshold, it is determined that the disease identification model has identified the corresponding box in the truth set; otherwise, it is determined that the disease identification model has not identified the corresponding box in the truth set. Summary of the Invention

[0004] This disclosure provides a method, apparatus, and electronic device for generating model performance information.

[0005] According to one aspect of this disclosure, a method for generating model performance information is provided, comprising:

[0006] Based on the target model, pavement defects are identified in the test image to obtain the pavement defect identification result of the test image. The pavement defect identification result is used to represent the pavement defect image area in the test image where pavement defects exist. The target model is a model used for pavement defect identification.

[0007] Based on the pavement defect image region, pavement defect identification information is obtained by identifying M image blocks. The test image is divided into the M image blocks, where M is an integer greater than 1.

[0008] The pavement defect identification information of the M image blocks is compared with the pre-acquired pre-defined pavement defect marking information of the M image blocks to obtain the difference information of the M image blocks;

[0009] The model performance information of the target model is generated based on the difference information of the M image patches.

[0010] According to another aspect of this disclosure, a method for identifying pavement defects is provided, comprising:

[0011] Acquire the target road surface image;

[0012] The target road surface image is input into the target model for road surface defect identification to obtain the road surface defect identification result of the target road surface image. The road surface defect identification result of the target road surface image is used to represent the road surface defect image area in the target road surface image where road surface defects exist.

[0013] The target model is a model for identifying road surface defects obtained based on the model performance information, and the model performance information is model performance information generated according to a model performance information generation method provided in this disclosure.

[0014] According to another aspect of this disclosure, a model performance information generation apparatus is provided, comprising:

[0015] The first identification module is used to identify road surface defects in a test image based on a target model, and to obtain the road surface defect identification result of the test image. The road surface defect identification result is used to represent the road surface defect image area in the test image where road surface defects exist. The target model is a model used for road surface defect identification.

[0016] The second recognition module is used to recognize pavement defect recognition information of M image blocks based on the pavement defect image region. The test image is divided into the M image blocks, where M is an integer greater than 1.

[0017] The difference comparison module is used to compare the pavement defect identification information of the M image blocks with the preset pavement defect marking information of the M image blocks obtained in advance, so as to obtain the difference information of the M image blocks.

[0018] The information generation module is used to generate model performance information of the target model based on the difference information of the M image blocks.

[0019] According to another aspect of this disclosure, a road surface defect identification device is provided, comprising:

[0020] The image acquisition module is used to acquire images of the target road surface;

[0021] The road surface defect identification module is used to input the target road surface image into the target model to identify road surface defects and obtain the road surface defect identification result of the target road surface image. The road surface defect identification result of the target road surface image is used to represent the road surface defect image area in the target road surface image where road surface defects exist.

[0022] The target model is a model for identifying road surface defects obtained based on the model performance information, and the model performance information is model performance information generated according to a model performance information generation method provided in this disclosure.

[0023] According to another aspect of this disclosure, an electronic device is provided, comprising:

[0024] At least one processor; and

[0025] A memory communicatively connected to the at least one processor; wherein,

[0026] The memory stores instructions that can be executed by the at least one processor, which, when executed by the at least one processor, enables the at least one processor to perform the model performance information generation method or the pavement defect identification method provided in this disclosure.

[0027] According to another aspect of this disclosure, a non-transitory computer-readable storage medium storing computer instructions is provided, wherein the computer instructions are used to cause the computer to execute the model performance information generation method or the pavement distress identification method provided in this disclosure.

[0028] According to another aspect of this disclosure, a computer program product is provided, including a computer program that, when executed by a processor, implements the model performance information generation method or the pavement defect identification method provided in this disclosure.

[0029] It should be understood that the description in this section is not intended to identify key or essential features of the embodiments of this disclosure, nor is it intended to limit the scope of this disclosure. Other features of this disclosure will become readily apparent from the following description. Attached Figure Description

[0030] The accompanying drawings are provided to better understand this solution and do not constitute a limitation of this disclosure. Wherein:

[0031] Figure 1 This is a flowchart of a method for generating model performance information provided in this disclosure;

[0032] Figure 2 This is a schematic diagram of a test image after image block segmentation provided in this disclosure;

[0033] Figure 3 This is a schematic diagram of a defective pixel block displayed in a test image, as provided in this disclosure;

[0034] Figure 4 This is a schematic diagram of a confusion matrix provided in this disclosure;

[0035] Figure 5 This is a flowchart of a pavement distress identification method provided in this disclosure;

[0036] Figure 6 This is a flowchart of a method for constructing a truth set provided in this publication;

[0037] Figures 7a to 7e This is a structural diagram of a model performance information generation device provided in this disclosure;

[0038] Figure 8 This is a structural diagram of a road surface defect identification device disclosed herein;

[0039] Figure 9 This is a block diagram of an electronic device used to implement the model performance information generation method or road surface defect identification method of the embodiments of this disclosure. Detailed Implementation

[0040] The exemplary embodiments of this disclosure are described below with reference to the accompanying drawings, including various details of the embodiments to aid understanding, and should be considered merely exemplary. Therefore, those skilled in the art will recognize that various changes and modifications can be made to the embodiments described herein without departing from the scope and spirit of this disclosure. Similarly, for clarity and brevity, descriptions of well-known functions and structures are omitted in the following description.

[0041] Please see Figure 1 , Figure 1 This is a flowchart of a method for generating model performance information provided in this disclosure, such as... Figure 1 As shown, it includes the following steps:

[0042] Step S101: Based on the target model, identify road surface defects in the test image to obtain the road surface defect identification results of the test image.

[0043] The pavement defect identification result is used to represent the pavement defect image region in the test image where pavement defects exist, and the target model is a model used for pavement defect identification.

[0044] The test image is any image in the test set, which is a collection of images formed by multiple pre-collected road surface images. The test image carries or corresponds to image annotation information, which is used to indicate whether there are road surface defects on the road surface indicated by the test image, and, if there are road surface defects on the road surface indicated by the test image, the number, type, and extent of the road surface defects.

[0045] In some implementations, the image annotation information may also represent the road surface category indicated by the test image and the scene in which the image was taken.

[0046] For example, the road surface type can be: asphalt road surface or cement road surface; the image shooting scene can be one of the following: tunnel entry / exit scene, evening scene, normal scene, backlight scene, shadow scene, or mottled scene.

[0047] By subdividing the road surface categories and image capture scenarios, a more applicable test set is constructed, which improves the realism of the test set to actual road surface image acquisition scenarios and makes the generated model performance information more reliable.

[0048] In some implementations, during the construction of the test set, an initial set can be constructed first based on the test data provided by the R&D personnel of the target model. Then, based on the differences in the test data corresponding to different road surface categories and / or the differences in the test data corresponding to different image shooting scenarios in the initial set, the initial set can be supplemented with data to make the number of test data for different road surface categories in the set more consistent, and to make the number of test data for different image shooting scenarios in the set more consistent, so as to obtain a more versatile test set.

[0049] For example, in the initial set, the ratio of the number of test data corresponding to asphalt pavement to the number of test data corresponding to cement pavement can be 1:10, while in the test set, the ratio of the number of test data corresponding to asphalt pavement to the number of test data corresponding to cement pavement can be 1:1.

[0050] Similarly, in the initial set, the ratio of test data for the tunnel scene to test data for the evening scene can be 2:7, while in the test set, the ratio of test data for the tunnel scene to test data for the evening scene can be 1:1.

[0051] Step S102: Identify the pavement defect identification information of M image blocks based on the pavement defect image region.

[0052] The test image is divided into M image blocks, where M is an integer greater than 1.

[0053] In some implementations, the test image can be divided into image blocks according to a pre-set standard image block format to obtain the M image blocks.

[0054] During the segmentation process, the resulting image blocks can be numbered in order from left to right and from top to bottom, and the sum of the areas of the M image blocks is set to equal the area of ​​the test image.

[0055] For example, such as Figure 2 As shown, if the test image (such as...) Figure 2 The width and height of the image blocks (shown in the solid-line box) are both 10. The width and height of the image blocks indicated by the standard image block format are 3 and 2 respectively. Therefore, the test image can be divided into 20 image blocks (e.g., ...). Figure 2(As shown in the dashed box in the middle), where the width and height of the 15 image blocks are 3 and 2 respectively, and the width and height of the 5 image blocks located at the edge are 1 and 2 respectively. The size of the 5 image blocks at the edge is cropped to prevent the area indicated by the image blocks from going beyond the edge of the test image.

[0056] In one embodiment, after the image blocks of the test image are divided, image blocks whose regions overlap with the road surface defect image regions in at least a portion can be identified as defective image blocks with road surface defects, and the defect type corresponding to the road surface defect image region can be taken as the defect type of the defective image block; while image blocks whose regions do not overlap with the road surface defect image regions in the entirety can be identified as background image blocks of normal road surfaces.

[0057] When the pavement defect identification information indicates that the corresponding image block is a defective image block, the pavement defect identification information can further indicate the defect type of the corresponding image block.

[0058] For example, the above-mentioned types of defects include: cracks, fissures, pits, subsidence, etc.

[0059] Step S103: Compare the pavement defect identification information of the M image blocks with the preset pavement defect marking information of the M image blocks obtained in advance to obtain the difference information of the M image blocks.

[0060] As mentioned above, the preset pavement defect marking information of the M image blocks can be obtained based on the image annotation information associated with the test image. The preset pavement defect marking information is used to indicate whether there is pavement defect in the corresponding image block, and further indicates the defect type of the corresponding image block if there is pavement defect.

[0061] By comparing the pavement defect identification information and the preset pavement defect marking information of each of the M image blocks, the difference information of the M image blocks can be obtained.

[0062] For example, the difference information includes first data indicating that the pavement defect identification information of the image block matches the preset pavement defect marking information, or second data indicating that the pavement defect identification information of the image block does not match the preset pavement defect marking information.

[0063] Among them, the mismatch between the pavement defect identification information of the indicator image block and the preset pavement defect marking information can be:

[0064] The case where the pavement defect identification information indicator image block is a background image block, while the preset pavement defect marking information indicator image block is a defect image block;

[0065] or,

[0066] The case where the pavement defect identification information indicator image block is a defect image block, while the preset pavement defect marking information indicator image block is a background image block;

[0067] or,

[0068] The road surface defect identification information indicator image block is the defect image block corresponding to the first defect type, while the preset road surface defect marking information indicator image block is the defect image block corresponding to the second defect type.

[0069] Step S104: Generate model performance information of the target model based on the difference information of the M image blocks.

[0070] The above model performance information is used to represent the target model's ability to identify road surface defects in the test image.

[0071] For example, the difference information of the M image patches can be processed by normalization to represent the model performance information of the target model by outputting a performance score; alternatively, the recognition accuracy of the target model on the test image can be calculated by the difference information of the M image patches, and the recognition accuracy can be converted into a performance level (such as low, medium, high, or first level, second level, etc.) according to a pre-set accuracy level rule to represent the model performance information of the target model by the performance level. This disclosure does not limit the specific representation form of the model performance information.

[0072] In this disclosure, the test image is divided into multiple image blocks, and the pavement defect identification information of multiple image blocks is compared with the preset pavement defect marking information, which replaces the comparison method of Intersection over Union (IOU) processing used in related technologies. This avoids the interference of human factors introduced in the threshold setting stage during IOU processing and improves the reliability of the final output model performance information.

[0073] In one embodiment, the model performance information is used for at least one of the following: evaluating the target model and debugging the target model.

[0074] In this embodiment, by applying model performance information to the evaluation and / or debugging process of the target model, the model performance information can be applied to more generalized scenarios, which can improve the applicability of the model performance information.

[0075] The evaluation of the target model using application model performance information can be as follows:

[0076] The model performance information is used as the evaluation result of the target model, and the target model is judged to meet the actual application requirements based on the evaluation result.

[0077] For example, when the model performance information is a normalized performance score (the higher the performance score, the better the target model's recognition effect on the test image), if the performance score corresponding to the evaluation result is greater than the preset evaluation threshold, then the target model is determined to meet the actual application requirements; if the performance score corresponding to the evaluation result is less than or equal to the preset evaluation threshold, then the target model is determined to not meet the actual application requirements.

[0078] The target model can be debugged using the application model performance information as follows:

[0079] When the model performance information is a normalized performance score, the performance score corresponding to the model performance information is compared with a pre-set debugging threshold. If the performance score is greater than the debugging threshold, it is determined that the parameter adjustment direction of the target model is correct (or the debugging process of the target model is ended); if the performance score is less than or equal to the debugging threshold, it is determined that the parameter adjustment direction of the target model is incorrect (or the debugging process of the target model still needs to continue).

[0080] In one embodiment, after comparing the pavement defect identification information of the M image blocks with the pre-acquired preset pavement defect marking information of the M image blocks to obtain the difference information of the M image blocks, the method further includes:

[0081] Based on the preset pavement defect marking information of the M image blocks, defective image blocks are identified in the M image blocks, and the defective image blocks are image blocks with pavement defects;

[0082] The display mode of the disease image block is determined based on the difference information of the disease image block. The display mode includes a first mode or a second mode. The first mode is used to indicate that the road surface disease identification information and the preset road surface disease marking information of the disease image block are the same. The second mode is used to indicate that the road surface disease identification information and the preset road surface disease marking information of the disease image block are different.

[0083] The disease image block is displayed in the test image according to the display mode of the disease image block.

[0084] In this embodiment, by displaying disease image blocks in the test image and distinguishing between disease image blocks correctly identified by the target model and disease image blocks incorrectly identified by the target model through different display models, the recognition effect of the target model on the test image can be conveniently and intuitively displayed, improving the visualization effect of model performance information and reducing the pressure of model performance information on storage space.

[0085] Wherein, the pavement defect identification information and the preset pavement defect marking information of the defect image block are the same, which can be: the defect type indicated by the pavement defect identification information and the defect type indicated by the preset pavement defect marking information are the same.

[0086] The difference between the pavement defect identification information and the preset pavement defect marking information in the defect image block can be: the defect type indicated by the pavement defect identification information is different from the defect type indicated by the preset pavement defect marking information, or the pavement defect identification information indicates that the defect image block does not have pavement defects.

[0087] For example, it can be as follows Figure 3 As shown, Figure 3 The solid-lined box represents the test image. Figure 3 The dashed boxes in the middle represent disease image blocks. Among them, the dashed box numbered 1 indicates the disease image block displayed in the first mode, and the dashed box numbered 2 indicates the disease image block displayed in the second mode.

[0088] In one example, the first mode can be a green highlight display mode, and the second mode can be a red highlight display mode; in another example, the first mode can be a solid display mode, and the second mode can be a hollow display mode; it should be noted that other ways of representing the first mode and the second mode can also be selected in the application based on actual needs, and this disclosure does not limit the specific representation of the first mode and the second mode.

[0089] In one embodiment, generating model performance information of the target model based on the difference information of the M image patches includes:

[0090] Based on the difference information of the M image blocks, a confusion matrix is ​​generated between the pavement defect identification information of the M image blocks and the preset pavement defect marking information of the M image blocks;

[0091] The model performance information of the target model is generated based on the confusion matrix, wherein the model performance information includes at least one of precision, recall, accuracy and F1 score, and the F1 score is used to represent the harmonic mean of the accuracy and the recall.

[0092] In this embodiment, by generating a confusion matrix, it is not only convenient to calculate multiple indicators such as accuracy, recall, precision and F1 score of model performance information, but also to intuitively display the difference information of M image blocks, thereby improving the visualization effect of difference information.

[0093] For example, based on the difference information of the M image blocks, a confusion matrix can be generated to obtain pavement defect identification information of the M image blocks and preset pavement defect marking information of the M image blocks as follows: Figure 4 As shown.

[0094] In one embodiment, generating the model performance information of the target model based on the difference information of the M image patches further includes:

[0095] Based on the difference information of the M image blocks and the image block mapping information of the pre-acquired test image, the disease identification data of the target model is determined;

[0096] The image block mapping information is used to indicate the N road surface defect markers included in the test image, and the road surface defect markers corresponding to the image blocks containing road surface defects among the M image blocks; the defect identification data is used to indicate the road surface defect markers identified by the target model among the N road surface defect markers, and the model performance information includes the defect identification data, where N is a positive integer.

[0097] In this embodiment, by setting the image block mapping information, the road surface defect markers identified by the target model in the N road surface defect markers can be determined accordingly. This can complement the defect image blocks identified by the target model in the multiple defect image blocks, thereby improving the richness of the model performance information.

[0098] The image block mapping information can be obtained based on the aforementioned image annotation information after the test image is divided into M image blocks. At this time, the aforementioned image annotation information not only indicates the road surface defect areas included in the test image and the defect types corresponding to the road surface defect areas, but also indicates the road surface defect identifier to which the road surface defect areas belong.

[0099] For example, for a certain test image, the image annotation information corresponding to the test image may include: defect area one, defect area two, and defect area three existing in the test image, wherein defect area one is marked as crack number one, corresponding to the road surface defect of the crack type; defect area two is marked as crack number two, corresponding to the road surface defect of the crack type; and defect area three is marked as pothole number one, corresponding to the road surface defect of the pothole type. In this example, "crack number one", "crack number two", and "pothole number one" can be understood as the aforementioned road surface defect markings.

[0100] By obtaining the aforementioned confusion matrix, the final output model performance information can include the defect image blocks identified by the target model among the multiple defect image blocks. That is, it represents the number and quantity of defect image blocks correctly identified by the target model among the multiple defect image blocks included in the test image. Furthermore, by setting the image block mapping information, this embodiment can supplement the defect identification data in the final output model performance information. That is, it represents the number and identification of the road defect markers correctly identified by the target model among the N road defect markers included in the test image.

[0101] In one example, the target model can identify the following disease image blocks among the multiple disease image blocks: If there are 5 disease image blocks numbered 11, 12, 13, 50, and 51 in the test image, where the 3 disease image blocks numbered 11, 12, and 13 correspond to the disease area of ​​"Crack 1", and the 2 disease image blocks numbered 50 and 51 correspond to the disease area of ​​"Pit 1", the target model correctly identifies the 3 disease image blocks numbered 11, 12, and 13, and incorrectly identifies the 2 disease image blocks numbered 50 and 51.

[0102] In this example, the pavement defect identifiers identified by the target model among the N pavement defect identifiers can be: the target model correctly identifies the defect area of ​​"Crack 1" and incorrectly identifies the defect area of ​​"Pothole 1".

[0103] It should be noted that, in the case where a road surface defect marker corresponds to P1 defect image blocks, if the target model correctly identifies the defect image block P2 in the area corresponding to the road surface defect marker, and the ratio of P2 / P1 is greater than or equal to a preset threshold, then the target model is determined to have correctly identified the road surface defect marker, where P2 is an integer less than or equal to P1, and P1 is a positive integer.

[0104] In one embodiment, the process of identifying pavement defect identification information from M image blocks based on the pavement defect image region includes:

[0105] The set of marker coordinates of the test image is obtained based on the pavement defect image area. The set of marker coordinates is used to indicate the point markers of multiple pixels in the test image. The point markers of the pixels include one of the following: a first marker indicating no defects, a second marker indicating a first type of defect, and a third marker indicating a second type of defect.

[0106] When a target image block includes pixels corresponding to road surface defects, the defect type corresponding to the set of defect points with the largest number of elements in the target image block is taken as the defect type corresponding to the target image block. Here, the target image block is any one of the M image blocks, the defect point set is the set of pixels in the target image block that correspond to the same defect type, and the road surface defect identification information of the image block includes the defect type corresponding to the image block.

[0107] In this embodiment, when the image block includes pixels corresponding to road surface defects, by counting the number of elements in the set of pixels corresponding to each defect type in the image block, and taking the defect type corresponding to the set of pixels with the largest number of elements as the defect type corresponding to the image block, the road surface defect identification information of the determined image block can be made more accurate.

[0108] For example, if an image block does not include the pixels corresponding to the disease, then the image block is a background image block;

[0109] If an image block contains 3 pixels corresponding to the crack type and 50 pixels corresponding to the undamaged state, then the image block is a defective image block corresponding to the crack type.

[0110] If an image block contains 3 pixels corresponding to the crack type, 4 pixels corresponding to the pit type, and 50 pixels corresponding to the undamaged condition, then the image block corresponds to a pit-type defect image block.

[0111] In one embodiment, obtaining the set of marker coordinates of the test image based on the pavement distress image region includes:

[0112] The test image is masked and filled based on the road surface defect image region to obtain the mask image of the test image;

[0113] Generate a set of marker coordinates for the test image based on the mask image.

[0114] In this embodiment, the conversion from road surface defect image area to pixel point is conveniently completed by mask filling, which replaces the manual conversion method. This not only improves the efficiency of obtaining the marker coordinate set, but also improves the accuracy of the marker coordinate set.

[0115] For example, when the test image indicates that there are multiple types of road surface defects in the road surface defect image area, the test image can be filled with multiple types of road surface defects one by one, and the filling form or content is different, so as to facilitate the subsequent marking of the coordinates of each pixel in the mask image.

[0116] For example, if the test image contains both crack-type and pothole-type pavement defects, the crack-type pavement defects in the test image can be filled with a first mask to obtain a first image, and then the pothole-type pavement defects in the first image can be filled with a second mask to obtain the mask image.

[0117] Please see Figure 5 , Figure 5 This is a flowchart of a pavement distress identification method provided in this disclosure, such as... Figure 5 As shown, it includes the following steps:

[0118] Step S501: Obtain the target road surface image.

[0119] The target road surface image can be an image input to the target model during the model application stage. The target road surface image can be a road surface image uploaded to the cloud server by the user in a smart transportation scenario, or a road surface image captured by a monitoring camera according to a preset sampling period in a road surface monitoring scenario. This disclosure does not limit the specific source of the target road surface image.

[0120] Step S501: Input the target road surface image into the target model to identify road surface defects, and obtain the road surface defect identification result of the target road surface image.

[0121] Wherein, the pavement defect identification result of the target pavement image is used to represent the pavement defect image area in the target pavement image where pavement defects exist, and the target model is a model for pavement defect identification obtained based on the model performance information, wherein the model performance information is model performance information generated according to the model performance information generation method provided in this disclosure.

[0122] In this disclosure, the application of model performance information generation method for debugging and evaluating pavement defect identification model can obtain a target model with better pavement defect identification effect. Based on the target model, pavement defect identification of target pavement image can make the pavement defect identification result of the target pavement image more reliable.

[0123] It should be noted that, as Figure 6 As shown, in one embodiment, the annotation results (i.e., image annotation information) of each image (i.e., test image) in the annotation file (i.e., the file where the test set is located) can be obtained by traversal. When the annotation result indicates that the number of diseases in the image is not 0, the original image path of the image is queried based on the image name, and the original image of the image is read by OpenCV software.

[0124] The image's height and width are obtained accordingly. After determining the grid size (grid_size), the number of rows and columns the image can be divided into is calculated. The grids (i.e., image patches) are numbered sequentially from 0 to 1, from left to right and from top to bottom. All grids are traversed, and the coordinates of the top-left and bottom-right corners are calculated based on the grid number. It is then checked whether the bottom-right corner coordinates are out of bounds; if so, the image's height and width are used as the bottom-right corner coordinates. Finally, a dictionary of all grid positions (grids) is obtained, thus completing the gridding of the image.

[0125] The algorithm iterates through all the lesions in the read image and performs inverse normalization based on the image's height and width to obtain the coordinates of each pixel in the image. Then, it performs mask filling on each lesion region according to the lesion label type to obtain a lesion mask image. In the lesion mask image, pixels that do not correspond to a lesion are marked as 0 (i.e., key is 0), and pixels that correspond to a lesion are marked as integers greater than 0.

[0126] Based on the aforementioned location dictionary, all grids are traversed, and the number of pixels of each disease type in each grid is counted. Then, pixels with key 0 (background) are removed, and the set of pixels with the largest number of pixels of the same disease type in the grid is taken as the grid type corresponding to the grid. Thus, the construction of the truth set of the image (that is, the preset road surface disease marking information of the M image blocks of the test image) is completed.

[0127] Please see Figure 7a , Figure 7a This disclosure provides a model performance information generation device, such as... Figure 7a As shown, the model performance information generation device 700 includes:

[0128] The first identification module 701 is used to identify road surface defects in a test image based on a target model, and obtain the road surface defect identification result of the test image. The road surface defect identification result is used to represent the road surface defect image area in the test image where road surface defects exist. The target model is a model used for road surface defect identification.

[0129] The second identification module 702 is used to identify pavement defect identification information of M image blocks based on the pavement defect image region, wherein the test image is divided into the M image blocks, and M is an integer greater than 1;

[0130] The difference comparison module 703 is used to compare the pavement defect identification information of the M image blocks with the preset pavement defect marking information of the M image blocks obtained in advance, and obtain the difference information of the M image blocks.

[0131] The information generation module 704 is used to generate model performance information of the target model based on the difference information of the M image blocks.

[0132] In one embodiment, the model performance information is used for at least one of the following: evaluating the target model and debugging the target model.

[0133] In one embodiment, such as Figure 7b As shown, the device 700 further includes a disease display module 705, which is used for:

[0134] Based on the preset pavement defect marking information of the M image blocks, defective image blocks are identified in the M image blocks, and the defective image blocks are image blocks with pavement defects;

[0135] The display mode of the disease image block is determined based on the difference information of the disease image block. The display mode includes a first mode or a second mode. The first mode is used to indicate that the road surface disease identification information and the preset road surface disease marking information of the disease image block are the same. The second mode is used to indicate that the road surface disease identification information and the preset road surface disease marking information of the disease image block are different.

[0136] The disease image block is displayed in the test image according to the display mode of the disease image block.

[0137] In one embodiment, such as Figure 7c As shown, the information generation module 704 includes:

[0138] The matrix generation unit 7041 is used to generate a confusion matrix of pavement defect identification information and preset pavement defect marking information of the M image blocks based on the difference information of the M image blocks.

[0139] Information generation unit 7042 is used to generate model performance information of the target model based on the confusion matrix, wherein the model performance information includes at least one of precision, recall, accuracy and F1 score, and the F1 score is used to represent the harmonic mean of the accuracy and the recall.

[0140] In one embodiment, such as Figure 7d As shown, the information generation module 704 further includes:

[0141] The data determination unit 7043 is used to determine the disease identification data of the target model based on the difference information of the M image blocks and the image block mapping information of the test image obtained in advance;

[0142] The image block mapping information is used to indicate the N road surface defect markers included in the test image, and the road surface defect markers corresponding to the image blocks containing road surface defects among the M image blocks; the defect identification data is used to indicate the road surface defect markers identified by the target model among the N road surface defect markers, and the model performance information includes the defect identification data, where N is a positive integer.

[0143] In one embodiment, such as Figure 7e As shown, the second identification module 702 includes:

[0144] The set acquisition unit 7021 is used to acquire a set of marker coordinates of the test image based on the pavement defect image area. The set of marker coordinates is used to indicate the point markers of multiple pixels in the test image. The point markers of the pixels include one of a first marker indicating no defects, a second marker indicating a first type of defect, and a third marker indicating a second type of defect.

[0145] The type identification unit 7022 is used to identify the disease type corresponding to the set of disease points with the largest number of elements in the target image block as the disease type corresponding to the target image block when the target image block includes pixels corresponding to road surface defects. The target image block is any one of the M image blocks, the set of disease points is the set of pixels in the target image block that correspond to the same disease type, and the road surface defect identification information of the image block includes the disease type corresponding to the image block.

[0146] In one embodiment, the set acquisition unit 7021 is used for:

[0147] The test image is masked and filled based on the road surface defect image region to obtain the mask image of the test image;

[0148] Generate a set of marker coordinates for the test image based on the mask image.

[0149] The model performance information generation device provided in this disclosure can realize all the processes of the model performance information generation method provided in this disclosure and achieve the same technical effect. To avoid repetition, it will not be described in detail here.

[0150] Please see Figure 8 , Figure 8 This disclosure provides a road surface defect identification device, such as... Figure 8 As shown, the road surface defect identification device 800 includes:

[0151] Image acquisition module 801 is used to acquire images of the target road surface;

[0152] The defect identification module 802 is used to input the target road surface image into the target model to identify road surface defects and obtain the road surface defect identification result of the target road surface image. The road surface defect identification result of the target road surface image is used to represent the road surface defect image area in the target road surface image where road surface defects exist.

[0153] The target model is a model for identifying road surface defects obtained based on the model performance information, and the model performance information is model performance information generated according to the model performance information generation method provided in this disclosure.

[0154] The pavement defect identification device provided in this disclosure can realize all the processes of the pavement defect identification method provided in this disclosure and achieve the same technical effect. To avoid repetition, it will not be described in detail here.

[0155] According to embodiments of this disclosure, this disclosure also provides an electronic device, a readable storage medium, and a computer program product.

[0156] Figure 9 A schematic block diagram of an example electronic device 900 that can be used to implement embodiments of the present disclosure is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device may also represent various forms of mobile devices, such as personal digital processors, cellular phones, smartphones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely illustrative and are not intended to limit the implementation of the present disclosure described and / or claimed herein.

[0157] like Figure 9 As shown, device 900 includes a computing unit 901, which can perform various appropriate actions and processes based on a computer program stored in read-only memory (ROM) 902 or a computer program loaded from storage unit 908 into random access memory (RAM) 903. RAM 903 may also store various programs and data required for the operation of device 900. The computing unit 901, ROM 902, and RAM 903 are interconnected via bus 904. Input / output (I / O) interface 905 is also connected to bus 904.

[0158] Multiple components in device 900 are connected to I / O interface 905, including: input unit 906, such as keyboard, mouse, etc.; output unit 907, such as various types of monitors, speakers, etc.; storage unit 908, such as disk, optical disk, etc.; and communication unit 909, such as network card, modem, wireless transceiver, etc. Communication unit 909 allows device 900 to exchange information / data with other devices through computer networks such as the Internet and / or various telecommunications networks.

[0159] The computing unit 901 can be various general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of the computing unit 901 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special-purpose artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, digital signal processors (DSPs), and any suitable processor, controller, microcontroller, etc. The computing unit 901 performs the various methods and processes described above, such as model performance information generation or pavement defect identification methods. For example, in some embodiments, the model performance information generation or pavement defect identification method can be implemented as a computer software program tangibly contained in a machine-readable medium, such as storage unit 908. In some embodiments, part or all of the computer program can be loaded and / or installed on device 900 via ROM 902 and / or communication unit 909. When the computer program is loaded into RAM 903 and executed by the computing unit 901, one or more steps of the model performance information generation or pavement defect identification method described above can be performed. Alternatively, in other embodiments, the computing unit 901 may be configured by any other suitable means (e.g., by means of firmware) to perform a model performance information generation or pavement distress identification method.

[0160] Various embodiments of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), systems-on-chip (SoCs), complex programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments may include implementations in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which may be a dedicated or general-purpose programmable processor, capable of receiving data and instructions from a storage system, at least one input device, and at least one output device, and transmitting data and instructions to the storage system, the at least one input device, and the at least one output device.

[0161] The program code used to implement the methods of this disclosure may be written in any combination of one or more programming languages. This program code may be provided to a processor or controller of a general-purpose computer, special-purpose computer, or other programmable data processing apparatus, such that when executed by the processor or controller, the program code causes the functions / operations specified in the flowcharts and / or block diagrams to be implemented. The program code may be executed entirely on a machine, partially on a machine, as a standalone software package partially on a machine and partially on a remote machine, or entirely on a remote machine or server.

[0162] In the context of this disclosure, a machine-readable medium can be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, apparatus, or device. A machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium can be, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

[0163] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device for displaying information to the user (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor); and a keyboard and pointing device (e.g., a mouse or trackball) through which the user provides input to the computer. Other types of devices can also be used to provide interaction with the user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including sound input, voice input, or tactile input).

[0164] The systems and technologies described herein can be implemented in computing systems that include backend components (e.g., as a data server), or computing systems that include middleware components (e.g., an application server), or computing systems that include frontend components (e.g., a user computer with a graphical user interface or web browser through which a user can interact with embodiments of the systems and technologies described herein), or any combination of such backend, middleware, or frontend components. The components of the system can be interconnected via digital data communication of any form or medium (e.g., a communication network). Examples of communication networks include local area networks (LANs), wide area networks (WANs), and the Internet.

[0165] Computer systems can include clients and servers. Clients and servers are generally located far apart and typically interact via communication networks. Client-server relationships are created by computer programs running on the respective computers and having a client-server relationship with each other. Servers can be cloud servers, servers in distributed systems, or servers incorporating blockchain technology.

[0166] It should be understood that the various forms of processes shown above can be used to rearrange, add, or delete steps. For example, the steps described in this disclosure can be executed in parallel, sequentially, or in different orders, as long as the desired result of the technical solution disclosed in this disclosure can be achieved, and this is not limited herein.

[0167] The specific embodiments described above do not constitute a limitation on the scope of protection of this disclosure. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this disclosure should be included within the scope of protection of this disclosure.

Claims

1. A method for generating model performance information, comprising: Based on the target model, pavement defects are identified in the test image to obtain the pavement defect identification result of the test image. The pavement defect identification result is used to represent the pavement defect image area in the test image where pavement defects exist. The target model is a model used for pavement defect identification. Based on the pavement defect image region, pavement defect identification information is obtained by identifying M image blocks. The test image is divided into the M image blocks, where M is an integer greater than 1. The pavement defect identification information of the M image blocks is compared with the pre-acquired pre-defined pavement defect marking information of the M image blocks to obtain the difference information of the M image blocks; The model performance information of the target model is generated based on the difference information of the M image patches; The process of generating model performance information for the target model based on the difference information of the M image patches includes: Based on the difference information of the M image blocks, a confusion matrix is ​​generated between the pavement defect identification information of the M image blocks and the preset pavement defect marking information of the M image blocks. The confusion matrix is ​​used to display the difference information between the pavement defect identification information and the preset pavement defect marking information. The difference information is the difference information between the defect types of the corresponding image blocks indicated by the pavement defect identification information and the preset pavement defect marking information. The model performance information of the target model is generated based on the confusion matrix, wherein the model performance information includes at least one of precision, recall, accuracy and F1 score, and the F1 score is used to represent the harmonic mean of the accuracy and the recall.

2. The method according to claim 1, wherein, The model performance information is used for at least one of the following: evaluating the target model, or debugging the target model.

3. The method according to claim 1, wherein, After comparing the pavement defect identification information of the M image blocks with the pre-acquired preset pavement defect marking information of the M image blocks to obtain the difference information of the M image blocks, the method further includes: Based on the preset pavement defect marking information of the M image blocks, defective image blocks are identified in the M image blocks, and the defective image blocks are image blocks with pavement defects; The display mode of the disease image block is determined based on the difference information of the disease image block. The display mode includes a first mode or a second mode. The first mode is used to indicate that the road surface disease identification information and the preset road surface disease marking information of the disease image block are the same. The second mode is used to indicate that the road surface disease identification information and the preset road surface disease marking information of the disease image block are different. The disease image block is displayed in the test image according to the display mode of the disease image block.

4. The method according to claim 1, wherein, The step of generating model performance information for the target model based on the difference information of the M image patches further includes: Based on the difference information of the M image blocks and the image block mapping information of the pre-acquired test image, the disease identification data of the target model is determined; The image block mapping information is used to indicate the N road surface defect markers included in the test image, and the road surface defect markers corresponding to the image blocks containing road surface defects among the M image blocks; the defect identification data is used to indicate the road surface defect markers identified by the target model among the N road surface defect markers, and the model performance information includes the defect identification data, where N is a positive integer.

5. The method according to any one of claims 1 to 4, wherein, The road surface defect identification information based on the identification of M image blocks in the road surface defect image region includes: The set of marker coordinates of the test image is obtained based on the pavement defect image area. The set of marker coordinates is used to indicate the point markers of multiple pixels in the test image. The point markers of the pixels include one of the following: a first marker indicating no defects, a second marker indicating a first type of defect, and a third marker indicating a second type of defect. When a target image block includes pixels corresponding to road surface defects, the defect type corresponding to the set of defect points with the largest number of elements in the target image block is taken as the defect type corresponding to the target image block. Here, the target image block is any one of the M image blocks, the defect point set is the set of pixels in the target image block that correspond to the same defect type, and the road surface defect identification information of the image block includes the defect type corresponding to the image block.

6. The method according to claim 5, wherein, The step of obtaining the set of marked coordinates of the test image based on the pavement distress image region includes: The test image is masked and filled based on the road surface defect image region to obtain the mask image of the test image; Generate a set of marker coordinates for the test image based on the mask image.

7. A method for identifying pavement defects, comprising: Acquire the target road surface image; The target road surface image is input into the target model for road surface defect identification to obtain the road surface defect identification result of the target road surface image. The road surface defect identification result of the target road surface image is used to represent the road surface defect image area in the target road surface image where road surface defects exist. The target model is a model for identifying road surface defects obtained based on the model performance information, and the model performance information is the model performance information generated by the method described in any one of claims 1 to 6.

8. A model performance information generation device, comprising: The first identification module is used to identify road surface defects in a test image based on a target model, and to obtain the road surface defect identification result of the test image. The road surface defect identification result is used to represent the road surface defect image area in the test image where road surface defects exist. The target model is a model used for road surface defect identification. The second recognition module is used to recognize pavement defect recognition information of M image blocks based on the pavement defect image region. The test image is divided into the M image blocks, where M is an integer greater than 1. The difference comparison module is used to compare the pavement defect identification information of the M image blocks with the preset pavement defect marking information of the M image blocks obtained in advance, so as to obtain the difference information of the M image blocks. An information generation module is used to generate model performance information of the target model based on the difference information of the M image patches; The information generation module includes: A matrix generation unit is used to generate a confusion matrix of pavement defect identification information and preset pavement defect marking information of the M image blocks based on the difference information of the M image blocks. The confusion matrix is ​​used to display the difference information between the pavement defect identification information and the preset pavement defect marking information. The difference information is the difference information between the defect types of the corresponding image blocks indicated by the pavement defect identification information and the preset pavement defect marking information. An information generation unit is configured to generate model performance information of the target model based on the confusion matrix, wherein the model performance information includes at least one of precision, recall, accuracy and F1 score, and the F1 score is used to represent the harmonic mean of the accuracy and the recall.

9. The apparatus according to claim 8, wherein, The model performance information is used for at least one of the following: evaluating the target model, or debugging the target model.

10. The apparatus according to claim 8, wherein, The device further includes a lesion display module, which is used for: Based on the preset pavement defect marking information of the M image blocks, defective image blocks are identified in the M image blocks, and the defective image blocks are image blocks with pavement defects; The display mode of the disease image block is determined based on the difference information of the disease image block. The display mode includes a first mode or a second mode. The first mode is used to indicate that the road surface disease identification information and the preset road surface disease marking information of the disease image block are the same. The second mode is used to indicate that the road surface disease identification information and the preset road surface disease marking information of the disease image block are different. The disease image block is displayed in the test image according to the display mode of the disease image block.

11. The apparatus according to claim 8, wherein, The information generation module further includes: The data determination unit is used to determine the disease identification data of the target model based on the difference information of the M image blocks and the image block mapping information of the test image obtained in advance. The image block mapping information is used to indicate the N road surface defect markers included in the test image, and the road surface defect markers corresponding to the image blocks containing road surface defects among the M image blocks; the defect identification data is used to indicate the road surface defect markers identified by the target model among the N road surface defect markers, and the model performance information includes the defect identification data, where N is a positive integer.

12. The apparatus according to any one of claims 8 to 11, wherein, The second identification module includes: The set acquisition unit is used to acquire a set of marker coordinates of the test image based on the pavement defect image area, wherein the set of marker coordinates is used to indicate point markers of multiple pixels in the test image, and the point markers of the pixels include one of a first marker indicating no defects, a second marker indicating a first type of defect, and a third marker indicating a second type of defect; A type identification unit is used to identify the type of damage corresponding to the set of damage points with the largest number of elements in the target image block when the target image block includes pixels corresponding to road surface defects. The target image block is any one of the M image blocks, the set of damage points is the set of pixels in the target image block that corresponds to the same type of damage, and the road surface defect identification information of the image block includes the type of damage corresponding to the image block.

13. The apparatus according to claim 12, wherein, The set acquisition unit is used for: The test image is masked and filled based on the road surface defect image region to obtain the mask image of the test image; Generate a set of marker coordinates for the test image based on the mask image.

14. A pavement defect identification device, comprising: The image acquisition module is used to acquire images of the target road surface; The road surface defect identification module is used to input the target road surface image into the target model to identify road surface defects and obtain the road surface defect identification result of the target road surface image. The road surface defect identification result of the target road surface image is used to represent the road surface defect image area in the target road surface image where road surface defects exist. The target model is a model for identifying road surface defects obtained based on the model performance information, and the model performance information is the model performance information generated by the method described in any one of claims 1 to 6.

15. An electronic device comprising: At least one processor; as well as A memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor to enable the at least one processor to perform the method of any one of claims 1 to 7.

16. A non-transitory computer-readable storage medium storing computer instructions, wherein, The computer instructions are used to cause the computer to perform the method according to any one of claims 1 to 7.

17. A computer program product comprising a computer program that, when executed by a processor, implements the method of any one of claims 1 to 7.

Images