Method and device for generating a generalized small sample road anomaly detection model

Abstract

Embodiments of the present disclosure disclose a method and device for generating a generalized small sample road anomaly detection model. A specific embodiment of the method includes: performing base class and new class division on a data set; performing base class matching on each new class data; performing initial round training to obtain an initial base class loss value; based on a preset round, performing the following training steps: inputting the base class and new class matching pair into an initial road anomaly detection model to obtain a current round base class loss value; determining a dynamic weighting operator; determining a current round total loss value; performing parameter updating on the initial road anomaly detection model to obtain an updated road anomaly model; if the training round is equal to the preset round, determining the updated road anomaly model as the road anomaly detection model; if the training round is less than the preset round, performing the training steps again on the updated road anomaly model as the initial road anomaly detection model. The embodiment can generate a road anomaly detection model and improve the accuracy of road anomaly detection.

Description

Technical Field

[0001] The embodiments disclosed herein relate to the field of computer technology, and more specifically to a method and apparatus for generating a generalized small-sample road surface anomaly detection model. Background Technology

[0002] The occurrence of road surface anomalies such as potholes, damaged road markings, and obstacles poses a serious threat to traffic safety. Furthermore, the occurrence of road surface anomalies is both accidental and relatively scarce, resulting in a relative scarcity of road surface anomaly data. Currently, the common approach used to generate models for road surface anomaly detection is the few-shot object detection method. This method often assumes that the dataset contains abundant base class data and scarce new class data. By utilizing the base class data, the model can learn to locate and identify the rarely labeled new class of road surface anomalies.

[0003] However, when using the above method to generate road surface anomaly detection models, the following technical problems often arise:

[0004] Small sample object detection methods only evaluate the model's performance in detecting new classes, while ignoring the performance of basic classes. This can easily lead to false positives and false negatives, resulting in poor robustness in detecting road anomalies and consequently, low accuracy in detecting road anomalies.

[0005] The information disclosed in this background section is only intended to enhance the understanding of the background of the present disclosure concept, and therefore may contain information that does not constitute prior art known to those skilled in the art. Summary of the Invention

[0006] The summary portion of this disclosure is intended to provide a brief overview of the concepts, which will be described in detail in the detailed description portion. This summary portion is not intended to identify key or essential features of the claimed technical solutions, nor is it intended to limit the scope of the claimed technical solutions.

[0007] Some embodiments of this disclosure propose a method and apparatus for generating a generalized small-sample road surface anomaly detection model to solve one or more of the technical problems mentioned in the background section above.

[0008] In a first aspect, some embodiments of this disclosure provide a method for generating a generalized small-sample road anomaly detection model. The method includes: dividing a preset road anomaly dataset into base classes and new classes to obtain a base class dataset and a new class dataset; performing base class matching on each new class data in the new class dataset based on the base class dataset to obtain a set of base class and new class matching pairs; inputting the set of base class and new class matching pairs into an initial road anomaly detection model for initial training rounds to obtain an initial base class loss value; and, based on a preset number of rounds, performing the following model training steps: inputting the set of base class and new class matching pairs into the initial road anomaly detection model to obtain the base class loss value and the new class loss value for the current round. Loss value; Based on the initial base class loss value and the current round base class loss value, determine the dynamic trade-off operator; Based on the dynamic trade-off operator, the current round base class loss value, and the current round new class loss value, determine the current round total loss value; Based on the current round total loss value, update the backpropagation parameters of the initial road anomaly detection model to obtain an updated road anomaly detection model; In response to determining that the training round is equal to the preset round, determine the updated road anomaly detection model as the road anomaly detection model; In response to determining that the training round is less than the preset round, use the updated road anomaly detection model as the initial road anomaly detection model and execute the above model training steps again.

[0009] Secondly, some embodiments of this disclosure provide a device for generating a generalized small-sample road anomaly detection model. The device includes: a base class and new class partitioning unit configured to partition a preset road anomaly dataset into base classes and new classes, obtaining a base class dataset and a new class dataset; a base class matching unit configured to perform base class matching on each new class data in the new class dataset based on the base class dataset, obtaining a set of base class and new class matching pairs; an initial training round unit configured to input the set of base class and new class matching pairs into an initial road anomaly detection model for initial training rounds, obtaining an initial base class loss value; and a model training unit configured to perform the following model training steps based on a preset number of rounds: inputting the set of base class and new class matching pairs into the initial road anomaly detection model. In the detection model, the base class loss value and the new class loss value for the current round are obtained; based on the initial base class loss value and the current round base class loss value, a dynamic trade-off operator is determined; based on the dynamic trade-off operator, the current round base class loss value, and the current round new class loss value, the total loss value for the current round is determined; based on the current round total loss value, the backpropagation parameters of the initial road anomaly detection model are updated to obtain an updated road anomaly detection model; in response to determining that the training round is equal to the preset round, the updated road anomaly detection model is determined as the road anomaly detection model; in response to determining that the training round is less than the preset round, the updated road anomaly detection model is used as the initial road anomaly detection model, and the above model training steps are executed again.

[0010] Thirdly, some embodiments of this disclosure provide an electronic device, including: one or more processors; and a storage device having one or more programs stored thereon, wherein when the one or more programs are executed by the one or more processors, the one or more processors implement the method described in any implementation of the first aspect above.

[0011] Fourthly, some embodiments of this disclosure provide a computer-readable medium having a computer program stored thereon, wherein the program, when executed by a processor, implements the method described in any of the implementations of the first aspect above.

[0012] The above embodiments of this disclosure have the following beneficial effects: the generation method of the generalized small-sample road anomaly detection model in some embodiments of this disclosure can improve the model's detection accuracy for road anomalies. Specifically, the reason for the low detection accuracy of the model is that small-sample target detection methods only evaluate the model's performance in detecting new classes, while ignoring the performance of base classes, which easily leads to false detections and false negatives, resulting in poor robustness for road anomaly detection. Based on this, the generation method of the generalized small-sample road anomaly detection model in some embodiments of this disclosure first divides the preset road anomaly dataset into base classes and new classes, obtaining base class datasets and new class datasets. The large number of labeled base class datasets helps the model learn general features, enabling the model to better extract features and classify them from the small amount of new classes. Second, based on the base class dataset, base class matching is performed on each new class data in the new class dataset to obtain a set of base class and new class matching pairs. Thus, the model can mine the combined information of base classes and new classes, endowing the model with more common and individual knowledge of categories on limited data. Then, the set of matching pairs between the base class and the new class is input into the initial road surface anomaly detection model for initial training rounds to obtain the initial base class loss value. Finally, based on the preset rounds, the following model training steps are performed: the set of matching pairs between the base class and the new class is input into the initial road surface anomaly detection model to obtain the base class loss value and the new class loss value for the current round; a dynamic trade-off operator is determined based on the initial base class loss value and the base class loss value for the current round; the total loss value for the current round is determined based on the dynamic trade-off operator, the base class loss value for the current round, and the new class loss value for the current round; the backpropagation parameters of the initial road surface anomaly detection model are updated based on the total loss value for the current round to obtain the updated road surface anomaly detection model; in response to determining that the training rounds are equal to the preset rounds, the updated road surface anomaly detection model is determined as the road surface anomaly detection model; in response to determining that the training rounds are less than the preset rounds, the updated road surface anomaly detection model is used as the initial road surface anomaly detection model to perform the above model training steps again. By using a dynamic tradeoff operator to achieve adaptive parameter adjustment during model training, the model's detection performance for both base class and new class targets is improved, thereby enhancing the model's robustness and ultimately increasing the model's accuracy in detecting road anomalies. Attached Figure Description

[0013] The above and other features, advantages, and aspects of the embodiments of this disclosure will become more apparent from the accompanying drawings and the following detailed description. Throughout the drawings, the same or similar reference numerals denote the same or similar elements. It should be understood that the drawings are schematic, and elements are not necessarily drawn to scale.

[0014] Figure 1This is a flowchart of some embodiments of the method for generating a generalized small-sample road surface anomaly detection model according to the present disclosure;

[0015] Figure 2 This is a visualization diagram of road anomaly detection based on some embodiments of the method for generating a generalized small-sample road anomaly detection model according to this disclosure;

[0016] Figure 3 These are schematic diagrams of some embodiments of the device for generating a generalized small-sample road surface anomaly detection model according to this disclosure.

[0017] Figure 4 This is a schematic diagram of the structure of an electronic device suitable for implementing some embodiments of the present disclosure. Detailed Implementation

[0018] Embodiments of this disclosure will now be described in more detail with reference to the accompanying drawings. While some embodiments of this disclosure are shown in the drawings, it should be understood that this disclosure can be implemented in various forms and should not be construed as limited to the embodiments set forth herein. Rather, these embodiments are provided to provide a more thorough and complete understanding of this disclosure. It should be understood that the accompanying drawings and embodiments of this disclosure are for illustrative purposes only and are not intended to limit the scope of protection of this disclosure.

[0019] It should also be noted that, for ease of description, only the parts relevant to the invention are shown in the accompanying drawings. Unless otherwise specified, the embodiments and features described in this disclosure can be combined with each other.

[0020] It should be noted that the concepts of "first" and "second" mentioned in this disclosure are used only to distinguish different devices, modules or units, and are not used to limit the order of functions performed by these devices, modules or units or their interdependencies.

[0021] It should be noted that the terms "a" and "a plurality of" used in this disclosure are illustrative rather than restrictive, and those skilled in the art should understand that, unless otherwise expressly indicated in the context, they should be understood as "one or more".

[0022] The names of messages or information exchanged between multiple devices in the embodiments of this disclosure are for illustrative purposes only and are not intended to limit the scope of such messages or information.

[0023] This disclosure will now be described in detail with reference to the accompanying drawings and embodiments.

[0024] Figure 1 A flowchart 100 is shown, illustrating some embodiments of a method for generating a generalized small-sample road anomaly detection model according to this disclosure. The method for generating the generalized small-sample road anomaly detection model includes the following steps:

[0025] Step 101: Divide the preset road anomaly dataset into base class and new class to obtain base class dataset and new class dataset.

[0026] In some embodiments, the executing entity can divide a preset road anomaly dataset into base class and new class datasets to obtain base class datasets and new class datasets.

[0027] Optionally, the above road surface anomaly dataset is constructed using the following steps:

[0028] The first step is to obtain a sample road surface anomaly dataset. This dataset includes sample road surface anomaly images and corresponding sample road surface anomaly label groups. Here, each sample road surface anomaly image contains at least one road surface anomaly region. Next, the sample road surface anomaly label groups include sample road surface anomaly labels corresponding to each anomaly region. Typically, these labels consist of a rectangular frame defining the anomaly region and a category label indicating the type of road surface anomaly.

[0029] In practice, open-source road anomaly datasets with abundant data and high-quality annotations can be selected as sample road anomaly datasets.

[0030] As an example, an open-source road anomaly dataset may include, but is not limited to, at least one of the following: Road Damage Detector (RDD) dataset and Lost and Found dataset.

[0031] The second step involves performing data augmentation operations on each sample road surface anomaly data point in the aforementioned sample road surface anomaly dataset to obtain an augmented road surface anomaly dataset. This augmentation can be achieved through operations such as image rotation, image flipping, image cropping, and color transformation, resulting in an augmented road surface anomaly image set. Here, one sample road surface anomaly image can correspond to multiple augmented road surface anomaly images. Simultaneously, the same image rotation, image flipping, and image cropping operations are performed on the sample road surface anomaly label groups corresponding to the sample road surface anomaly images to obtain an augmented road surface anomaly label group set. Finally, the aforementioned augmented road surface anomaly image set and the corresponding augmented road surface anomaly label group set are combined to form the augmented road surface anomaly dataset.

[0032] The third step is to define the aforementioned enhanced road surface anomaly dataset as a road surface anomaly dataset. This dataset includes road surface anomaly images and corresponding road surface anomaly label groups. Here, the road surface anomaly label groups include road surface anomaly labels corresponding to each anomaly region. Typically, these labels consist of a rectangular bounding box defining the anomaly region and a category label indicating the type of anomaly. Specifically, the enhanced road surface anomaly image corresponding to each enhanced road surface anomaly data point in the dataset can be defined as a road surface anomaly image. Next, the enhanced road surface anomaly label groups corresponding to the enhanced road surface anomaly data are defined as road surface anomaly label groups.

[0033] In practice, data augmentation on sample road surface anomaly datasets can improve the generalization ability of trained models.

[0034] In some optional implementations of certain embodiments, the execution entity divides the preset road anomaly dataset into base class and new class datasets to obtain base class datasets and new class datasets, which may include the following steps:

[0035] The first step is to determine the road anomaly categories corresponding to each road anomaly data point in the aforementioned road anomaly dataset, thus obtaining a road anomaly category set. Specifically, each road anomaly label in the road anomaly label group corresponding to each road anomaly data point in the aforementioned road anomaly dataset corresponds to a road anomaly category in the road anomaly category set. Here, one road anomaly data point corresponds to at least one road anomaly category.

[0036] For example, a road surface anomaly data may simultaneously contain road surface anomaly categories such as "potholes", "damaged zebra crossings", and "obstacles".

[0037] The second step is to determine the category data volume of each road anomaly category in the aforementioned road anomaly category set based on the road anomaly label corresponding to each road anomaly data in the aforementioned road anomaly dataset, thus obtaining the category data volume set. Specifically, the category data volume of each road anomaly category in the aforementioned road anomaly category set can be determined by the number of each road anomaly label in the road anomaly label group corresponding to each road anomaly data in the aforementioned road anomaly dataset. For example, the number of road anomaly labels with the label "pothole" is the category data volume of the road anomaly category "pothole".

[0038] The third step involves identifying the road anomaly categories whose data volume is greater than or equal to a preset threshold within the aforementioned category data set, thus determining the basic categories and obtaining the basic category set. The preset threshold can be 10,000. Here, the basic categories are the road anomaly categories with the largest data volume in the road anomaly dataset.

[0039] In practice, road anomaly categories with more than 10,000 data points were selected from the RDD dataset as base categories. These categories have sufficient data to support adequate model training. Because the Lost and Found dataset has fewer than 500 samples in each category, making it relatively sparse, six data categories from the RDD dataset were uniformly selected as base classes.

[0040] The fourth step involves identifying new road anomaly categories whose data volume falls below a preset threshold within the aforementioned category dataset. These new categories are the newly discovered road anomaly categories. Specifically, they are road anomaly categories with relatively small data volumes within the road anomaly dataset.

[0041] In practice, categories with scarce data are chosen as new categories. These categories have limited data and require transfer learning from the base class. Both RDD and Lost and Found datasets selected four new classes. However, to ensure the reliability and stability of the experimental results, some implementations in this disclosure follow the data partitioning approach used in few-shot learning, setting three different dataset partitioning schemes for both datasets. That is, three classes are selected from the four new categories, forming three different data subsets. The final experimental results are averaged across these three subsets to reduce the randomness of the results.

[0042] The fifth step involves identifying the road anomaly data corresponding to the aforementioned basic categories in the road anomaly dataset as base class data, thus obtaining the base class dataset. Specifically, based on the road anomaly labels corresponding to each road anomaly data in the road anomaly dataset, the road anomaly data corresponding to each basic category and its corresponding road anomaly label are used as base class data, resulting in the base class dataset. Here, the base class dataset is a collection of road anomaly data corresponding to each basic category in the road anomaly dataset.

[0043] Step 6: Identify the road anomaly data in the aforementioned road anomaly dataset that corresponds to the newly added category as the new category data, thus obtaining the new category dataset. Specifically, based on the road anomaly labels corresponding to each road anomaly data in the road anomaly dataset, the road anomaly data corresponding to the road anomaly labels and the newly added category are used as the new category data, resulting in the new category dataset. Here, the new category dataset is a collection of road anomaly data in the road anomaly dataset that corresponds to the newly added category.

[0044] In practice, the following technical problems have been found when using a two-stage training strategy of pre-training and fine-tuning:

[0045] Models pre-trained on base classes may treat new classes as background, and such recognition errors are difficult to correct in subsequent fine-tuning stages, thus reducing the model's detection accuracy for new classes. A training strategy combining base class and new class matching pairs can fully leverage the advantages of both category-related and category-independent knowledge, leading to the following solution.

[0046] Step 102: Based on the base class dataset, perform base class matching on each new class data in the new class dataset to obtain a set of matching pairs between the base class and the new class;

[0047] In some embodiments, the execution entity may perform base class matching on each new class data in the new class dataset to obtain a set of base class and new class matching pairs.

[0048] In some optional implementations of certain embodiments, the execution entity performs base class matching on each new class data in the new class dataset to obtain a set of base class and new class matching pairs, which may include the following steps:

[0049] The first step is to match each new category in the newly added category set with each base category in the aforementioned base category set to generate a set of base class and new category pairs. Each base class and new category pair is a combination of a new category and a base category. The set of base class and new category pairs contains all possible combinations of new and base categories.

[0050] The second step involves allocating data to each base class and new class category pair in the aforementioned base class and new class category pair sets according to a preset ratio, resulting in a base class and new class matching pair set. Specifically, based on the base class and new class category pairs, a preset number of base class data and new class data are randomly sampled from the base class dataset and new class dataset respectively, according to a preset ratio, to obtain base class data subsets and new class data subsets. These base class data subsets and new class data subsets are then combined to form base class and new class matching pairs.

[0051] In practice, assume that the "Road Transverse Cracks" class in the base class and the "Manhole Cover Damage" class in the new class form a base class-new class pair. We can randomly sample eight base class data points (corresponding to "Road Transverse Cracks") from the base class dataset and eight new class data points (corresponding to "Manhole Cover Damage") from the new class dataset, forming a base class-new class pair of size 16, with 16 representing the size of the mini-batch during model training. This ensures that the number of base class and new class pairs is equal in each round, thus avoiding data imbalance.

[0052] Step 102 and its related content, as an inventive point of this disclosure, solve the aforementioned technical problem of "low model detection accuracy for new classes." Factors leading to this problem often include: a model pre-trained on base classes may treat new classes as background, and such recognition errors are difficult to correct in subsequent fine-tuning stages. Solving these factors can improve the model's detection accuracy for new classes. To achieve this, firstly, all class combinations are traversed to construct all possible base class and new class pairs. This fully leverages both category-related and category-independent knowledge, improving the ability to recognize new class data. Next, a balanced ratio of base class and new class matching pairs is constructed. This ensures that the number of base classes and new classes is equal in each round, thus avoiding data imbalance. Simultaneously, mini-batch training with base class and new class matching pairs allows the model to better escape local optima, thereby accelerating model convergence. Therefore, the above steps can improve the model's detection accuracy for new classes.

[0053] Step 103: Input the set of matching pairs between the base class and the new class into the initial road surface anomaly detection model for the initial rounds of training to obtain the initial base class loss value.

[0054] In some embodiments, the execution entity can input the set of matching pairs between the base class and the new class into the initial road surface anomaly detection model for initial training rounds to obtain the initial base class loss value.

[0055] In some optional implementations of certain embodiments, the execution entity inputs the base class and new class matching pair set into the initial road surface anomaly detection model for initial training rounds to obtain the initial base class loss value, which may include the following steps:

[0056] The first step is to determine the initial parameters of the aforementioned initial road surface anomaly detection model. This initial road surface anomaly detection model is a pre-defined target detection model. Here, the initial parameters of the initial road surface anomaly detection model can be determined using the parameters of a publicly available pre-trained model, or they can be set to random numbers.

[0057] As an example, the preset object detection model may include, but is not limited to, at least one of the following: Faster R-CNN (Faster Regions-Convolutional Neural Network), SSD (SingleShot MultiBox Detector), etc.

[0058] In practice, some embodiments of this disclosure are a general network model learning strategy for training network models, independent of the selection of network frameworks.

[0059] The second step involves sequentially inputting each base class and new class matching pair from the aforementioned base class and new class matching pair set into the initial road surface anomaly detection model to obtain the base class detection result set. Specifically, the base class dataset corresponding to each base class and new class matching pair is input into the initial road surface anomaly detection model to obtain the base class detection result set. Here, the base class detection result is the road surface anomaly detection result made by the initial road surface anomaly detection model for road surface anomaly images in the base class data. Furthermore, the base class detection result includes at least one bounding box defining the road surface anomaly region and the corresponding road surface anomaly result category.

[0060] The third step is to determine the loss value between each base class detection result in the aforementioned base class detection result set and the corresponding road surface anomaly label group in the aforementioned base class dataset, thus obtaining the initial base class loss value. Here, the initial base class loss value represents the loss value obtained by the model for each base class data after the initial training rounds. This loss value can include classification loss and regression loss. Here, the classification loss value between the road surface anomaly result category and the true category label can be determined using cross-entropy loss. Then, the regression loss value between the result bounding box and the true bounding box can be determined using smooth L1 loss.

[0061] In practice, the initial base class loss value is represented as l b (θ 0 Where l represents the loss value, and b is the identifier of the base class. b This represents the base class loss value. θ represents the model parameters. 0 This represents the parameters of the initial round model.

[0062] Step 104: Based on the preset number of rounds, perform the following model training steps:

[0063] Step 1041: Input the base class and new class matching pairs into the initial road anomaly detection model to obtain the base class loss value and the new class loss value for the current round. Specifically, inputting the base class and new class matching pairs into the initial road anomaly detection model can be achieved through the loss value determination step in step 103, which yields the base class loss value and the new class loss value for the current round. This will not be elaborated further here. Here, the aforementioned base class loss value for the current round represents the loss value obtained by the model after one round of training for each base class data. Similarly, the aforementioned new class loss value for the current round represents the loss value obtained by the model after one round of training for each new class data.

[0064] Step 1042: Determine the dynamic trade-off operator based on the initial base class loss value and the current round base class loss value.

[0065] In some embodiments, the execution entity may determine the dynamic trade-off operator based on the initial base class loss value and the current round base class loss value.

[0066] In some optional implementations of certain embodiments, the execution entity determines the dynamic tradeoff operator based on the initial base class loss value and the current round base class loss value, which may include the following steps:

[0067] The first step is to perform a hyperbolic tangent transform on the initial base class loss value and the current round base class loss value to obtain the current round base class training level value. This current round base class training level value represents the degree of base class training as measured by the change in the base class training loss value. Here, the current round base class training level value can be obtained by performing a hyperbolic tangent transform on the initial base class loss value and the current round base class loss value using the following formula:

[0068]

[0069] Where t represents the current training epoch, and p represents the training level value. b This represents the training level of the base class. This represents the training level of the base class in the current t-th round. tanh() represents the hyperbolic tangent function. b (θ t ) represents the base class loss value in the current round t.

[0070] In practice, the loss value generally decreases with the increase of training epochs. Therefore, the difference between the initial base class loss value and the current base class loss value can reflect the training level of the model. To more accurately quantify the training level of the base class in the current epoch, the initial base class loss value and the current base class loss value are normalized using the hyperbolic tangent function and then divided. This result reflects the training level of the base class in the current epoch.

[0071] The second step is to determine the dynamic trade-off operator based on the base class training level values ​​for the current training round. The dynamic trade-off operator, determined by the base class training level values, is a numerical guide for model training. The dynamic trade-off operator can be determined using the following formula based on the base class training level values ​​for the current training round:

[0072]

[0073] Where λ represents the dynamic trade-off operator. t This represents the dynamic tradeoff operator in the current round t. α is a parameter used in multi-task learning to control the sensitivity of the tradeoff operator to changes in training progress, typically ranging from 0.5 to 2.

[0074] Step 1043: Determine the total loss value for the current round based on the dynamic trade-off operator, the base class loss value for the current round, and the new class loss value for the current round. The total loss value for the current round can be determined using the following formula:

[0075] l main =λ t l n (θ t )+l b (θ t )

[0076] Among them, l main This represents the total loss value. 'n' is the identifier for the new class (new). n (θ t ) represents the loss value of the new class in the current round t.

[0077] Step 1044: Based on the total loss value of the current round, update the parameters of the initial road anomaly detection model using backpropagation to obtain the updated road anomaly detection model. The gradients of each layer of the initial road anomaly detection model can be determined using the backpropagation algorithm. Then, based on the gradients of each layer, update the parameters of the initial road anomaly detection model to obtain the updated road anomaly detection model.

[0078] Step 1045: In response to determining that the training rounds equal the preset rounds, the updated road surface anomaly detection model is determined as the road surface anomaly detection model. Specifically, if the training rounds have already reached the preset rounds, the aforementioned updated road surface anomaly detection model is used as the road surface anomaly detection model.

[0079] Step 1046: In response to determining that the number of training rounds is less than the preset number of rounds, the updated road surface anomaly detection model is used as the initial road surface anomaly detection model, and the above model training steps are executed again. Specifically, if the number of training rounds has not reached the preset number of rounds, the updated road surface anomaly detection model is used as the initial road surface anomaly detection model, and the above model training steps are executed again.

[0080] Optionally, after step 104 above, the following steps are also included:

[0081] The first step is to acquire an image of the road surface anomaly to be tested. This image can be a photograph taken from a traffic scene or a frame from a traffic surveillance video.

[0082] The second step involves inputting the aforementioned road surface anomaly image into the aforementioned road surface anomaly detection model to obtain a road surface anomaly detection result image. The aforementioned road surface anomaly detection model is generated by steps 101-104. Furthermore, the aforementioned road surface anomaly detection result image includes bounding boxes labeling the road surface anomaly areas and the corresponding road surface anomaly categories.

[0083] In practice, a visual illustration of the above-mentioned road anomaly detection model and the current advanced algorithm TFA-cos (Two-stage Fine-tuning Approach-cosine similarity, a two-stage fine-tuning algorithm based on cosine similarity classifier) ​​for road anomaly detection is shown below. Figure 2 As shown. Among them, Figure 2 Each column corresponds to the test sample image used in the test experiment. The first row is the test sample image and its corresponding ground truth (GT) label. The second row is the road anomaly detection result image output by the Ours w / DLW (Ours with Dynamic Lexicographic Weighting) road anomaly detection model, including the road anomaly region annotation box and the corresponding road anomaly category. The third row is the TFA-cos road anomaly detection result image output by the TFA-cos model. It can be seen that the road anomaly detection model (Ours w / DLW) generated by some embodiments of this disclosure can detect road anomalies that TFA-cos does not detect, with fewer false positives and false negatives, and the detection results are closer to the ground truth. This indicates that the road anomaly detection model generated by some embodiments of this disclosure has higher accuracy.

[0084] The above embodiments of this disclosure have the following beneficial effects: the generation method of the generalized small-sample road anomaly detection model in some embodiments of this disclosure can improve the detection accuracy of the road anomaly detection model. Specifically, the reason for the low detection accuracy of the model is that the small-sample target detection method only evaluates the performance of the model in detecting new classes, while ignoring the performance of the base classes, which easily leads to false detections and false negatives, resulting in poor robustness for road anomaly detection. Based on this, the generation method of the generalized small-sample road anomaly detection model in some embodiments of this disclosure first divides the preset road anomaly dataset into base classes and new classes to obtain base class datasets and new class datasets. The large number of labeled base class datasets helps the model learn general features, enabling the model to better extract features and classify new classes from small datasets. Second, based on the base class dataset, base class matching is performed on each new class data in the new class dataset to obtain a set of base class and new class matching pairs. Thus, the model can mine the combined information of base classes and new classes, endowing the model with more common and individual knowledge of categories on limited data. Then, the set of matching pairs between the base class and the new class is input into the initial road surface anomaly detection model for initial training rounds to obtain the initial base class loss value. Finally, based on the preset rounds, the following model training steps are performed: the set of matching pairs between the base class and the new class is input into the initial road surface anomaly detection model to obtain the base class loss value and the new class loss value for the current round; a dynamic trade-off operator is determined based on the initial base class loss value and the base class loss value for the current round; the total loss value for the current round is determined based on the dynamic trade-off operator, the base class loss value for the current round, and the new class loss value for the current round; the backpropagation parameters of the initial road surface anomaly detection model are updated based on the total loss value for the current round to obtain the updated road surface anomaly detection model; in response to determining that the training rounds are equal to the preset rounds, the updated road surface anomaly detection model is determined as the road surface anomaly detection model; in response to determining that the training rounds are less than the preset rounds, the updated road surface anomaly detection model is used as the initial road surface anomaly detection model to perform the above model training steps again. By using dynamic trade-off operators to achieve adaptive parameter adjustment during model training, the model's detection performance for both base class and new class targets is improved, thereby enhancing the model's robustness and ultimately increasing the detection accuracy of the road anomaly detection model.

[0085] Further reference Figure 3 As an implementation of the methods shown in the above figures, this disclosure provides some embodiments of a device for generating a generalized small-sample road anomaly detection model. These device embodiments are similar to... Figure 1 Corresponding to the method embodiments shown, the device can be specifically applied to various electronic devices.

[0086] like Figure 3As shown, the generalized small-sample road anomaly detection model generation device 300 of some embodiments includes: a base class and new class partitioning unit 301, a base class matching unit 302, an initial round training unit 303, and a model training unit 304. The base class and new class partitioning unit 301 is configured to partition a preset road anomaly dataset into base classes and new classes, obtaining a base class dataset and a new class dataset; the base class matching unit 302 is configured to perform base class matching on each new class data in the new class dataset based on the base class dataset, obtaining a set of base class and new class matching pairs; the initial round training unit 303 is configured to input the set of base class and new class matching pairs into the initial road anomaly detection model for initial round training, obtaining an initial base class loss value; the model training unit 304 is configured to perform the following model training steps based on a preset round: inputting the set of base class and new class matching pairs into the initial road anomaly detection model to obtain the base class for the current round. The initial base class loss value and the new class loss value in the current round are used to determine the dynamic trade-off operator. Based on the dynamic trade-off operator, the current base class loss value, and the new class loss value in the current round, the total loss value in the current round is determined. Based on the total loss value in the current round, the backpropagation parameters of the initial road anomaly detection model are updated to obtain an updated road anomaly detection model. In response to determining that the training round is equal to a preset round, the updated road anomaly detection model is identified as the road anomaly detection model. In response to determining that the training round is less than a preset round, the updated road anomaly detection model is used as the initial road anomaly detection model, and the above model training steps are executed again.

[0087] It is understandable that the units described in the device 300 are related to the reference. Figure 1 The steps in the described method correspond accordingly. Therefore, the operations, features, and beneficial effects described above for the method also apply to the device 300 and the units contained therein, and will not be repeated here.

[0088] The following is for reference. Figure 4 It shows a schematic diagram of the structure of an electronic device (e.g., a computing device) 400 suitable for implementing some embodiments of the present disclosure. Figure 4 The electronic device shown is merely an example and should not be construed as limiting the functionality and scope of the embodiments of this disclosure.

[0089] like Figure 4As shown, the electronic device 400 may include a processing unit (e.g., a central processing unit, a graphics processing unit, etc.) 401, which can perform various appropriate actions and processes according to a program stored in a read-only memory 402 or a program loaded from a storage device 408 into a random access memory 403. The random access memory 403 also stores various programs and data required for the operation of the electronic device 400. The processing unit 401, the read-only memory 402, and the random access memory 403 are interconnected via a bus 404. An input / output interface 405 is also connected to the bus 404.

[0090] Typically, the following devices can be connected to I / O interface 405: input devices 406 including, for example, touchscreens, touchpads, keyboards, mice, cameras, microphones, accelerometers, gyroscopes, etc.; output devices 407 including, for example, liquid crystal displays (LCDs), speakers, vibrators, etc.; storage devices 408 including, for example, magnetic tapes, hard disks, etc.; and communication devices 409. Communication device 409 allows electronic device 400 to communicate wirelessly or wiredly with other devices to exchange data. Although Figure 4 An electronic device 400 with various devices is shown; however, it should be understood that it is not required to implement or possess all of the devices shown. More or fewer devices may be implemented or possessed alternatively. Figure 4 Each box shown can represent a device or multiple devices as needed.

[0091] In particular, according to some embodiments of this disclosure, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, some embodiments of this disclosure include a computer program product comprising a computer program carried on a computer-readable medium, the computer program containing program code for performing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via a communication device 409, or installed from a storage device 408, or installed from a read-only memory 402. When the computer program is executed by the processing device 401, it performs the functions defined in the methods of some embodiments of this disclosure.

[0092] It should be noted that, in some embodiments of this disclosure, the computer-readable medium may be a computer-readable signal medium or a computer-readable storage medium, or any combination thereof. A computer-readable storage medium may be, for example,—but not limited to—an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of a computer-readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination thereof. In some embodiments of this disclosure, a computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, apparatus, or device. In some embodiments of this disclosure, a computer-readable signal medium may include a data signal propagated in baseband or as part of a carrier wave, carrying computer-readable program code. Such propagated data signals may take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. A computer-readable signal medium can be any computer-readable medium other than a computer-readable storage medium, which can send, propagate, or transmit a program for use by or in connection with an instruction execution system, apparatus, or device. The program code contained on the computer-readable medium can be transmitted using any suitable medium, including but not limited to: wires, optical fibers, RF (radio frequency), etc., or any suitable combination thereof.

[0093] In some implementations, clients and servers can communicate using any currently known or future-developed network protocol such as HTTP (Hypertext Transfer Protocol) and can interconnect with digital data communication (e.g., communication networks) of any form or medium. Examples of communication networks include local area networks (“LANs”), wide area networks (“WANs”), the Internet (e.g., the Internet of Things), and end-to-end networks (e.g., ad hoc end-to-end networks), as well as any currently known or future-developed networks.

[0094] The aforementioned computer-readable medium may be included within the aforementioned electronic device; or it may exist independently and not assembled into the electronic device. The aforementioned computer-readable medium carries one or more programs, which, when executed by the electronic device, cause the electronic device to: divide a preset road anomaly dataset into base classes and new classes, obtaining base class datasets and new class datasets; perform base class matching on each new class data in the new class dataset based on the base class dataset, obtaining a set of base class and new class matching pairs; input the set of base class and new class matching pairs into an initial road anomaly detection model for initial training rounds, obtaining an initial base class loss value; and, based on a preset number of rounds, perform the following model training steps: input the set of base class and new class matching pairs into the initial road anomaly detection model, obtaining the current round base class loss value and the current round's loss value. The following steps are performed: 1. Calculate the loss value for the new class in the current round; 2. Based on the initial base class loss value and the current round base class loss value, determine the dynamic trade-off operator; 3. Based on the dynamic trade-off operator, the current round base class loss value, and the current round new class loss value, determine the total loss value for the current round; 4. Update the backpropagation parameters of the initial road anomaly detection model based on the current round total loss value to obtain an updated road anomaly detection model; 5. In response to determining that the training round is equal to a preset round, determine the updated road anomaly detection model as the road anomaly detection model; 6. In response to determining that the training round is less than a preset round, use the updated road anomaly detection model as the initial road anomaly detection model and repeat the above model training steps.

[0095] Computer program code for performing operations of some embodiments of this disclosure can be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, and C++, and conventional procedural programming languages ​​such as the "C" language or similar programming languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving remote computers, the remote computer can be connected to the user's computer via any type of network—including a local area network (LAN) or a wide area network (WAN)—or can be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0096] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of this disclosure. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, can be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0097] The units described in some embodiments of this disclosure can be implemented in software or hardware. The described units can also be housed in a processor; for example, a processor may be described as including a base class and new class partitioning unit, a base class matching unit, an initial round training unit, and a model training unit. The names of these units do not necessarily limit the unit itself; for example, the base class matching unit may also be described as "a unit that performs base class matching on new classes."

[0098] The functions described above in this document can be performed, at least in part, by one or more hardware logic components. For example, exemplary types of hardware logic components that can be used, without limitation, include: Field Programmable Gate Arrays (FPGAs), Application-Specific Integrated Circuits (ASICs), Application Standard Products (ASSPs), System-on-Chip (SoCs), Complex Programmable Logic Devices (CPLDs), and so on.

[0099] The above description is merely a selection of preferred embodiments of this disclosure and an explanation of the technical principles employed. Those skilled in the art should understand that the scope of the invention involved in the embodiments of this disclosure is not limited to technical solutions formed by specific combinations of the above-described technical features, but should also cover other technical solutions formed by arbitrary combinations of the above-described technical features or their equivalents without departing from the above-described inventive concept. For example, technical solutions formed by substituting the above-described features with (but not limited to) technical features with similar functions disclosed in the embodiments of this disclosure.

Claims

1. A method for generating a generalized small-sample road surface anomaly detection model, comprising: The pre-defined road anomaly dataset is divided into base class and new class to obtain base class dataset and new class dataset; Based on the base class dataset, perform base class matching on each new class data in the new class dataset to obtain a set of base class and new class matching pairs; The set of matching pairs between the base class and the new class is input into the initial road surface anomaly detection model for the initial rounds of training to obtain the initial base class loss value. Based on the preset number of rounds, the following model training steps are performed: Input the set of matching pairs between the base class and the new class into the initial road surface anomaly detection model to obtain the base class loss value and the new class loss value for the current round. Based on the initial base class loss value and the current round base class loss value, determine the dynamic trade-off operator; The total loss value for the current round is determined based on the dynamic trade-off operator, the base class loss value for the current round, and the new class loss value for the current round. Based on the total loss value of the current round, the backpropagation parameters of the initial road surface anomaly detection model are updated to obtain the updated road surface anomaly detection model; In response to determining that the training rounds are equal to the preset rounds, the updated road surface anomaly detection model is determined as the road surface anomaly detection model; In response to the determination that the number of training rounds is less than the preset number of rounds, the updated road surface anomaly detection model is used as the initial road surface anomaly detection model to perform the model training step again.

2. The method according to claim 1, wherein, The road surface anomaly dataset is constructed using the following steps: Obtain a sample road surface anomaly dataset, wherein the sample road surface anomaly data includes sample road surface anomaly images and corresponding sample road surface anomaly label groups, wherein the sample road surface anomaly images contain at least one road surface anomaly region, and the sample road surface anomaly label groups include sample road surface anomaly labels corresponding to each road surface anomaly region. Data augmentation is performed on each sample road surface anomaly data in the sample road surface anomaly dataset to obtain an augmented road surface anomaly dataset. The enhanced road surface anomaly dataset is defined as a road surface anomaly dataset, wherein the road surface anomaly data includes road surface anomaly images and corresponding road surface anomaly label groups.

3. The method according to claim 1, wherein, The process of dividing a preset road anomaly dataset into base and new classes to obtain base class datasets and new class datasets includes: Determine each road anomaly category corresponding to each road anomaly data in the road anomaly dataset to obtain a road anomaly category set, wherein one road anomaly data corresponds to at least one road anomaly category; Based on the road anomaly label corresponding to each road anomaly data in the road anomaly dataset, determine the category data volume of each road anomaly category in the road anomaly category set to obtain the category data volume set; The road surface anomaly categories corresponding to the category data volume that is greater than or equal to a preset threshold in the category data volume set are determined as the basic categories, thus obtaining the basic category set; The road surface anomaly categories corresponding to the category data volume that is less than a preset threshold in the category data volume set are identified as new categories, thus obtaining a new category set; The road surface anomaly data corresponding to the basic category in the road surface anomaly dataset are determined as base class data to obtain the base class dataset; The road surface anomaly data corresponding to the newly added category in the road surface anomaly dataset are identified as new category data, thus obtaining the new category dataset.

4. The method according to claim 1, wherein, The step of inputting the base class and new class matching pair set into the initial road surface anomaly detection model for initial training rounds to obtain the initial base class loss value includes: Determine the initial parameters of the initial road surface anomaly detection model, wherein the initial road surface anomaly detection model is a preset target detection model; Each base class and new class matching pair in the base class and new class matching pair set is sequentially input into the initial road surface anomaly detection model to obtain the base class detection result set; Determine the loss value between each base class detection result in the base class detection result set and the road surface anomaly label group corresponding to each base class data in the base class dataset to obtain the initial base class loss value.

5. The method according to claim 1, wherein, The step of determining the dynamic tradeoff operator based on the initial base class loss value and the current round base class loss value includes: Perform a hyperbolic tangent transform on the initial base class loss value and the current round base class loss value to obtain the current round base class training degree value, wherein the current round base class training degree value represents the base class training degree as measured by the change of the base class training loss value; Based on the current training level of the base class, determine the dynamic tradeoff operator.

6. The method according to claim 1, wherein, The method further includes: Acquire anomaly images of the road surface to be tested; The road surface anomaly image to be tested is input into the road surface anomaly detection model to obtain a road surface anomaly detection result map, wherein the road surface anomaly detection result map includes a road surface anomaly area label box and the corresponding road surface anomaly category.

7. A device for generating a generalized small-sample road surface anomaly detection model, comprising: The base class and new class partitioning unit is configured to partition the preset road anomaly dataset into base class and new class datasets to obtain base class datasets and new class datasets; The base class matching unit is configured to perform base class matching on each new class data in the new class dataset based on the base class dataset, to obtain a set of base class and new class matching pairs; The initial training unit is configured to input the set of matching pairs between the base class and the new class into the initial road surface anomaly detection model for initial training to obtain the initial base class loss value; The model training unit is configured to perform the following model training steps based on a preset number of rounds: Input the set of matching pairs between the base class and the new class into the initial road surface anomaly detection model to obtain the base class loss value and the new class loss value for the current round. Based on the initial base class loss value and the current round base class loss value, determine the dynamic trade-off operator; The total loss value for the current round is determined based on the dynamic trade-off operator, the base class loss value for the current round, and the new class loss value for the current round. Based on the total loss value of the current round, the backpropagation parameters of the initial road surface anomaly detection model are updated to obtain the updated road surface anomaly detection model; In response to determining that the training rounds are equal to the preset rounds, the updated road surface anomaly detection model is determined as the road surface anomaly detection model; In response to the determination that the number of training rounds is less than the preset number of rounds, the updated road surface anomaly detection model is used as the initial road surface anomaly detection model to perform the model training step again.

8. An electronic device, comprising: One or more processors; A storage device on which one or more programs are stored; When the one or more programs are executed by the one or more processors, the one or more processors implement the method as described in any one of claims 1 to 6.

9. A computer-readable medium having a computer program stored thereon, wherein, When the program is executed by the processor, it implements the method as described in any one of claims 1 to 6.

Images