Labeling method applied to damaged image of motor vehicle and related equipment

Abstract

The embodiment of the invention belongs to the technical field of image processing, and relates to a marking method and related equipment applied to a damaged image of a motor vehicle, and the method comprises the steps: constructing a marking data set containing the damaged image and corresponding three-dimensional label data, and carrying out the training through a deep learning model, and automatic identification and prediction of the damage degree and the maintenance scheme of the motor vehicle are realized. Firstly, a data set is split, an initial model is trained by using a training set, then model parameters are optimized through a verification set, and finally, a target deep learning model which can be used for an actual scene is obtained. According to the method, manual marking and machine learning are combined, so that the accuracy and efficiency of motor vehicle damage evaluation are greatly improved, an intelligent solution is provided for services such as insurance claim settlement and maintenance evaluation, and the method has important practical value.

Description

Technical Field

[0001] The present application relates to the field of image processing technology, and in particular to a labeling method and related equipment applied to damaged motor vehicle images. Background Art

[0002] In the image annotation scenario, the current annotation tools in the industry only provide labels for the names of objects. However, there are no dedicated annotation tools that provide car damage labels and repair solution labels.

[0003] When it is necessary to annotate images of damaged cars, the traditional implementation method requires workers to rely on experience to determine the damage location and corresponding repair plan.

[0004] However, the applicant found that the traditional labeling method is easily affected by personal ability and subjective factors, resulting in inconsistent evaluation results; at the same time, manual evaluation is time-consuming and difficult to meet the needs of rapid claims settlement. It can be seen that the traditional labeling method has problems such as strong subjectivity, low efficiency, and insufficient accuracy. Summary of the invention

[0005] The purpose of the embodiments of the present application is to propose a labeling method and related equipment for damaged motor vehicle images, so as to solve the problems of strong subjectivity, low efficiency, and insufficient accuracy of traditional labeling methods.

[0006] In order to solve the above technical problems, the present application embodiment provides a method for labeling damaged images of motor vehicles, which adopts the following technical solutions:

[0007] Receiving a labeled image data set sent by a user terminal, wherein the labeled image data set includes a damaged image of a motor vehicle and three-dimensional label data corresponding to the damaged image of the motor vehicle, wherein the three-dimensional label data includes a damaged component label, a damage degree label, and a repair solution label;

[0008] Splitting the labeled image data set to obtain a training image data set and a verification image data set;

[0009] Calling a first initial deep learning model, using the damaged motor vehicle image of the training image data set as input data and the three-dimensional label data of the training image data set as output data to perform model training processing on the first initial deep learning model, to obtain an intermediate deep learning model;

[0010] Performing verification training processing on the intermediate deep learning model according to the verification image data set to obtain a target deep learning model;

[0011] Acquire a damaged image of a motor vehicle to be identified;

[0012] The damaged image of the motor vehicle to be identified is input into the target deep learning model for label prediction processing to obtain a label prediction result.

[0013] Furthermore, after the step of inputting the damaged motor vehicle image to be identified into the target deep learning model for label prediction processing to obtain a label prediction result, the following steps are also included:

[0014] Determine whether the label prediction result meets a preset confidence threshold;

[0015] If the label prediction result meets the preset confidence threshold, the label prediction result is used as the final prediction result;

[0016] If the label prediction result does not meet the preset confidence threshold, the label prediction result is corrected according to data adjustment and model optimization processing to obtain the final prediction result.

[0017] Furthermore, if the label prediction result does not meet the preset confidence threshold, the label prediction result is corrected according to data adjustment and model optimization processing to obtain the final prediction result, which specifically includes the following steps:

[0018] Performing data cleaning processing on the training image data set to obtain a cleaned image data set;

[0019] Performing enhancement processing on the cleaned image data set according to a data enhancement technology to obtain an enhanced image data set;

[0020] Performing balancing processing on the enhanced image data set according to an undersampling method and an oversampling method to obtain a balanced image data set;

[0021] Calling a second initial deep learning model, and performing model training processing on the second initial deep learning model according to the balanced image data set to obtain a second intermediate deep learning model and a training prediction label;

[0022] Receiving corrected label data corresponding to the training prediction label sent by a user terminal;

[0023] Calculating the confidence of the corrected label data according to a confidence algorithm to obtain a confidence list;

[0024] Evaluate the corrected label data and the confidence list according to the accuracy evaluation index, the recall evaluation index and the F1 value evaluation index to obtain an evaluation result;

[0025] Performing parameter adjustment processing on the second intermediate deep learning model according to the evaluation result to obtain a second target deep learning model;

[0026] Inputting the damaged motor vehicle image to be identified into the second target deep learning model for label prediction processing to obtain a second label prediction result;

[0027] Determining whether the second tag prediction result meets a preset confidence threshold;

[0028] If the second label prediction result meets the preset confidence threshold, the second label prediction result is used as the final prediction result;

[0029] If the second label prediction result still does not meet the preset confidence threshold, the second label prediction result is re-corrected according to data adjustment and model optimization processing until a final prediction result that meets the preset confidence threshold is obtained.

[0030] Furthermore, after the step of acquiring the damaged image of the motor vehicle to be identified, the following steps are also included:

[0031] Performing format verification processing on the damaged motor vehicle image to be identified to obtain a format verification result;

[0032] Performing size verification processing on the damaged motor vehicle image to be identified to obtain a size verification result;

[0033] Performing resolution verification processing on the damaged motor vehicle image to be identified to obtain a resolution verification result;

[0034] Performing image direction verification processing on the damaged motor vehicle image to be identified to obtain an image direction verification result;

[0035] If the format verification result, the size verification result, the resolution verification result, and the image direction verification result all meet the preset requirements, then it is confirmed that the damaged motor vehicle image to be identified meets the specification requirements;

[0036] If at least one of the format verification result, the size verification result, the resolution verification result and the image direction verification result does not meet the preset requirements, it is confirmed that the damaged motor vehicle image to be identified does not meet the standard requirements, and image adjustment processing is performed on the damaged motor vehicle image to be identified.

[0037] Furthermore, after the step of inputting the damaged motor vehicle image to be identified into the target deep learning model for label prediction processing to obtain a label prediction result, the following steps are also included:

[0038] Sending the label prediction result to the user terminal for verification by an annotator;

[0039] Receiving correction data corresponding to the label prediction result sent by the user terminal;

[0040] Correcting the label prediction result according to the correction data to obtain a corrected prediction result;

[0041] The target deep learning model is optimized and trained according to the modified prediction results to obtain an optimized deep learning model.

[0042] Furthermore, after the step of inputting the damaged motor vehicle image to be identified into the target deep learning model for label prediction processing to obtain a label prediction result, the following steps are also included:

[0043] Obtain a list of hyperparameters that need to be tuned for the target deep learning model according to the label prediction results;

[0044] Setting a value range and a step size for each parameter in the hyperparameter list to obtain a parameter tuning search space;

[0045] Performing a sampling operation in the parameter tuning search space according to a preset search algorithm to obtain a hyperparameter combination;

[0046] Applying the hyperparameter combinations to the target deep learning model for model training and evaluation respectively, obtaining model performance indicators corresponding to each hyperparameter combination and optimizing the deep learning model;

[0047] The optimized deep learning model with the maximum value of the model performance index is taken as the optimal deep learning model.

[0048] In order to solve the above technical problems, the embodiment of the present application further provides a marking device for a damaged image of a motor vehicle, which adopts the following technical solution:

[0049] a marked image acquisition module, configured to receive a marked image data set sent by a user terminal, wherein the marked image data set includes a damaged image of a motor vehicle and three-dimensional label data corresponding to the damaged image of the motor vehicle, wherein the three-dimensional label data includes a damaged component label, a damage degree label, and a repair solution label;

[0050] A splitting module is used to split the labeled image data set to obtain a training image data set and a verification image data set;

[0051] A model training module is used to call a first initial deep learning model, and use the damaged motor vehicle image of the training image data set as input data and the three-dimensional label data of the training image data set as output data to perform model training processing on the first initial deep learning model to obtain an intermediate deep learning model;

[0052] A verification training module, used to perform verification training processing on the intermediate deep learning model according to the verification image data set to obtain a target deep learning model;

[0053] A damaged image acquisition module, used to acquire a damaged image of a motor vehicle to be identified;

[0054] The label prediction acquisition module is used to input the damaged motor vehicle image to be identified into the target deep learning model for label prediction processing to obtain a label prediction result.

[0055] Furthermore, the model also includes:

[0056] A confidence judgment module is used to judge whether the label prediction result meets a preset confidence threshold;

[0057] A first judgment result module is used to use the label prediction result as the final prediction result if the label prediction result meets a preset confidence threshold;

[0058] The second judgment result module is used to correct the label prediction result according to data adjustment and model optimization processing to obtain the final prediction result if the label prediction result does not meet the preset confidence threshold.

[0059] Furthermore, the model also includes:

[0060] A result sending module, used for sending the label prediction result to the user terminal for verification by the annotator;

[0061] A correction data acquisition module, used for receiving correction data corresponding to the label prediction result sent by the user terminal;

[0062] A correction module, used to correct the label prediction result according to the correction data to obtain a corrected prediction result;

[0063] The optimization training module is used to optimize the training of the target deep learning model according to the modified prediction results to obtain an optimized deep learning model.

[0064] In order to solve the above technical problems, the embodiment of the present application further provides a computer device, which adopts the following technical solution:

[0065] The method comprises a memory and a processor, wherein the memory stores computer-readable instructions, and the processor implements the steps of the above-mentioned marking method applied to damaged images of motor vehicles when executing the computer-readable instructions.

[0066] In order to solve the above technical problems, the embodiment of the present application further provides a computer-readable storage medium, which adopts the following technical solution:

[0067] The computer-readable storage medium stores computer-readable instructions, and when the computer-readable instructions are executed by the processor, the steps of the above-mentioned method for marking damaged images of motor vehicles are implemented.

[0068] The present application provides a method for labeling damaged motor vehicle images, comprising: receiving a labeled image dataset sent by a user terminal, wherein the labeled image dataset includes a damaged motor vehicle image and three-dimensional label data corresponding to the damaged motor vehicle image, and the three-dimensional label data includes a damaged component label, a damage degree label, and a repair plan label; splitting the labeled image dataset to obtain a training image dataset and a verification image dataset; calling a first initial deep learning model, using the damaged motor vehicle image of the training image dataset as input data and the three-dimensional label data of the training image dataset as output data to perform model training on the first initial deep learning model to obtain an intermediate deep learning model; performing verification training on the intermediate deep learning model according to the verification image dataset to obtain a target deep learning model; obtaining a damaged motor vehicle image to be identified; inputting the damaged motor vehicle image to be identified into the target deep learning model for label prediction processing to obtain a label prediction result. Compared with the prior art, the present application greatly improves the accuracy and efficiency of motor vehicle damage assessment by combining manual labeling with machine learning, and provides an intelligent solution for insurance claims, repair valuation and other businesses, which has important practical value. BRIEF DESCRIPTION OF THE DRAWINGS

[0069] In order to more clearly illustrate the scheme in the present application, a brief introduction is given below to the drawings required for use in the description of the embodiments of the present application. Obviously, the drawings described below are some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0070] Figure 1 is an exemplary system architecture diagram to which the present application may be applied;

[0071] Figure 2 is a flowchart of the implementation of the marking method for a damaged motor vehicle image provided in the first embodiment of the present application;

[0072] Figure 3 is a structural schematic diagram of a marking device for a damaged motor vehicle image provided in Embodiment 2 of the present application;

[0073] Figure 4 It is a structural diagram of an embodiment of a computer device according to the present application. DETAILED DESCRIPTION

[0074] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as those commonly understood by technicians in the technical field of the present application; the terms used in the specification of the application herein are only for the purpose of describing specific embodiments and are not intended to limit the present application; the terms "including" and "having" and any variations thereof in the specification and claims of the present application and the above-mentioned drawings are intended to cover non-exclusive inclusions. The terms "first", "second", etc. in the specification and claims of the present application or the above-mentioned drawings are used to distinguish different objects, not to describe a specific order.

[0075] Reference to "embodiments" herein means that a particular feature, structure, or characteristic described in conjunction with the embodiments may be included in at least one embodiment of the present application. The appearance of the phrase in various locations in the specification does not necessarily refer to the same embodiment, nor is it an independent or alternative embodiment that is mutually exclusive with other embodiments. It is explicitly and implicitly understood by those skilled in the art that the embodiments described herein may be combined with other embodiments.

[0076] In order to enable those skilled in the art to better understand the solution of the present application, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the accompanying drawings.

[0077] like Figure 1 As shown, the system architecture 100 may include terminal devices 101, 102, 103, a network 104 and a server 105. The network 104 is used to provide a medium for communication links between the terminal devices 101, 102, 103 and the server 105. The network 104 may include various connection types, such as wired, wireless communication links or optical fiber cables, etc.

[0078] Users can use terminal devices 101, 102, 103 to interact with server 105 through network 104 to receive or send messages, etc. Various communication client applications can be installed on terminal devices 101, 102, 103, such as web browser applications, shopping applications, search applications, instant messaging tools, email clients, social platform software, etc.

[0079] Terminal devices 101, 102, 103 can be various electronic devices with display screens and supporting web browsing, including but not limited to smart phones, tablet computers, e-book readers, MP3 players (Moving Picture Experts Group Audio Layer III), MP4 (Moving Picture Experts Group Audio Layer IV) players, laptop computers, desktop computers, etc.

[0080] The server 105 may be a server that provides various services, such as a background server that provides support for web pages displayed on the terminal devices 101 , 102 , and 103 .

[0081] It should be noted that the marking method for damaged motor vehicle images provided in the embodiments of the present application is generally executed by a server / terminal device, and accordingly, the marking device for damaged motor vehicle images is generally arranged in a server / terminal device.

[0082] It should be understood that Figure 1 The number of terminal devices, networks and servers in the embodiment is only for illustration. Any number of terminal devices, networks and servers may be provided according to implementation requirements.

[0083] Embodiment 1

[0084] Continue to refer Figure 2 , shows a flow chart of an embodiment of a method for labeling a damaged motor vehicle image according to the present application. The method for labeling a damaged motor vehicle image includes: step S201, step S202, step S203, step S204, step S205 and step S206.

[0085] In step S201, a labeled image data set sent by a user terminal is received, wherein the labeled image data set includes a damaged image of a motor vehicle and three-dimensional label data corresponding to the damaged image of the motor vehicle, and the three-dimensional label data includes a damaged component label, a damage degree label, and a repair plan label.

[0086] In the embodiments of the present application, the user terminal refers to a terminal device used to execute the image processing method for preventing document abuse provided by the present application. The user terminal can be a mobile terminal such as a mobile phone, a smart phone, a laptop computer, a digital broadcast receiver, a PDA (personal digital assistant), a PAD (tablet computer), a PMP (portable multimedia player), a navigation device, etc., as well as a fixed terminal such as a digital TV, a desktop computer, etc. It should be understood that the examples of user terminals here are only for the convenience of understanding and are not used to limit the present application.

[0087] In an embodiment of the present application, damaged car pictures are collected and stored in a database, and labels for damaged car parts, damage degree labels, and repair plan labels are constructed, a labeling rule document is formulated, and labeling personnel and quality inspection personnel are determined.

[0088] In the embodiment of the present application, pre-labeling is required, and professional labeling experts are arranged to label the damage in the image of the sample data. After the labeling is completed, quality inspection experts check the batch of labeled data to ensure the labeling quality.

[0089] In step S202, the labeled image data set is split to obtain a training image data set and a verification image data set.

[0090] In step S203, the first initial deep learning model is called, and the damaged motor vehicle images of the training image data set are used as input data and the three-dimensional label data of the training image data set are used as output data to perform model training processing on the first initial deep learning model to obtain an intermediate deep learning model.

[0091] In an embodiment of the present application, the manually annotated data set images and the corresponding json files are uploaded to the cloud server to train the model; 10% of the data are extracted as test data and 90% as training data. During the model training process, the corresponding parameters are continuously updated through the back propagation algorithm.

[0092] In step S204, verification training is performed on the intermediate deep learning model according to the verification image data set to obtain a target deep learning model.

[0093] In the embodiment of the present application, after the model training is completed, the model recognizes the test data, and the model capability is ensured to be continuously improved during the recognition process until the optimal model is output.

[0094] In step S205, a damaged image of a motor vehicle to be identified is acquired.

[0095] In step S206, the damaged image of the motor vehicle to be identified is input into the target deep learning model for label prediction processing to obtain a label prediction result.

[0096] In an embodiment of the present application, a method for labeling a damaged motor vehicle image is provided, comprising: receiving a labeled image data set sent by a user terminal, wherein the labeled image data set includes a damaged motor vehicle image, three-dimensional label data corresponding to the damaged motor vehicle image, and the three-dimensional label data includes a damaged component label, a damage degree label, and a repair plan label; splitting the labeled image data set to obtain a training image data set and a verification image data set; calling a first initial deep learning model, using the damaged motor vehicle image of the training image data set as input data and the three-dimensional label data of the training image data set as output data to perform model training on the first initial deep learning model to obtain an intermediate deep learning model; performing verification training on the intermediate deep learning model according to the verification image data set to obtain a target deep learning model; obtaining a damaged motor vehicle image to be identified; inputting the damaged motor vehicle image to be identified into the target deep learning model for label prediction processing to obtain a label prediction result. Compared with the prior art, the present application greatly improves the accuracy and efficiency of motor vehicle damage assessment by combining manual labeling with machine learning, and provides an intelligent solution for insurance claims, repair valuation and other businesses, which has important practical value.

[0097] In some optional implementations of the embodiments of the present application, after step S206, the following steps are further included:

[0098] Determine whether the label prediction result meets the preset confidence threshold;

[0099] If the label prediction result meets the preset confidence threshold, the label prediction result is used as the final prediction result;

[0100] If the label prediction result does not meet the preset confidence threshold, the label prediction result is corrected according to data adjustment and model optimization to obtain the final prediction result.

[0101] In the embodiment of the present application, the preset confidence threshold is mainly used to uniquely distinguish whether the label prediction result has reference value. The preset confidence threshold can be specifically adjusted according to the actual situation. As an example, the preset confidence threshold can be 8.5, 9, etc. It should be understood that the examples of the preset confidence threshold here are only for the convenience of understanding and are not used to limit the present application.

[0102] In practical applications, if the confidence of the prediction result is higher than 9, the preliminary label prediction result is output; if the confidence of the prediction result is lower than 9, data enhancement processing is required, such as rotation, flipping, brightness adjustment, etc., and the model is retrained to optimize the result.

[0103] In the embodiments of the present application, the preliminary label prediction results are corrected and improved through data adjustment and model optimization processing, such as using the Adam optimizer to adjust parameters, and the accuracy of the final generated label prediction results reaches more than 95%.

[0104] In some optional implementations of the embodiments of the present application, if the label prediction result does not meet the preset confidence threshold, the label prediction result is corrected according to data adjustment and model optimization processing to obtain the final prediction result, which specifically includes the following steps:

[0105] Performing data cleaning on the training image data set to obtain a cleaned image data set;

[0106] The cleaned image data set is enhanced according to the data enhancement technology to obtain an enhanced image data set;

[0107] The enhanced image data set is balanced according to the under-sampling method and the over-sampling method to obtain a balanced image data set;

[0108] Calling a second initial deep learning model, and performing model training processing on the second initial deep learning model according to the balanced image data set to obtain a second intermediate deep learning model and a training prediction label;

[0109] Receiving corrected label data corresponding to the training prediction label sent by the user terminal;

[0110] Calculate the confidence of the corrected label data according to the confidence algorithm to obtain a confidence list;

[0111] The corrected label data and the confidence list are evaluated and processed according to the accuracy evaluation index, the recall evaluation index and the F1 value evaluation index to obtain the evaluation results;

[0112] According to the evaluation result, the second intermediate deep learning model is subjected to parameter adjustment processing to obtain a second target deep learning model;

[0113] Inputting the damaged motor vehicle image to be identified into the second target deep learning model for label prediction processing to obtain a second label prediction result;

[0114] Determine whether the second label prediction result meets a preset confidence threshold;

[0115] If the second label prediction result meets the preset confidence threshold, the second label prediction result is used as the final prediction result;

[0116] If the second label prediction result still does not meet the preset confidence threshold, the second label prediction result is re-corrected according to data adjustment and model optimization processing until a final prediction result that meets the preset confidence threshold is obtained.

[0117] In an embodiment of the present application, when processing data, the Pandas library is first used to clean the data, rows containing NaN values ​​are deleted, and the Z-score method is used to filter out outliers, such as using data outside 3 times the standard deviation as outliers.

[0118] In an embodiment of the present application, in terms of data enhancement, by using Python's Augmentor library, the image data is rotated (±15 degrees), scaled (8 to 2 times) and horizontally flipped to increase the diversity of training samples.

[0119] In the embodiment of the present application, in order to balance the proportion of each type of samples in the data set, the SMOTE algorithm is used for oversampling and the random undersampling method is used to oversample the minority class and undersample the majority class to ensure a balanced number of samples in each class.

[0120] In an embodiment of the present application, a pre-trained ResNet50 model is used during model training. Through the transfer learning method, a new dataset is used for fine-tuning based on the weights pre-trained on the ImageNet dataset.

[0121] In the embodiments of the present application, after obtaining the preliminary label prediction results, some correction rules are formulated by combining the knowledge and experience of experts in the field. For example, in medical image classification, label correction is performed based on the image characteristics of a specific disease.

[0122] In an embodiment of the present application, the confidence score of each predicted label is calculated using a confidence calculation formula, such as a Softmax function, to obtain a confidence list.

[0123] In the embodiments of the present application, in order to evaluate the quality of the prediction results, evaluation indicators such as accuracy, recall and F1 value are used, among which the F1 value refers to the harmonic mean of accuracy and recall, which is used to comprehensively evaluate the performance of the model.

[0124] In the embodiment of the present application, these indicators are used for evaluation based on the confidence list and the corrected prediction labels, such as calculating that the accuracy of the model is 95%, the recall rate is 90%, and the F1 value is 92%.

[0125] In the embodiment of the present application, on this basis, the grid search method is used to adjust the hyperparameters of the model, and the selected parameters include learning rate (values ​​between 001 and 01) and batch size (16, 32, 64), and the optimal parameter combination is determined by cross-validation.

[0126] In an embodiment of the present application, retraining is performed using the optimized model to generate new preliminary prediction labels.

[0127] In the embodiment of the present application, the correction rules and confidence calculation are applied again to obtain the final prediction result and confidence score.

[0128] In the embodiment of the present application, by comparing with the previous evaluation indicators, it is determined whether the model performance meets expectations. For example, the accuracy of the new model is improved to 94%, the recall rate reaches 92%, and the F1 value is 93%.

[0129] In the embodiment of the present application, based on business needs, it is finally evaluated whether the model needs further optimization. If it is determined that the performance of the model has met the actual application requirements, it is decided to adopt the model for actual application.

[0130] In some optional implementations of the embodiments of the present application, after step S205, the following steps are further included:

[0131] Performing format verification processing on the damaged motor vehicle image to be identified to obtain a format verification result;

[0132] Performing size verification processing on the damaged motor vehicle image to be identified to obtain a size verification result;

[0133] Performing resolution verification processing on the damaged motor vehicle image to be identified to obtain a resolution verification result;

[0134] Performing image direction verification processing on the damaged motor vehicle image to be identified to obtain an image direction verification result;

[0135] If the format verification result, the size verification result, the resolution verification result, and the image direction verification result all meet the preset requirements, then it is confirmed that the damaged motor vehicle image to be identified meets the specification requirements;

[0136] If at least one of the format verification result, size verification result, resolution verification result and image orientation verification result does not meet the preset requirements, it is confirmed that the damaged motor vehicle image to be identified does not meet the specification requirements, and image adjustment processing is performed on the damaged motor vehicle image to be identified.

[0137] In an embodiment of the present application, the OpenCV library can be used to load the image to be recognized, and the image file is first read and the image object is obtained.

[0138] In the embodiment of the present application, the image can be loaded by calling the cvimread() function, and the format attributes of the image object can be obtained.

[0139] In an embodiment of the present application, assume that we have a function get_image_format(image_path) to determine whether the image format is a common format such as JPG, PNG, BMP, etc. This function can obtain the file extension through os.path.splitext(image_path) and determine whether it is in the list of allowed formats, such as .jpgpngbmp.

[0140] In the embodiment of the present application, the size attribute of the image file is obtained by the os.path.getsize(image_path) function. Assuming that we need the image file size not to exceed 5MB (5242880 bytes), a conditional judgment statement if image_size<=5242880 can be used to determine whether the image size is compliant.

[0141] In the embodiment of the present application, we can use the cvLaplacian() method to detect the clarity of the image object obtained by the cvimread() function.

[0142] In the embodiment of the present application, the specific method of resolution verification processing is to calculate the Laplace transform of the image and then find its variance value. If the variance value is greater than a certain threshold (such as 100), the image is considered to be clear and complete.

[0143] In the embodiment of the present application, it can be implemented by the following code: laplacian_var=cvLaplacian(image,cvCV_64F).var(), and the condition laplacian_var>100 is used to determine whether the image quality is qualified.

[0144] In the embodiment of the present application, to determine whether the image orientation is correct, we can use the image rotation method in OpenCV. Specifically, assuming that the correct orientation of the image is vertical, the image can be rotated and corrected by the cvgetRotationMatrix2D() and cvwarpAffine() methods. First, the rotation matrix is ​​calculated: rotation_matrix = cvgetRotationMatrix2D ((image_width / 2, image_height / 2), rotation_angle, 1), and then the rotation is applied by cvwarpAffine (image, rotation_matrix, (image_width, image_height)).

[0145] In an embodiment of the present application, we can determine the rotation angle rotation_angle by analyzing the direction of the straight line or text in the image, for example, by detecting the straight line angle through Hough transform. If the detected straight line angle deviates from the vertical direction by more than a certain threshold (such as 10 degrees), image rotation correction is performed.

[0146] In the embodiment of the present application, finally, by combining the image format verification result, the image size verification result, the image quality verification result and the image direction verification result, it is comprehensively judged whether the image to be identified meets the requirements of all business attributes.

[0147] In the embodiment of the present application, it is assumed that we have a comprehensive judgment function validate_image(image_path), which can call the verification function of each step in sequence, and combine all verification results after each step to determine whether subsequent image recognition processing can be performed.

[0148] In the embodiments of the present application, the specific implementation of each step has its detailed numerical and algorithmic basis to ensure that the entire process has a strict logical relationship and thinking chain.

[0149] In some optional implementations of the embodiments of the present application, after step S206, the following steps are further included:

[0150] Send the label prediction results to the user terminal for verification by the annotator;

[0151] Receiving correction data corresponding to the label prediction result sent by the user terminal;

[0152] Correct the label prediction result according to the correction data to obtain a corrected prediction result;

[0153] According to the corrected prediction results, the target deep learning model is optimized and trained to obtain an optimized deep learning model.

[0154] In an embodiment of the present application, the identified JSON result is displayed on the annotation page, and the annotator checks it and performs addition, reduction, and correction operations to ensure that the annotation data is changed to the correct result.

[0155] In an embodiment of the present application, the auditor formulates spot check tasks based on the number of additions, deletions and corrections to the annotated items in the json file identified by AI, to ensure that the annotated content modified by the annotator is accurate and has no omissions or mislabeling.

[0156] In an embodiment of the present application, the modified items, added items, and removed items of the marked items in the json file are output to the algorithm personnel responsible for the model quality. The algorithm personnel correct the erroneous parts of the model recognition by updating the parameters and enhance the model recognition capability based on the supplementary parts of the Json file.

[0157] In an embodiment of the present application, 90% of the model iteration data is extracted to train the model, and after the model is completed, the remaining 10% of the annotations are tested, and this cycle is continued. While continuously improving the accuracy of the model, the complexity of the work of the annotation personnel can be reduced.

[0158] In some optional implementations of the embodiments of the present application, after step S206, the following steps are further included:

[0159] Obtain a list of hyperparameters that need to be tuned for the target deep learning model based on the label prediction results;

[0160] Set the value range and step size for each parameter in the hyperparameter list to obtain the parameter tuning search space;

[0161] Perform sampling operations in the parameter tuning search space according to a preset search algorithm to obtain a hyperparameter combination;

[0162] Apply the hyperparameter combinations to the target deep learning model for model training and evaluation, obtain the model performance indicators corresponding to each hyperparameter combination, and optimize the deep learning model;

[0163] The optimized deep learning model with the maximum value of the model performance index is taken as the optimal deep learning model.

[0164] In an embodiment of the present application, based on the evaluation results of the deep learning model, a list of hyperparameters that need to be tuned is obtained, such as learning rate, batch size, number of hidden layers, etc.

[0165] In an embodiment of the present application, for each hyperparameter, a value range and a step size are set, for example, the value range of the learning rate is 001 to 1, and the step size is 001, to generate a search space for hyperparameter tuning.

[0166] In an embodiment of the present application, the preset search algorithm is mainly used to sample in the hyperparameter search space to obtain a hyperparameter combination. The preset search algorithm may be an optimization algorithm such as a grid search or a random search.

[0167] In an embodiment of the present application, a hyperparameter combination is applied to a deep learning model to perform model training and evaluation to obtain model performance indicators. Then, based on the model performance indicators, a set of hyperparameter combinations with the best model performance is selected from the hyperparameter combinations, and the deep learning model trained by the set of hyperparameter combinations with the best performance is used as the optimal deep learning model for subsequent annotation applications.

[0168] In the embodiments of the present application, through optimization algorithms such as grid search, the optimization space of hyperparameter combinations can be efficiently explored to find the optimal model configuration, thereby improving the accuracy and reliability of damage identification and maintenance plan formulation.

[0169] The embodiments of the present application can acquire and process relevant data based on artificial intelligence technology. Among them, artificial intelligence (AI) is the theory, method, technology and application system that uses digital computers or machines controlled by digital computers to simulate, extend and expand human intelligence, perceive the environment, acquire knowledge and use knowledge to obtain the best results.

[0170] AI basic technologies generally include sensors, dedicated AI chips, cloud computing, distributed storage, big data processing technology, operation / interaction systems, mechatronics, etc. AI software technologies mainly include computer vision technology, robotics technology, biometrics technology, speech processing technology, natural language processing technology, and machine learning / deep learning.

[0171] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiments can be implemented by instructing the relevant hardware through computer-readable instructions, and the computer-readable instructions can be stored in a computer-readable storage medium. When the program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, the aforementioned storage medium can be a non-volatile storage medium such as a disk, an optical disk, a read-only memory (ROM), or a random access memory (RAM).

[0172] It should be understood that, although the steps in the flowchart of the accompanying drawings are displayed in sequence as indicated by the arrows, these steps are not necessarily executed in sequence in the order indicated by the arrows. Unless otherwise specified herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least a part of the steps in the flowchart of the accompanying drawings may include multiple sub-steps or multiple stages, and these sub-steps or stages are not necessarily executed at the same time, but can be executed at different times, and their execution order is not necessarily sequential, but can be executed in turn or alternately with other steps or at least a part of the sub-steps or stages of other steps.

[0173] Embodiment 2

[0174] Further references Figure 3 , as a response to the above Figure 2In order to realize the method shown in the figure, the present application provides an embodiment of a marking device for a damaged motor vehicle image. Figure 2 Corresponding to the method embodiment shown, the device can be specifically applied to various electronic devices.

[0175] like Figure 3 As shown, the marking device 200 applied to the damaged image of the motor vehicle in the embodiment of the present application includes:

[0176] The labeled image acquisition module 210 is used to receive a labeled image data set sent by a user terminal, wherein the labeled image data set includes a damaged image of a motor vehicle and three-dimensional label data corresponding to the damaged image of the motor vehicle, and the three-dimensional label data includes a damaged component label, a damage degree label, and a repair solution label;

[0177] A splitting module 220 is used to split the labeled image data set to obtain a training image data set and a verification image data set;

[0178] The model training module 230 is used to call the first initial deep learning model, use the damaged motor vehicle image of the training image data set as input data and the three-dimensional label data of the training image data set as output data to perform model training processing on the first initial deep learning model to obtain an intermediate deep learning model;

[0179] A verification training module 240 is used to perform verification training processing on the intermediate deep learning model according to the verification image data set to obtain a target deep learning model;

[0180] A damaged image acquisition module 250 is used to acquire a damaged image of a motor vehicle to be identified;

[0181] The label prediction acquisition module 260 is used to input the damaged image of the motor vehicle to be identified into the target deep learning model for label prediction processing to obtain a label prediction result.

[0182] In an embodiment of the present application, a labeling device 200 for a damaged image of a motor vehicle is provided, comprising: a labeled image acquisition module 210, for receiving a labeled image data set sent by a user terminal, wherein the labeled image data set comprises a damaged image of a motor vehicle and three-dimensional label data corresponding to the damaged image of the motor vehicle, and the three-dimensional label data comprises a damaged component label, a damage degree label, and a repair plan label; a splitting module 220, for splitting the labeled image data set to obtain a training image data set and a verification image data set; a model training module 230, for calling a first initial deep learning model, and performing model training processing on the first initial deep learning model using the damaged image of the motor vehicle in the training image data set as input data and the three-dimensional label data of the training image data set as output data to obtain an intermediate deep learning model; a verification training module 240, for performing verification training processing on the intermediate deep learning model according to the verification image data set to obtain a target deep learning model; a damaged image acquisition module 250, for obtaining a damaged image of a motor vehicle to be identified; and a label prediction acquisition module 260, for inputting the damaged image of the motor vehicle to be identified into the target deep learning model for label prediction processing to obtain a label prediction result. Compared with the existing technology, this application greatly improves the accuracy and efficiency of motor vehicle damage assessment by combining manual labeling with machine learning, provides intelligent solutions for insurance claims, repair estimates and other businesses, and has important practical value.

[0183] In some optional implementations of the embodiments of the present application, the above-mentioned marking device 200 applied to the damaged image of the motor vehicle further includes:

[0184] A confidence judgment module is used to judge whether the label prediction result meets the preset confidence threshold;

[0185] The first judgment result module is used to take the label prediction result as the final prediction result if the label prediction result meets the preset confidence threshold;

[0186] The second judgment result module is used to correct the label prediction result according to data adjustment and model optimization processing to obtain the final prediction result if the label prediction result does not meet the preset confidence threshold.

[0187] In some optional implementations of the embodiments of the present application, the above-mentioned marking device 200 applied to the damaged image of the motor vehicle further includes:

[0188] The result sending module is used to send the label prediction results to the user terminal for the annotator to verify;

[0189] A correction data acquisition module, used to receive correction data corresponding to the label prediction result sent by the user terminal;

[0190] A correction module is used to correct the label prediction result according to the correction data to obtain a corrected prediction result;

[0191] The optimization training module is used to optimize the training of the target deep learning model according to the modified prediction results to obtain an optimized deep learning model.

[0192] To solve the above technical problems, the present application also provides a computer device. Figure 4 , Figure 4 This is a basic structural block diagram of a computer device according to an embodiment of the present application.

[0193] The computer device 300 includes a memory 310, a processor 320, and a network interface 330 that are interconnected and communicated through a system bus. It should be noted that the figure only shows a computer device 300 having components 310-330, but it should be understood that it is not required to implement all the components shown, and more or fewer components can be implemented instead. Among them, those skilled in the art can understand that the computer device here is a device that can automatically perform numerical calculations and / or information processing according to pre-set or stored instructions, and its hardware includes but is not limited to microprocessors, application specific integrated circuits (Application Specific Integrated Circuit, ASIC), programmable gate arrays (Field-Programmable Gate Array, FPGA), digital processors (Digital Signal Processor, DSP), embedded devices, etc.

[0194] The computer device may be a computing device such as a desktop computer, a notebook, a PDA, a cloud server, etc. The computer device may interact with a user through a keyboard, a mouse, a remote controller, a touch pad, or a voice control device.

[0195] The memory 310 includes at least one type of readable storage medium, and the readable storage medium includes flash memory, hard disk, multimedia card, card-type memory (for example, SD or DX memory, etc.), random access memory (RAM), static random access memory (SRAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), programmable read-only memory (PROM), magnetic memory, magnetic disk, optical disk, etc. In some embodiments, the memory 310 can be an internal storage unit of the computer device 300, such as a hard disk or memory of the computer device 300. In other embodiments, the memory 310 can also be an external storage device of the computer device 300, such as a plug-in hard disk equipped on the computer device 300, a smart memory card (Smart Media Card, SMC), a secure digital (Secure Digital, SD) card, a flash card, etc. Of course, the memory 310 can also include both the internal storage unit of the computer device 300 and its external storage device. In the embodiment of the present application, the memory 310 is generally used to store an operating system and various application software installed on the computer device 300, such as computer-readable instructions for a method for marking damaged images of motor vehicles, etc. In addition, the memory 310 can also be used to temporarily store various data that have been output or are to be output.

[0196] The processor 320 may be a central processing unit (CPU), a controller, a microcontroller, a microprocessor, or other data processing chip in some embodiments. The processor 320 is generally used to control the overall operation of the computer device 300. In the embodiment of the present application, the processor 320 is used to run the computer-readable instructions stored in the memory 310 or process data, such as running the computer-readable instructions of the marking method applied to the damaged motor vehicle image.

[0197] The network interface 330 may include a wireless network interface or a wired network interface. The network interface 330 is generally used to establish a communication connection between the computer device 300 and other electronic devices.

[0198] The computer equipment provided in this application greatly improves the accuracy and efficiency of motor vehicle damage assessment by combining manual labeling with machine learning, provides intelligent solutions for insurance claims, repair estimates and other businesses, and has important practical value.

[0199] The present application also provides another embodiment, namely, providing a computer-readable storage medium, wherein the computer-readable storage medium stores computer-readable instructions, and the computer-readable instructions can be executed by at least one processor to enable the at least one processor to perform the steps of the above-mentioned method for labeling damaged images of motor vehicles.

[0200] The computer-readable storage medium provided in this application greatly improves the accuracy and efficiency of motor vehicle damage assessment by combining manual labeling with machine learning, provides intelligent solutions for insurance claims, repair estimates and other businesses, and has important practical value.

[0201] Through the description of the above implementation methods, those skilled in the art can clearly understand that the above-mentioned embodiment methods can be implemented by means of software plus a necessary general hardware platform, and of course by hardware, but in many cases the former is a better implementation method. Based on such an understanding, the technical solution of the present application, or the part that contributes to the prior art, can be embodied in the form of a software product, which is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk), and includes a number of instructions for a terminal device (which can be a mobile phone, computer, server, air conditioner, or network device, etc.) to execute the methods described in each embodiment of the present application.

[0202] Obviously, the embodiments described above are only some embodiments of the present application, rather than all embodiments. The preferred embodiments of the present application are given in the accompanying drawings, but they do not limit the patent scope of the present application. The present application can be implemented in many different forms. On the contrary, the purpose of providing these embodiments is to make the understanding of the disclosure of the present application more thorough and comprehensive. Although the present application is described in detail with reference to the aforementioned embodiments, for those skilled in the art, it is still possible to modify the technical solutions recorded in the aforementioned specific implementation methods, or to perform equivalent replacement of some of the technical features therein. Any equivalent structure made using the contents of the specification and drawings of this application, directly or indirectly used in other related technical fields, is similarly within the scope of patent protection of this application.

Claims

1. A method for labeling damaged images of motor vehicles, characterized in that: The steps include: Receiving a labeled image data set sent by a user terminal, wherein the labeled image data set includes a damaged image of a motor vehicle and three-dimensional label data corresponding to the damaged image of the motor vehicle, wherein the three-dimensional label data includes a damaged component label, a damage degree label, and a repair solution label; Splitting the labeled image data set to obtain a training image data set and a verification image data set; Calling a first initial deep learning model, using the damaged motor vehicle image of the training image data set as input data and the three-dimensional label data of the training image data set as output data to perform model training processing on the first initial deep learning model, to obtain an intermediate deep learning model; Performing verification training processing on the intermediate deep learning model according to the verification image data set to obtain a target deep learning model; Acquire a damaged image of a motor vehicle to be identified; The damaged image of the motor vehicle to be identified is input into the target deep learning model for label prediction processing to obtain a label prediction result.

2. The method for labeling damaged motor vehicle images according to claim 1, characterized in that: After the step of inputting the damaged motor vehicle image to be identified into the target deep learning model for label prediction processing to obtain a label prediction result, the following steps are also included: Determine whether the label prediction result meets a preset confidence threshold; If the label prediction result meets the preset confidence threshold, the label prediction result is used as the final prediction result; If the label prediction result does not meet the preset confidence threshold, the label prediction result is corrected according to data adjustment and model optimization processing to obtain the final prediction result.

3. The method for labeling damaged motor vehicle images according to claim 2, characterized in that: If the label prediction result does not meet the preset confidence threshold, the label prediction result is corrected according to data adjustment and model optimization processing to obtain the final prediction result, which specifically includes the following steps: Performing data cleaning processing on the training image data set to obtain a cleaned image data set; Performing enhancement processing on the cleaned image data set according to a data enhancement technology to obtain an enhanced image data set; Performing balancing processing on the enhanced image data set according to an undersampling method and an oversampling method to obtain a balanced image data set; Calling a second initial deep learning model, and performing model training processing on the second initial deep learning model according to the balanced image data set to obtain a second intermediate deep learning model and a training prediction label; Receiving corrected label data corresponding to the training prediction label sent by a user terminal; Calculating the confidence of the corrected label data according to a confidence algorithm to obtain a confidence list; Evaluate the corrected label data and the confidence list according to the accuracy evaluation index, the recall evaluation index and the F1 value evaluation index to obtain an evaluation result; Performing parameter adjustment processing on the second intermediate deep learning model according to the evaluation result to obtain a second target deep learning model; Inputting the damaged motor vehicle image to be identified into the second target deep learning model for label prediction processing to obtain a second label prediction result; Determining whether the second tag prediction result meets a preset confidence threshold; If the second label prediction result meets the preset confidence threshold, the second label prediction result is used as the final prediction result; If the second label prediction result still does not meet the preset confidence threshold, the second label prediction result is re-corrected according to data adjustment and model optimization processing until a final prediction result that meets the preset confidence threshold is obtained.

4. The labeling method for damaged motor vehicle images according to claim 1, characterized in that: After the step of acquiring the damaged image of the motor vehicle to be identified, the following steps are also included: Performing format verification processing on the damaged motor vehicle image to be identified to obtain a format verification result; Performing size verification processing on the damaged motor vehicle image to be identified to obtain a size verification result; Performing resolution verification processing on the damaged motor vehicle image to be identified to obtain a resolution verification result; Performing image direction verification processing on the damaged motor vehicle image to be identified to obtain an image direction verification result; If the format verification result, the size verification result, the resolution verification result, and the image direction verification result all meet the preset requirements, then it is confirmed that the damaged motor vehicle image to be identified meets the specification requirements; If at least one of the format verification result, the size verification result, the resolution verification result and the image direction verification result does not meet the preset requirements, it is confirmed that the damaged motor vehicle image to be identified does not meet the standard requirements, and image adjustment processing is performed on the damaged motor vehicle image to be identified.

5. The method for labeling damaged motor vehicle images according to claim 1, characterized in that: After the step of inputting the damaged motor vehicle image to be identified into the target deep learning model for label prediction processing to obtain a label prediction result, the following steps are also included: Sending the label prediction result to the user terminal for verification by an annotator; Receiving correction data corresponding to the label prediction result sent by the user terminal; Correcting the label prediction result according to the correction data to obtain a corrected prediction result; The target deep learning model is optimized and trained according to the modified prediction results to obtain an optimized deep learning model.

6. The method for labeling damaged motor vehicle images according to claim 1, characterized in that: After the step of inputting the damaged motor vehicle image to be identified into the target deep learning model for label prediction processing to obtain a label prediction result, the following steps are also included: Obtain a list of hyperparameters that need to be tuned for the target deep learning model according to the label prediction results; Setting a value range and a step size for each parameter in the hyperparameter list to obtain a parameter tuning search space; Performing a sampling operation in the parameter tuning search space according to a preset search algorithm to obtain a hyperparameter combination; Applying the hyperparameter combinations to the target deep learning model for model training and evaluation respectively, obtaining model performance indicators corresponding to each hyperparameter combination and optimizing the deep learning model; The optimized deep learning model with the maximum value of the model performance index is taken as the optimal deep learning model.

7. A device for labeling damaged images of motor vehicles, characterized in that: include: a marked image acquisition module, configured to receive a marked image data set sent by a user terminal, wherein the marked image data set includes a damaged image of a motor vehicle and three-dimensional label data corresponding to the damaged image of the motor vehicle, wherein the three-dimensional label data includes a damaged component label, a damage degree label, and a repair solution label; A splitting module is used to split the labeled image data set to obtain a training image data set and a verification image data set; A model training module is used to call a first initial deep learning model, and use the damaged motor vehicle image of the training image data set as input data and the three-dimensional label data of the training image data set as output data to perform model training processing on the first initial deep learning model to obtain an intermediate deep learning model; A verification training module, used to perform verification training processing on the intermediate deep learning model according to the verification image data set to obtain a target deep learning model; A damaged image acquisition module, used to acquire a damaged image of a motor vehicle to be identified; The label prediction acquisition module is used to input the damaged motor vehicle image to be identified into the target deep learning model for label prediction processing to obtain a label prediction result.

8. The marking device for damaged motor vehicle images according to claim 7, characterized in that: The model also includes: A confidence judgment module is used to judge whether the label prediction result meets a preset confidence threshold; A first judgment result module is used to use the label prediction result as the final prediction result if the label prediction result meets a preset confidence threshold; The second judgment result module is used to correct the label prediction result according to data adjustment and model optimization processing to obtain the final prediction result if the label prediction result does not meet the preset confidence threshold.

9. A computer device comprising a memory and a processor, characterized in that: The memory stores computer-readable instructions, and when the processor executes the computer-readable instructions, the steps of the marking method applied to a damaged image of a motor vehicle according to any one of claims 1 to 6 are implemented.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer-readable instructions, and when the computer-readable instructions are executed by a processor, the steps of the marking method applied to a damaged image of a motor vehicle according to any one of claims 1 to 6 are implemented.

Images