Vehicle maintenance plan prediction method and device, computer device, and storage medium

Abstract

The embodiment of the application belongs to the technical field of image processing, and relates to a vehicle repair scheme prediction method and device, a computer device and a storage medium. The method comprises the following steps: constructing a model training set according to historical repair data and labeling information; taking damaged position information, historical repair schemes and damaged components as damaged position labels, repair scheme labels and damaged component labels respectively when an initial yolov7 detection model calculates a loss; inputting the model training set into the initial yolov7 detection model, and performing a model training operation on the initial yolov7 detection model according to a damaged position loss function, a repair scheme loss function and a component category loss function to obtain a trained repair scheme prediction model; and inputting a to-be-detected damaged image into the trained repair scheme prediction model to perform a repair scheme prediction operation and obtain a repair scheme prediction result. The embodiment achieves more robust and reliable prediction of damage results of different components in a case.

Description

Technical Field

[0001] The present application relates to the field of image processing technology, and in particular to a vehicle maintenance plan prediction method, apparatus, computer equipment, and storage medium. Background Art

[0002] With the rapid development of society, private cars have become an indispensable means of transportation. The increasing number of vehicles has undoubtedly increased the incidence of traffic accidents. After a traffic accident, insurance companies typically conduct damage assessments at the scene. This involves examining photos taken at the scene to determine the vehicle's repair plan, which serves as the basis for the insurance company's claim settlement. Because damage assessments consume significant human resources and the results are highly subjective, vehicle repair plan detection systems based on deep learning methods are gradually replacing manual operations. These systems can accurately detect the type of repair plan for a vehicle component based on one or more images.

[0003] Existing object detectors are primarily based on CNNs, and their detection accuracy continues to improve. They can identify various repair options for vehicle components involved in an accident, swiftly resolving accident claims. Vehicle repair option detection accurately identifies the type of repair option for each vehicle body component from one or more images. This is typically combined with business feedback logic, using pre-designed post-processing logic to generate appropriate repair options based on model detection.

[0004] However, the applicant has found that due to the large number of vehicle body parts in the image, different parts are easily misdetected due to their similar appearance or close location, resulting in inaccurate predictions in traditional vehicle component maintenance plan detection. Summary of the Invention

[0005] The purpose of the embodiments of the present application is to propose a vehicle maintenance plan prediction method, apparatus, computer equipment and storage medium to solve the problem of inaccurate prediction of traditional vehicle component maintenance plan detection.

[0006] In order to solve the above technical problems, the present application provides a method for predicting vehicle maintenance plans, which adopts the following technical solutions:

[0007] Reading a system database, and obtaining historical maintenance data from the system database, wherein the historical maintenance data includes historical damaged images, historical maintenance types, and historical maintenance plans;

[0008] receiving annotation information corresponding to the historical damaged image sent by the annotation terminal, wherein the annotation information includes damaged location information and damaged component category;

[0009] Constructing a model training set based on the historical maintenance data and the annotation information;

[0010] call an initial yolov7 detection model, and input the damaged position information, the historical repair scheme and the damaged component category into the initial yolov7 detection model as a damaged position label, a repair scheme label and a damaged component label when the initial yolov7 detection model calculates a loss, wherein a convolution layer of a CBS module in a backbone feature extraction network of the initial yolov7 detection model is a pyramid squeeze attention module;

[0011] input the model training set into the initial yolov7 detection model, and perform a model training operation on the initial yolov7 detection model according to a damaged position loss function, a repair scheme loss function and a component category loss function, to obtain a trained repair scheme prediction model;

[0012] receive a repair scheme prediction request carrying a to-be-detected damaged image sent by a requesting terminal;

[0013] input the to-be-detected damaged image into the trained repair scheme prediction model to perform a repair scheme prediction operation, and obtain a repair scheme prediction result.

[0014] Further, in the reading system database, after the step of obtaining historical repair data in the system database, the following steps are further included:

[0015] perform denoising processing on the historical damaged image according to a wavelet transform algorithm, to obtain a denoised damaged image;

[0016] perform contrast enhancement processing on the denoised damaged image according to a histogram equalization algorithm, to obtain an enhanced damaged image;

[0017] perform edge detection processing on the enhanced damaged image according to a Laplacian operator, to obtain a preprocessed historical damaged image.

[0018] Further, the model training set includes a confusion category training set, and the step of constructing a model training set according to the historical repair data and the annotation information specifically includes the following steps:

[0019] obtain an intersection-over-union ratio of an annotation box of a historical damaged image carrying the annotation information;

[0020] calculate an intersection-over-union overlap value between historical damaged images of different damaged component categories according to the intersection-over-union ratio of the annotation box, respectively;

[0021] compose historical damaged images corresponding to intersection-over-union overlap values satisfying a preset intersection-over-union overlap threshold into the confusion category training set, wherein the confusion category training set is annotated with a confusion component category;

[0022] The step of calling the initial yolov7 detection model and using the damaged location information, the historical repair plan, and the damaged component category as the damaged location label, the repair plan label, and the damaged component label when the initial yolov7 detection model calculates the loss specifically includes the following steps:

[0023] Using the confused component category as the confused label when calculating the loss of the initial yolov7 detection model;

[0024] The step of inputting the model training set into the initial YOLOv7 detection model, and performing a model training operation on the initial YOLOv7 detection model according to the damaged location loss function, the maintenance solution loss function, and the component category loss function to obtain a trained maintenance solution prediction model specifically includes the following steps:

[0025] The confusion category training set is input into the initial yolov7 detection model, and the initial yolov7 detection model is trained according to the confusion category loss function to obtain the trained maintenance plan prediction model.

[0026] Furthermore, the confusion category loss function is expressed as:

[0027]

[0028] Among them, f gt Represents the predicted probability of the true category of the prediction box, f pred Represents the predicted probability of the set of confused categories corresponding to a single category of the true category of the prediction box.

[0029] Furthermore, after the step of inputting the damaged image to be detected into the trained maintenance solution prediction model to perform a maintenance solution prediction operation and obtain a maintenance solution prediction result, the following steps are also included:

[0030] Performing rationality verification on the maintenance plan prediction result to obtain a maintenance plan verification result;

[0031] The maintenance plan prediction result is optimized and adjusted according to the maintenance plan verification result to obtain a target maintenance plan.

[0032] Furthermore, after the step of inputting the damaged image to be detected into the trained maintenance solution prediction model to perform a maintenance solution prediction operation and obtain a maintenance solution prediction result, the following steps are also included:

[0033] The maintenance plan prediction result is displayed in a visual form, wherein the visualization includes a schematic diagram of the damaged location and a description of the maintenance steps.

[0034] In order to solve the above technical problems, the embodiment of the present application further provides a vehicle maintenance plan prediction device, which adopts the following technical solution:

[0035] A historical data acquisition module is used to read a system database and acquire historical maintenance data from the system database, wherein the historical maintenance data includes historical damaged images, historical maintenance types, and historical maintenance plans;

[0036] a marking information acquisition module, configured to receive marking information corresponding to the historical damaged image sent by a marking terminal, wherein the marking information includes damaged location information and damaged component category;

[0037] A training set construction module, configured to construct a model training set based on the historical maintenance data and the annotation information;

[0038] A loss label determination module is used to call the initial Yolov7 detection model and use the damaged location information, the historical repair plan, and the damaged component category as the damaged location label, the repair plan label, and the damaged component label when the initial Yolov7 detection model calculates the loss, wherein the convolution layer of the CBS module in the backbone feature extraction network of the initial Yolov7 detection model is a pyramid squeeze attention module;

[0039] A model training module is used to input the model training set into the initial YOLOv7 detection model, and perform a model training operation on the initial YOLOv7 detection model according to the damaged location loss function, the repair solution loss function and the component category loss function to obtain a trained repair solution prediction model;

[0040] a request receiving module, configured to receive a repair plan prediction request carrying a damaged image to be inspected, sent by a requesting terminal;

[0041] The solution prediction module is used to input the damaged image to be detected into the trained maintenance solution prediction model to perform a maintenance solution prediction operation and obtain a maintenance solution prediction result.

[0042] Furthermore, the device further comprises:

[0043] a denoising module, configured to perform denoising processing on the historical damaged image according to a wavelet transform algorithm to obtain a denoised damaged image;

[0044] A contrast enhancement module, configured to perform contrast enhancement processing on the denoised damaged image according to a histogram equalization algorithm to obtain an enhanced damaged image;

[0045] An edge detection module is configured to perform edge detection on the enhanced damaged image according to a Laplace operator to obtain a preprocessed historical damaged image.

[0046] To solve the above technical problems, the embodiment of the application further provides a computer device, which adopts the technical scheme as follows:

[0047] The computer device comprises a memory and a processor, the memory stores computer readable instructions, and the processor implements the steps of the vehicle maintenance scheme prediction method as described above when executing the computer readable instructions.

[0048] To solve the above technical problems, the embodiment of the application further provides a computer readable storage medium, which adopts the technical scheme as follows:

[0049] The computer readable storage medium stores computer readable instructions, and the computer readable instructions are executed by a processor to implement the steps of the vehicle maintenance scheme prediction method as described above.

[0050] The present application provides a vehicle maintenance plan prediction method, comprising: reading a system database, obtaining historical maintenance data from the system database, wherein the historical maintenance data includes historical damaged images, historical maintenance types, and historical maintenance plans; receiving annotation information corresponding to the historical damaged images sent by an annotation terminal, wherein the annotation information includes damaged location information and damaged component categories; constructing a model training set based on the historical maintenance data and the annotation information; calling an initial Yo L OV7 detection model, and using the damaged location information, the historical maintenance plan, and the damaged component category as damaged location labels, maintenance plan labels, and damaged component labels when the initial Yo L OV7 detection model calculates losses, respectively, wherein the convolution layer of the CBS module in the backbone feature extraction network of the initial Yo L OV7 detection model is a pyramid squeezed attention module; inputting the model training set into the initial Yo L OV7 detection model, and performing an on-line feature extraction on the initial Yo L OV7 detection model according to a damaged location loss function, a maintenance plan loss function, and a component category loss function. The ov7 detection model performs a model training operation to obtain a trained repair solution prediction model; a repair solution prediction request containing a damaged image to be detected is received from a requesting terminal; the damaged image to be detected is input into the trained repair solution prediction model to perform a repair solution prediction operation to obtain a repair solution prediction result. Compared with the prior art, this application optimizes the underlying convolutional layer structure of the network model backbone to more fully extract multi-scale feature map spatial information and realize cross-dimensional channel attention important feature interaction; based on the large number of vehicle body parts in the target task, and the close location or similar appearance of parts that easily lead to mutual false detection, a new loss function is introduced by statistically analyzing the confusion category set of a single class to optimize the false detection problem of vehicle body parts, and achieve more robust and reliable prediction of damage assessment results for different parts of the entire case. BRIEF DESCRIPTION OF THE DRAWINGS

[0051] In order to more clearly illustrate the solutions in this application, a brief introduction will be given below to the drawings required for use in the description of the embodiments of this application. Obviously, the drawings described below are some embodiments of this application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0052] Figure 1 is an exemplary system architecture diagram to which the present application may be applied;

[0053] Figure 2 This is a flowchart of the vehicle maintenance plan prediction method provided in Example 1 of the present application;

[0054] Figure 3 This is a schematic diagram of the structure of the vehicle maintenance plan prediction device provided in Example 2 of the present application;

[0055] Figure 4 It is a structural diagram of an embodiment of a computer device according to the present application. DETAILED DESCRIPTION

[0056] Unless otherwise defined, all technical and scientific terms used herein have the same meanings as commonly understood by those skilled in the art to which this application belongs. The terms used in the specification of the application are for the purpose of describing specific embodiments only and are not intended to limit this application. The terms "including" and "having" and any variations thereof in the specification and claims of this application and the above-mentioned drawings are intended to cover non-exclusive inclusions. The terms "first", "second", etc. in the specification and claims of this application or the above-mentioned drawings are used to distinguish different objects, not to describe a specific order.

[0057] References herein to "embodiments" mean that a particular feature, structure, or characteristic described in connection with the embodiments may be included in at least one embodiment of the present application. The appearance of this phrase in various places in the specification does not necessarily refer to the same embodiment, nor does it constitute an independent or alternative embodiment that is mutually exclusive of other embodiments. It is understood, both explicitly and implicitly, by those skilled in the art that the embodiments described herein may be combined with other embodiments.

[0058] In order to enable those skilled in the art to better understand the solution of the present application, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the accompanying drawings.

[0059] like Figure 1 As shown, system architecture 100 may include a terminal device 101, a network 102, and a server 103. Terminal device 101 may be a laptop computer 1011, a tablet computer 1012, or a mobile phone 1013. Network 102 is a medium for providing a communication link between terminal device 101 and server 103. Network 102 may include various connection types, such as wired or wireless communication links or fiber optic cables.

[0060] The user can use the terminal device 101 to interact with the server 103 via the network 102 to receive or send messages, etc. Various communication client applications can be installed on the terminal device 101, such as web browser applications, shopping applications, search applications, instant messaging tools, email clients, social platform software, etc.

[0061] The terminal device 101 can be various electronic devices with a display screen and supporting web browsing, in addition to the notebook computer 1011, the tablet computer 1012 or the mobile phone 1013, the terminal device 101 can also be an electronic book reader, an MP3 player (Movi ng Pi cture Experts Group Aud i o Layer I I I, dynamic image expert compression standard audio layer I I I), an MP4 (Movi ng Pi cture Experts Group Aud i o Layer IV, dynamic image expert compression standard audio layer IV) player, a laptop computer and a desktop computer, etc.

[0062] The server 103 can be a server providing various services, for example, a background server supporting the page displayed on the terminal device 101.

[0063] It should be noted that the vehicle maintenance scheme prediction method provided in the embodiments of the present application is generally executed by a server / terminal device, and accordingly, the vehicle maintenance scheme prediction apparatus is generally arranged in the server / terminal device.

[0064] It should be understood that, Figure 1 The number of terminal devices, networks and servers in

[0065] Embodiment one

[0066] With reference to Figure 2 , a flow chart of one embodiment of the vehicle maintenance scheme prediction method according to the present application is shown. The vehicle maintenance scheme prediction method comprises: step S201, step S202, step S203, step S204, step S205, step S206 and step S207.

[0067] In step S201, a system database is read, and historical maintenance data is obtained from the system database, wherein the historical maintenance data comprises historical damaged images, historical maintenance types and historical maintenance schemes.

[0068] In the embodiments of the present application, historical pictures of component maintenance schemes are screened out from the system database of the vehicle maintenance business scenario, and the component maintenance scheme categories and labeling criteria are defined, including 5 kinds of component maintenance schemes of paint spraying, small sheet metal, medium sheet metal, large sheet metal and replacement, and the maintenance schemes for the damaged positions of the corresponding damaged components.

[0069] In step S202, the labeling information corresponding to the historical damaged images sent by the labeling terminal is received, wherein the labeling information comprises damaged position information and damaged component categories.

[0070] In an embodiment of the present application, based on the component repair plan form defined in step S201, historical pictures of the vehicle body component repair plan are manually annotated in batches. The component repair plan form appearing in each picture is marked with a rectangular frame, and its component repair plan type, repair plan and the name label of the vehicle body component to which it belongs are recorded.

[0071] In step S203, a model training set is constructed based on the historical maintenance data and annotation information.

[0072] In step S204, the initial yolov7 detection model is called, and the damaged location information, historical repair plan and damaged component category are used as the damaged location label, repair plan label and damaged component label when the initial yolov7 detection model calculates the loss, respectively. Among them, the convolution layer of the CBS module in the backbone feature extraction network of the initial yolov7 detection model is a pyramid squeeze attention module.

[0073] In an embodiment of the present application, the marked component repair plan location and repair plan are used as the location and category labels when the network calculates the loss, and the component category is used as a secondary label. The selected network model is used to learn and infer the damaged component location, repair plan and component category of the output image.

[0074] In an embodiment of the present application, yo l ov7 is selected as the basic network model, and the 3*3 convolution module of the backbone is replaced with the PSA module to fully extract the spatial information of the multi-scale feature map and realize the interaction of important features of cross-dimensional channel attention, effectively establish the long-term dependency relationship between multi-scale channel attention, improve the backbone module's multi-level feature acquisition of the vehicle, and assign different attention weights, so that the model pays attention to the vehicle parts with more serious parts maintenance solutions to improve the accuracy of detection.

[0075] In step S205, the model training set is input into the initial yolov7 detection model, and the initial yolov7 detection model is trained according to the damaged location loss function, the maintenance plan loss function and the component category loss function to obtain a trained maintenance plan prediction model.

[0076] In step S206, a repair solution prediction request carrying a damaged image to be detected is received from a requesting terminal.

[0077] In step S207, the damaged image to be detected is input into the trained maintenance plan prediction model to perform a maintenance plan prediction operation to obtain a maintenance plan prediction result.

[0078] In an embodiment of the present application, a vehicle maintenance plan prediction method is provided, comprising: reading a system database, obtaining historical maintenance data from the system database, wherein the historical maintenance data includes historical damaged images, historical maintenance types, and historical maintenance plans; receiving annotation information corresponding to the historical damaged images sent by an annotation terminal, wherein the annotation information includes damaged location information and damaged component categories; constructing a model training set based on the historical maintenance data and the annotation information; calling an initial YOLOv7 detection model, and using the damaged location information, the historical maintenance plan, and the damaged component category as damaged location labels, maintenance plan labels, and damaged component labels when the initial YOLOv7 detection model calculates losses, respectively, wherein the convolution layer of the CBS module in the backbone feature extraction network of the initial YOLOv7 detection model is a pyramid squeezed attention module; inputting the model training set into the initial YOLOv7 detection model, and performing a model training operation on the initial YOLOv7 detection model according to a damaged location loss function, a maintenance plan loss function, and a component category loss function to obtain a trained maintenance plan prediction model; receiving a maintenance plan prediction request carrying a damaged image to be detected sent by a request terminal; inputting the damaged image to be detected into the trained maintenance plan prediction model to perform a maintenance plan prediction operation to obtain a maintenance plan prediction result. Compared with the existing technology, this application optimizes the underlying convolutional layer structure of the network model backbone to more fully extract the spatial information of multi-scale feature maps and realize the interaction of important features of cross-dimensional channel attention; since there are many body parts in the target task, and the parts are close in position or similar in appearance, it is easy to cause mutual false detection. A new loss function is introduced by statistically analyzing the confusion category set of a single class to optimize the false detection problem of body parts and achieve more robust and reliable prediction of the damage assessment results of different parts of the whole case.

[0079] In some optional implementations of the embodiments of the present application, after step S201, the following steps are further included:

[0080] De-noising the historical damaged image according to the wavelet transform algorithm to obtain a de-noised damaged image;

[0081] Performing contrast enhancement processing on the denoised damaged image according to the histogram equalization algorithm to obtain an enhanced damaged image;

[0082] The enhanced damaged image is subjected to edge detection processing according to the Laplace operator to obtain a preprocessed historical damaged image.

[0083] In an embodiment of the present application, a batch of damaged vehicle images are obtained as a training data set based on a pre-established vehicle damaged image database.

[0084] In an embodiment of the present application, a wavelet transform algorithm is used to perform denoising on the acquired image of the damaged vehicle to remove high-frequency noise in the image and obtain a denoised image of the damaged vehicle.

[0085] In an embodiment of the present application, the denoised damaged vehicle image is contrast enhanced by a histogram equalization algorithm to improve the contrast and clarity of the image, thereby obtaining an enhanced damaged vehicle image.

[0086] In an embodiment of the present application, the Laplace operator is used to perform edge detection on the enhanced damaged vehicle image to highlight the edge contour of the damaged area of ​​the vehicle, thereby obtaining an edge-enhanced damaged vehicle image.

[0087] In some optional implementations of the embodiments of the present application, the model training set includes a confusion category training set, and the step S203 specifically includes the following steps:

[0088] Obtain the intersection-over-union ratio of the annotation boxes of the historical damaged image that carries the annotation information;

[0089] Calculate the overlap values ​​of the historical damaged images of different damaged component categories based on the intersection-over-union ratio of the annotation boxes;

[0090] The historical damaged images corresponding to the IoU overlap values ​​that meet a preset IoU overlap threshold are used to form a confusion category training set, wherein the confusion category training set is annotated with confusion component categories;

[0091] The above step S204 specifically includes the following steps:

[0092] The confused part category is used as the confused label when calculating the loss of the initial yolov7 detection model;

[0093] The above step S205 specifically includes the following steps:

[0094] The confusion category training set is input into the initial yolov7 detection model, and the initial yolov7 detection model is trained according to the confusion category loss function to obtain a trained maintenance plan prediction model.

[0095] In the embodiment of the present application, in addition to the position component maintenance plan function, classification loss function and confidence loss function of the yo l ov7 model itself, the iou values ​​of the component maintenance plan component boxes of different component categories in the training set are statistically analyzed, and the potential mutual false detection component categories with higher iou overlap between the labeling boxes of different types of components are selected to form a confusion category set corresponding to a single category, and the confusion category set corresponding to the single category with all higher iou of a single component category is recorded.

[0096] In the embodiment of the present application, the different types of markings with higher i ou include two situations:

[0097] (1) There is a door trim panel on the car door, and there may be a door trim strip component on the door trim panel. Both may be marked with the component repair plan at the same time. The two are located close to each other and may be mistakenly identified as different components during model detection;

[0098] (2) Parts with similar appearance may be misidentified with each other (this can be determined by analyzing the test set). Therefore, a single part category A may be a part pair with multiple parts of categories X, Y, and Z.

[0099] In an embodiment of the present application, a set of component categories with a high degree of correlation in the target is analyzed according to target task statistics, and a penalty term is introduced into the loss function to optimize the problem of false detection of target components.

[0100] In some optional implementations of the embodiments of the present application, the above confusion category loss function is expressed as:

[0101]

[0102] Among them, f gt Represents the predicted probability of the true category of the prediction box, f pred Represents the predicted probability of the set of confused categories corresponding to a single category of the true category of the prediction box.

[0103] In the embodiment of the present application, when calculating the loss function, the loss of all relevant potential mutually misdetected components of the target category is calculated based on the set of confusion categories corresponding to the single category of all components obtained through statistics and testing. where f gt is the predicted probability of the true category of the prediction box, f pred The predicted probability of the set of confused categories corresponding to the single category of the predicted box's true category is finally summed up and added to the total loss function of the model for calculation.

[0104] In the embodiment of the present application, when the predicted probability a of the true category gt is equal to the predicted probability b of a certain category in the corresponding confusion category set, (ab)=1, then 1-(ab)=0, and the loss function value is 0.

[0105] In some optional implementations of the embodiments of the present application, after step S207, the following steps are further included:

[0106] Perform rationality verification on the maintenance plan prediction results to obtain maintenance plan verification results;

[0107] According to the maintenance plan verification results, the maintenance plan prediction results are optimized and adjusted to obtain the target maintenance plan.

[0108] In the embodiment of the present application, in order to ensure the feasibility and practicality of the prediction results, it is necessary to verify their rationality. This rationality verification can use simulation technology to virtually verify the maintenance plan and evaluate the feasibility and effectiveness of the plan. Specifically:

[0109] 1. Virtual construction of maintenance plan:

[0110] (1) Data collection: First, it is necessary to collect detailed data of the maintenance object, including the technical parameters of the equipment, structural drawings, maintenance history records, etc. These data are the basis for building a virtual model.

[0111] (2) Model building: Using tools such as 3D modeling software, a virtual model of the maintenance object is built based on the collected data. The model should reflect the actual structure and performance of the equipment as accurately as possible.

[0112] (3) Scenario Construction: Build a maintenance scenario in a virtual environment, including the location of the equipment, the layout of the surrounding environment, the operating space of the maintenance personnel, etc. Ensure that the scenario is consistent with the actual situation so that a realistic simulation test can be carried out.

[0113] 2. Virtual verification of maintenance plans:

[0114] (1) Plan input: The maintenance plan to be verified is input into the simulation system, including maintenance steps, required tools, personnel configuration and other information.

[0115] (2) Simulation execution: Execute the maintenance plan in a virtual environment, simulating the maintenance personnel's operation process. The system will judge the rationality and effectiveness of the maintenance plan based on the preset parameters and logic.

[0116] (3) Performance evaluation: Through simulation, the maintenance solution is evaluated in terms of:

[0117] ① Technical feasibility: Check whether the technical measures in the maintenance plan are feasible and whether there are any technical obstacles.

[0118] ② Economic rationality: Evaluate the cost-effectiveness of the maintenance plan, including maintenance costs, downtime losses, etc.

[0119] ③ Safety compliance: Verify whether the maintenance plan complies with safety standards and regulatory requirements to ensure the safety of personnel and equipment during the maintenance process.

[0120] Time efficiency: Evaluate the execution time of maintenance plans to ensure that maintenance tasks are completed within the predetermined time window.

[0121] In the embodiment of the present application, the plan is optimized and adjusted according to the maintenance plan verification result to generate a final maintenance plan.

[0122] In the embodiments of the present application, the use of simulation technology to virtually verify the maintenance plan is a scientific and efficient method. It can help us identify problems and deficiencies in the maintenance plan in advance and make targeted optimizations and adjustments, thereby ensuring the smooth progress of maintenance work and the normal operation of equipment.

[0123] In some optional implementations of the embodiments of the present application, after the above step S207, the following steps are further included:

[0124] The repair plan prediction results are displayed in a visual form, where the visualization includes a schematic diagram of the damaged location and instructions for repair steps.

[0125] In the embodiment of the present application, the maintenance plan is presented to the maintenance personnel in a visual form, including the following, to assist the maintenance personnel in performing accurate and efficient maintenance operations. Specifically:

[0126] 1. Schematic diagram of damaged location:

[0127] A damage location diagram is an integral part of the repair plan prediction results. It graphically displays the specific location of damage in the equipment or structure. This diagram typically includes the following information:

[0128] (1) Overall view of the equipment: First, provide an overall view of the equipment or structure to help the audience establish a holistic understanding of the maintenance object.

[0129] (2) Highlighting of damaged areas: In the overall view, the damaged areas are highlighted using different colors, shadows, or borders to make them clear at a glance.

[0130] (3) Display of damage details: For areas with more serious damage, detailed drawings or cross-sectional drawings can be provided to show the specific form and extent of the damage and the possible scope of impact.

[0131] (4) Marking information: Mark the name, number, location coordinates and other information of the damaged part in the schematic diagram to facilitate accurate positioning of subsequent repair work.

[0132] 2. Repair steps:

[0133] The repair step description is another important visual element in the repair plan prediction results. It describes in detail the entire process from damage discovery to repair completion. To enhance readability and comprehension, the repair step description is usually visualized in the following form:

[0134] (1) Flowchart: Use a flowchart to show the logical sequence and process relationships of the maintenance steps. Each step is represented by a graphic block, and arrows are used to connect the steps to show the sequence and dependency relationships.

[0135] (2) Step number and title: Assign a unique number and a concise title to each maintenance step to help the audience quickly locate and understand the content of each step.

[0136] (3) Step description: Provide a detailed description of each step next to or below the flowchart, including required tools, materials, personnel configuration, operation points, etc. The description should be as concise as possible and avoid using too many professional terms.

[0137] (4) Illustration assistance: For some complex maintenance steps, illustrations or animations can be used to assist in the explanation. Illustrations can show key operations or equipment state changes during the maintenance process, and animations can simulate the dynamic effects of the maintenance process, making it easier for the audience to understand the maintenance steps.

[0138] In the embodiments of the present application, the damaged location diagram and the maintenance step explanation are combined to form a complete and intuitive maintenance scheme prediction result visualization report. This report not only helps technicians quickly understand the maintenance requirements and steps, but also provides decision support for management, ensuring the smooth progress of maintenance and the timely recovery of equipment. In the comprehensive display, other visualization elements such as timeline, progress bar, risk prompt, etc. can be added to further improve the information content and readability of the report.

[0139] In some optional implementations of the embodiments of the present application, the generated maintenance scheme is added to the knowledge base, enriching the case data of the knowledge base and realizing the continuous accumulation and optimization of knowledge.

[0140] The embodiments of the present application can acquire and process related data based on artificial intelligence technology. Artificial intelligence (AI) is the use of digital computers or computer-controlled machines to simulate, extend and expand human intelligence, perceive the environment, acquire knowledge and use knowledge to obtain the best results.

[0141] The basic technology of artificial intelligence generally includes technologies such as sensors, special artificial intelligence chips, cloud computing, distributed storage, big data processing, operation / interaction systems, mechatronics, etc. Artificial intelligence software technology mainly includes computer vision technology, robot technology, biometric technology, speech processing technology, natural language processing technology, and machine learning / deep learning, etc.

[0142] Those skilled in the art will appreciate that all or part of the processes in the above-described method embodiments can be implemented by instructing related hardware via computer-readable instructions. The computer-readable instructions can be stored in a computer-readable storage medium, and when the program is executed, it can include the processes in the above-described method embodiments. The aforementioned storage medium can be a non-volatile storage medium such as a magnetic disk, an optical disk, a read-only memory (ROM), or a random access memory (RAM).

[0143] It should be understood that although the steps in the flowcharts of the accompanying drawings are shown in sequence as indicated by the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless otherwise specified herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some of the steps in the flowcharts of the accompanying drawings may include multiple sub-steps or multiple stages, and these sub-steps or stages are not necessarily executed at the same time, but can be executed at different times, and their execution order is not necessarily sequential, but can be executed in turn or alternately with other steps or at least a portion of the sub-steps or stages of other steps.

[0144] Example 2

[0145] Further references Figure 3 , as a response to the above Figure 2 The present application provides an embodiment of a vehicle maintenance plan prediction device. Figure 2 Corresponding to the method embodiment shown, the device can be specifically applied to various electronic devices.

[0146] like Figure 3 As shown, the vehicle maintenance plan prediction device 200 of the embodiment of the present application includes:

[0147] The historical data acquisition module 210 is used to read the system database and obtain historical maintenance data from the system database, wherein the historical maintenance data includes historical damaged images, historical maintenance types, and historical maintenance plans;

[0148] The annotation information acquisition module 220 is configured to receive annotation information corresponding to the historical damaged image sent by the annotation terminal, wherein the annotation information includes damaged location information and damaged component category;

[0149] A training set construction module 230 is used to construct a model training set based on historical maintenance data and annotation information;

[0150] The loss label determination module 240 is used to call the initial Yolov7 detection model and use the damaged location information, historical repair plan, and damaged component category as the damaged location label, repair plan label, and damaged component label when the initial Yolov7 detection model calculates the loss, wherein the convolution layer of the CBS module in the backbone feature extraction network of the initial Yolov7 detection model is a pyramid squeeze attention module;

[0151] The model training module 250 is used to input the model training set into the initial Yolov7 detection model, and perform a model training operation on the initial Yolov7 detection model according to the damaged location loss function, the repair solution loss function and the component category loss function to obtain a trained repair solution prediction model;

[0152] The request receiving module 260 is configured to receive a repair solution prediction request including a damaged image to be inspected, sent by a requesting terminal;

[0153] The solution prediction module 270 is used to input the damaged image to be detected into the trained maintenance solution prediction model to perform a maintenance solution prediction operation and obtain a maintenance solution prediction result.

[0154] In an embodiment of the present application, a vehicle maintenance plan prediction device 200 is provided, comprising: a historical data acquisition module 210 for reading a system database and acquiring historical maintenance data in the system database, wherein the historical maintenance data includes historical damaged images, historical maintenance types, and historical maintenance plans; an annotation information acquisition module 220 for receiving annotation information corresponding to historical damaged images sent by an annotation terminal, wherein the annotation information includes damaged location information and damaged component categories; a training set construction module 230 for constructing a model training set based on the historical maintenance data and annotation information; a loss label determination module 240 for calling an initial yolov7 detection model and calculating the damaged location information, historical maintenance plans, and damaged component categories as the initial yolov7 detection model. The damaged position label, repair plan label and damaged component label at the time of loss, wherein the convolution layer of the CBS module in the backbone feature extraction network of the initial yolov7 detection model is a pyramid squeeze attention module; the model training module 250 is used to input the model training set into the initial yolov7 detection model, and perform a model training operation on the initial yolov7 detection model according to the damaged position loss function, the repair plan loss function and the component category loss function to obtain a trained repair plan prediction model; the request receiving module 260 is used to receive a repair plan prediction request carrying a damaged image to be detected sent by the request terminal; the solution prediction module 270 is used to input the damaged image to be detected into the trained repair plan prediction model to perform a repair plan prediction operation to obtain a repair plan prediction result. Compared with the existing technology, this application optimizes the underlying convolutional layer structure of the network model backbone to more fully extract the spatial information of multi-scale feature maps and realize the interaction of important features of cross-dimensional channel attention; since there are many body parts in the target task, and the parts are close in position or similar in appearance, it is easy to cause mutual false detection. A new loss function is introduced by statistically analyzing the confusion category set of a single class to optimize the false detection problem of body parts and achieve more robust and reliable prediction of the damage assessment results of different parts of the whole case.

[0155] In some optional implementations of the embodiments of the present application, the vehicle maintenance plan prediction device 200 further includes:

[0156] A denoising module is used to perform denoising on the historical damaged image according to a wavelet transform algorithm to obtain a denoised damaged image;

[0157] A contrast enhancement module is used to perform contrast enhancement processing on the denoised damaged image according to a histogram equalization algorithm to obtain an enhanced damaged image;

[0158] The edge detection module is used to perform edge detection processing on the enhanced damaged image according to the Laplace operator to obtain a pre-processed historical damaged image.

[0159] To solve the above technical problems, the embodiment of the present application further provides a computer device. For details, please refer to Figure 4 , Figure 4 The basic structure block diagram of the computer device of the embodiment of the present application is shown in FIG. 3.

[0160] The computer device 300 comprises a memory 310, a processor 320 and a network interface 330 which are connected to each other through a system bus. It should be noted that only the computer device 300 with components 310-330 is shown in the figure, but it should be understood that it is not required to implement all the shown components, and more or less components can be alternatively implemented. Among them, the computer device herein is a device capable of automatically performing numerical calculation and / or information processing according to pre-set or stored instructions, which hardware includes but is not limited to microprocessor, application specific integrated circuit (ASIC), field-programmable gate array (FPGA), digital signal processor (DSP), embedded device, etc.

[0161] The computer device can be a desktop computer, a notebook computer, a palm computer, a cloud server and the like. The computer device can interact with the user through a keyboard, a mouse, a remote controller, a touchpad, a voice control device and the like.

[0162] The memory 310 includes at least one type of readable storage medium, including flash memory, hard disk, multimedia card, card-type memory (e.g., SD or DX memory), random access memory (RAM), static random access memory (SRAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), programmable read-only memory (PROM), magnetic storage, magnetic disk, optical disk, etc. In some embodiments, the memory 310 may be an internal storage unit of the computer device 300, such as a hard disk or memory of the computer device 300. In other embodiments, the memory 310 may also be an external storage device of the computer device 300, such as a plug-in hard disk equipped on the computer device 300, a smart media card (SMC), a secure digital (SD) card, a flash card, etc. Of course, the memory 310 may also include both the internal storage unit of the computer device 300 and its external storage device. In the embodiment of the present application, the memory 310 is generally used to store the operating system and various application software installed on the computer device 300, such as computer-readable instructions for the vehicle maintenance plan prediction method. In addition, the memory 310 can also be used to temporarily store various data that has been output or is about to be output.

[0163] In some embodiments, the processor 320 can be a central processing unit (CPU), a controller, a microcontroller, a microprocessor, or other data processing chip. The processor 320 is generally used to control the overall operation of the computer device 300. In the embodiment of the present application, the processor 320 is used to execute computer-readable instructions or process data stored in the memory 310, such as computer-readable instructions for executing the vehicle maintenance plan prediction method.

[0164] The network interface 330 may include a wireless network interface or a wired network interface. The network interface 330 is generally used to establish a communication connection between the computer device 300 and other electronic devices.

[0165] The computer device provided in this application optimizes the underlying convolutional layer structure of the network model backbone to more fully extract the spatial information of the multi-scale feature map and realize the interaction of important features of cross-dimensional channel attention; since there are many vehicle body parts in the target task, and the parts are close in position or similar in appearance, it is easy to cause mutual false detection. A new loss function is introduced by statistically analyzing the confusion category set of a single class to optimize the false detection problem of vehicle body parts and achieve more robust and reliable prediction of the damage assessment results of different parts of the whole case.

[0166] The present application also provides another embodiment, namely, providing a computer-readable storage medium, which stores computer-readable instructions, and the computer-readable instructions can be executed by at least one processor to enable the at least one processor to perform the steps of the vehicle maintenance plan prediction method as described above.

[0167] The computer-readable storage medium provided in this application optimizes the underlying convolutional layer structure of the network model backbone to more fully extract the spatial information of the multi-scale feature map and realize the interaction of important features of cross-dimensional channel attention; since there are many vehicle body parts in the target task, and the parts are close in position or similar in appearance, it is easy to cause mutual false detection. A new loss function is introduced by statistically analyzing the confusion category set of a single class to optimize the false detection problem of vehicle body parts and achieve more robust and reliable prediction of the damage assessment results of different parts of the whole case.

[0168] Through the description of the above implementation methods, those skilled in the art can clearly understand that the above-mentioned embodiment methods can be implemented by means of software plus the necessary general hardware platform, and of course can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art, can be embodied in the form of a software product, which is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk), and includes a number of instructions for enabling a terminal device (which can be a mobile phone, computer, server, air conditioner, or network device, etc.) to execute the methods described in each embodiment of the present application.

[0169] Obviously, the embodiments described above are only some of the embodiments of the present application, rather than all of the embodiments. The preferred embodiments of the present application are given in the accompanying drawings, but they do not limit the patent scope of the present application. The present application can be implemented in many different forms. On the contrary, the purpose of providing these embodiments is to make the understanding of the disclosure of the present application more thorough and comprehensive. Although the present application has been described in detail with reference to the aforementioned embodiments, for those skilled in the art, it is still possible to modify the technical solutions described in the aforementioned specific embodiments, or to make equivalent replacements for some of the technical features therein. Any equivalent structure made using the contents of the present application specification and the accompanying drawings, directly or indirectly used in other related technical fields, is also within the scope of patent protection of the present application.

Claims

1. A vehicle maintenance plan prediction method, characterized in that: The steps include: Reading a system database, and obtaining historical maintenance data from the system database, wherein the historical maintenance data includes historical damaged images, historical maintenance types, and historical maintenance plans; receiving annotation information corresponding to the historical damaged image sent by the annotation terminal, wherein the annotation information includes damaged location information and damaged component category; Constructing a model training set based on the historical maintenance data and the annotation information; Calling the initial Yolov7 detection model, and using the damaged location information, the historical repair plan, and the damaged component category as the damaged location label, the repair plan label, and the damaged component label when the initial Yolov7 detection model calculates the loss, wherein the convolution layer of the CBS module in the backbone feature extraction network of the initial Yolov7 detection model is a pyramid squeeze attention module; Inputting the model training set into the initial YOLOv7 detection model, and performing a model training operation on the initial YOLOv7 detection model according to the damaged location loss function, the repair solution loss function, and the component category loss function to obtain a trained repair solution prediction model; receiving a repair plan prediction request including a damaged image to be inspected, sent by a requesting terminal; Inputting the damaged image to be detected into the trained maintenance solution prediction model to perform a maintenance solution prediction operation to obtain a maintenance solution prediction result; The model training set also includes a confusion category training set, and the step of constructing the model training set based on the historical maintenance data and the annotation information further includes the following steps: Obtaining an intersection-over-union ratio of the annotation boxes of the historical damaged image carrying the annotation information; Calculating the intersection-over-union (IoU) overlap values ​​between historical damaged images of different damaged component categories according to the IoU of the annotation boxes; The historical damaged images corresponding to the IoU overlap values ​​that meet a preset IoU overlap threshold form the confusion category training set, wherein the confusion category training set is annotated with confusion component categories; The step of calling the initial yolov7 detection model and using the damaged location information, the historical repair plan, and the damaged component category as the damaged location label, the repair plan label, and the damaged component label when the initial yolov7 detection model calculates the loss specifically includes the following steps: The damaged location information, the historical maintenance plan, the damaged component category, and the confused component category are used as the damaged location label, the maintenance plan label, the damaged component label, and the confused label when the initial yolov7 detection model calculates the loss, respectively; The step of inputting the model training set into the initial YOLOv7 detection model, and performing a model training operation on the initial YOLOv7 detection model according to the damaged location loss function, the maintenance solution loss function, and the component category loss function to obtain a trained maintenance solution prediction model specifically includes the following steps: The model training set is input into the initial YOLOv7 detection model, and the initial YOLOv7 detection model is subjected to model training operation according to the damaged location loss function, the repair plan loss function, the component category loss function and the confusion category loss function to obtain a trained repair plan prediction model.

2. The vehicle maintenance plan prediction method according to claim 1, characterized in that: After the step of reading the system database and obtaining the historical maintenance data in the system database, the following steps are also included: Performing denoising on the historical damaged image according to a wavelet transform algorithm to obtain a denoised damaged image; performing contrast enhancement processing on the denoised damaged image according to a histogram equalization algorithm to obtain an enhanced damaged image; Edge detection processing is performed on the enhanced damaged image according to the Laplace operator to obtain a pre-processed historical damaged image.

3. The vehicle maintenance plan prediction method according to claim 1, characterized in that: The confusion category loss function is expressed as: in, Represents the predicted probability of the true category of the prediction box, Represents the predicted probability of the set of confused categories corresponding to a single category of the true category of the prediction box.

4. The vehicle maintenance plan prediction method according to claim 1, characterized in that: After the step of inputting the damaged image to be detected into the trained maintenance solution prediction model to perform a maintenance solution prediction operation and obtain a maintenance solution prediction result, the following steps are also included: Performing rationality verification on the maintenance plan prediction result to obtain a maintenance plan verification result; The maintenance plan prediction result is optimized and adjusted according to the maintenance plan verification result to obtain a target maintenance plan.

5. The vehicle maintenance plan prediction method according to claim 1, characterized in that: After the step of inputting the damaged image to be detected into the trained maintenance solution prediction model to perform a maintenance solution prediction operation and obtain a maintenance solution prediction result, the following steps are also included: The maintenance plan prediction result is displayed in a visual form, wherein the visualization includes a schematic diagram of the damaged location and a description of the maintenance steps.

6. A vehicle maintenance plan prediction device, characterized in that: include: A historical data acquisition module is used to read a system database and acquire historical maintenance data from the system database, wherein the historical maintenance data includes historical damaged images, historical maintenance types, and historical maintenance plans; a marking information acquisition module, configured to receive marking information corresponding to the historical damaged image sent by a marking terminal, wherein the marking information includes damaged location information and damaged component category; A training set construction module, configured to construct a model training set based on the historical maintenance data and the annotation information; A loss label determination module is used to call the initial Yolov7 detection model and use the damaged location information, the historical repair plan, and the damaged component category as the damaged location label, the repair plan label, and the damaged component label when the initial Yolov7 detection model calculates the loss, wherein the convolution layer of the CBS module in the backbone feature extraction network of the initial Yolov7 detection model is a pyramid squeeze attention module; A model training module is used to input the model training set into the initial YOLOv7 detection model, and perform a model training operation on the initial YOLOv7 detection model according to the damaged location loss function, the repair solution loss function and the component category loss function to obtain a trained repair solution prediction model; a request receiving module, configured to receive a repair plan prediction request carrying a damaged image to be inspected, sent by a requesting terminal; A solution prediction module is used to input the damaged image to be detected into the trained maintenance solution prediction model to perform a maintenance solution prediction operation and obtain a maintenance solution prediction result; The model training set also includes a confusion category training set, and the training set construction module is further used to obtain the intersection-over-union ratio of the annotation boxes of the historical damaged images carrying the annotation information; The training set construction module is further used to calculate the intersection-over-union (IoU) overlap values ​​between historical damaged images of different damaged component categories based on the IoU of the annotation boxes; The training set construction module is further configured to form the confusion category training set from historical damaged images corresponding to intersection-over-union (IoU) overlap values ​​that meet a preset IoU overlap threshold, wherein the confusion category training set is annotated with confusion component categories; The loss label determination module is further configured to use the damaged location information, the historical maintenance plan, the damaged component category, and the confused component category as the damaged location label, the maintenance plan label, the damaged component label, and the confused label, respectively, when the initial yolov7 detection model calculates the loss; The model training module is also used to input the model training set into the initial YOLOv7 detection model, and perform model training operations on the initial YOLOv7 detection model according to the damaged location loss function, the repair plan loss function, the component category loss function and the confusion category loss function to obtain a trained repair plan prediction model.

7. The vehicle maintenance plan prediction device according to claim 6, characterized in that: The device further comprises: a denoising module, configured to perform denoising processing on the historical damaged image according to a wavelet transform algorithm to obtain a denoised damaged image; A contrast enhancement module, configured to perform contrast enhancement processing on the denoised damaged image according to a histogram equalization algorithm to obtain an enhanced damaged image; The edge detection module is used to perform edge detection processing on the enhanced damaged image according to the Laplace operator to obtain a pre-processed historical damaged image.

8. A computer device comprising a memory and a processor, characterized in that: The memory stores computer-readable instructions, and when the processor executes the computer-readable instructions, the steps of the vehicle maintenance plan prediction method according to any one of claims 1 to 5 are implemented.

9. A computer-readable storage medium, characterized in that The computer-readable storage medium stores computer-readable instructions, which, when executed by a processor, implement the steps of the vehicle maintenance plan prediction method according to any one of claims 1 to 5.

Images