Vehicle maintenance scheme prediction method and device, computer equipment and storage medium

Abstract

The embodiment of the invention belongs to the technical field of image processing, and relates to a vehicle maintenance scheme prediction method and device, computer equipment and a storage medium, and the method comprises the steps: constructing a model training set according to historical maintenance data and annotation information; respectively taking the damaged position information, the historical maintenance scheme and the damaged part as a damaged position label, a maintenance scheme label and a damaged part label when the loss is calculated by the initial yolov7 detection model; inputting the model training set into the initial yolov7 detection model, and performing model training operation on the initial yolov7 detection model according to a damaged position loss function, a maintenance scheme loss function and a part category loss function to obtain a trained maintenance scheme prediction model; and inputting a to-be-detected damaged image into the trained maintenance scheme prediction model to carry out maintenance scheme prediction operation so as to obtain a maintenance scheme prediction result. According to the method, more robust and more reliable prediction of the loss assessment results of different parts of the whole scheme is realized.

Description

Technical Field

[0001] The present application relates to the field of image processing technology, and in particular to a vehicle maintenance plan prediction method, device, computer equipment and storage medium. Background Art

[0002] With the rapid development of society, private cars have become one of the indispensable means of transportation, and the increasing number of vehicles has undoubtedly increased the incidence of traffic accidents. After a traffic accident occurs, the insurance company usually goes to the scene of the accident to determine the damage, that is, by observing the photos taken on the scene to determine the vehicle parts repair plan, which serves as the basis for the insurance company's claims. Since the damage assessment process consumes a lot of human resources and the results obtained are highly subjective, the vehicle parts repair plan detection system based on deep learning methods has gradually replaced manual operations. It can accurately detect the type of vehicle parts repair plan through one or more pictures.

[0003] Existing target detectors are mainly based on CNN, and the detection accuracy is constantly improving. They can detect various parts repair solutions for accident vehicles and quickly resolve accident claims. Vehicle parts repair solution detection is to accurately detect the types of parts repair solutions for each part of the vehicle body through one or more pictures. Usually, the appropriate repair solution results are given based on the model detection and the designed post-processing logic combined with the business feedback logic.

[0004] However, the applicant has found that due to the large number of vehicle body parts in the image, different parts are easily misdetected due to their similar appearance or close location, resulting in inaccurate predictions in traditional vehicle component maintenance solution detection. Summary of the invention

[0005] The purpose of the embodiments of the present application is to propose a vehicle maintenance plan prediction method, device, computer equipment and storage medium to solve the problem of inaccurate prediction in traditional vehicle component maintenance plan detection.

[0006] In order to solve the above technical problems, the embodiment of the present application provides a vehicle maintenance plan prediction method, which adopts the following technical solutions:

[0007] Reading a system database, and obtaining historical maintenance data in the system database, wherein the historical maintenance data includes historical damaged images, historical maintenance types, and historical maintenance plans;

[0008] Receiving annotation information corresponding to the historical damaged image sent by the annotation terminal, wherein the annotation information includes damaged location information and damaged component category;

[0009] Constructing a model training set according to the historical maintenance data and the annotation information;

[0010] Calling the initial yolov7 detection model, and using the damaged location information, the historical maintenance plan, and the damaged component category as the damaged location label, the maintenance plan label, and the damaged component label when the initial yolov7 detection model calculates the loss, wherein the convolution layer of the CBS module in the backbone feature extraction network of the initial yolov7 detection model is a pyramid squeeze attention module;

[0011] Input the model training set into the initial yolov7 detection model, and perform model training operation on the initial yolov7 detection model according to the damaged position loss function, the maintenance plan loss function and the component category loss function to obtain a trained maintenance plan prediction model;

[0012] receiving a repair plan prediction request including a damaged image to be detected, sent by a requesting terminal;

[0013] The damaged image to be detected is input into the trained maintenance plan prediction model to perform a maintenance plan prediction operation to obtain a maintenance plan prediction result.

[0014] Furthermore, after the step of reading the system database and obtaining the historical maintenance data in the system database, the following steps are also included:

[0015] Performing denoising on the historical damaged image according to a wavelet transform algorithm to obtain a denoised damaged image;

[0016] Performing contrast enhancement processing on the denoised damaged image according to a histogram equalization algorithm to obtain an enhanced damaged image;

[0017] The enhanced damaged image is subjected to edge detection processing according to the Laplace operator to obtain a pre-processed historical damaged image.

[0018] Furthermore, the model training set includes a confusion category training set, and the step of constructing the model training set according to the historical maintenance data and the annotation information specifically includes the following steps:

[0019] Obtaining an intersection-over-union ratio of a labeling box of a historical damaged image carrying the labeling information;

[0020] According to the annotation box intersection-and-union ratio, respectively calculate the intersection-and-union ratio overlap values ​​between the historical damaged images of different damaged component categories;

[0021] The historical damaged images corresponding to the IoU overlap values ​​that meet the preset IoU overlap threshold form the confusion category training set, wherein the confusion category training set is annotated with the confusion component category;

[0022] The step of calling the initial yolov7 detection model and using the damaged location information, the historical maintenance plan and the damaged component category as the damaged location label, the maintenance plan label and the damaged component label when the initial yolov7 detection model calculates the loss specifically includes the following steps:

[0023] Using the confusing component category as the confusing label when the initial yolov7 detection model calculates the loss;

[0024] The step of inputting the model training set into the initial yolov7 detection model, and performing a model training operation on the initial yolov7 detection model according to the damaged location loss function, the maintenance solution loss function and the component category loss function to obtain a trained maintenance solution prediction model specifically includes the following steps:

[0025] The confusion category training set is input into the initial yolov7 detection model, and the initial yolov7 detection model is subjected to model training operation according to the confusion category loss function to obtain the trained maintenance plan prediction model.

[0026] Furthermore, the confusion category loss function is expressed as:

[0027]

[0028] Among them, f gt represents the predicted probability of the true category of the prediction box, f pred Represents the predicted probability of the set of confused categories corresponding to a single category of the true category of the prediction box.

[0029] Furthermore, after the step of inputting the damaged image to be detected into the trained maintenance solution prediction model to perform a maintenance solution prediction operation and obtain a maintenance solution prediction result, the following steps are also included:

[0030] Performing rationality verification on the maintenance plan prediction result to obtain a maintenance plan verification result;

[0031] The maintenance plan prediction result is optimized and adjusted according to the maintenance plan verification result to obtain a target maintenance plan.

[0032] Furthermore, after the step of inputting the damaged image to be detected into the trained maintenance solution prediction model to perform a maintenance solution prediction operation and obtain a maintenance solution prediction result, the following steps are also included:

[0033] The maintenance plan prediction result is displayed in a visual form, wherein the visualization includes a schematic diagram of the damaged location and a description of the maintenance steps.

[0034] In order to solve the above technical problems, the embodiment of the present application further provides a vehicle maintenance plan prediction device, which adopts the following technical solution:

[0035] A historical data acquisition module, used to read a system database and acquire historical maintenance data from the system database, wherein the historical maintenance data includes historical damaged images, historical maintenance types, and historical maintenance plans;

[0036] A marking information acquisition module, used to receive the marking information corresponding to the historical damaged image sent by the marking terminal, wherein the marking information includes damaged location information and damaged component category;

[0037] A training set construction module, used to construct a model training set according to the historical maintenance data and the annotation information;

[0038] A loss label determination module is used to call the initial yolov7 detection model, and use the damaged location information, the historical maintenance plan and the damaged component category as the damaged location label, the maintenance plan label and the damaged component label when the initial yolov7 detection model calculates the loss, wherein the convolution layer of the CBS module in the backbone feature extraction network of the initial yolov7 detection model is a pyramid squeeze attention module;

[0039] A model training module, used for inputting the model training set into the initial yolov7 detection model, and performing a model training operation on the initial yolov7 detection model according to a damaged location loss function, a maintenance solution loss function and a component category loss function, to obtain a trained maintenance solution prediction model;

[0040] A request receiving module, used for receiving a repair plan prediction request carrying a damaged image to be detected, sent by a request terminal;

[0041] The solution prediction module is used to input the damaged image to be detected into the trained maintenance solution prediction model to perform a maintenance solution prediction operation to obtain a maintenance solution prediction result.

[0042] Furthermore, the device also includes:

[0043] A denoising module, used for denoising the historical damaged image according to a wavelet transform algorithm to obtain a denoised damaged image;

[0044] A contrast enhancement module, used for performing contrast enhancement processing on the denoised damaged image according to a histogram equalization algorithm to obtain an enhanced damaged image;

[0045] The edge detection module is used to perform edge detection processing on the enhanced damaged image according to the Laplace operator to obtain a pre-processed historical damaged image.

[0046] In order to solve the above technical problems, the embodiment of the present application further provides a computer device, which adopts the following technical solution:

[0047] The method comprises a memory and a processor, wherein the memory stores computer-readable instructions, and the processor implements the steps of the vehicle maintenance plan prediction method as described above when executing the computer-readable instructions.

[0048] In order to solve the above technical problems, the embodiment of the present application further provides a computer-readable storage medium, which adopts the following technical solution:

[0049] The computer-readable storage medium stores computer-readable instructions, and when the computer-readable instructions are executed by the processor, the steps of the vehicle maintenance plan prediction method described above are implemented.

[0050] The present application provides a vehicle maintenance plan prediction method, comprising: reading a system database, obtaining historical maintenance data from the system database, wherein the historical maintenance data includes historical damaged images, historical maintenance types, and historical maintenance plans; receiving annotation information corresponding to the historical damaged images sent by an annotation terminal, wherein the annotation information includes damaged location information and damaged component categories; building a model training set according to the historical maintenance data and the annotation information; calling an initial yo l ov7 detection model, and using the damaged location information, the historical maintenance plan, and the damaged component category as damaged location labels, maintenance plan labels, and damaged component labels when the initial yo l ov7 detection model calculates losses, respectively, wherein the convolution layer of the CBS module in the backbone feature extraction network of the initial yo l ov7 detection model is a pyramid squeeze attention module; inputting the model training set into the initial yo l ov7 detection model, and performing an on-screen analysis of the initial yo l ov7 detection model according to a damaged location loss function, a maintenance plan loss function, and a component category loss function. The ov7 detection model performs model training operations to obtain a trained maintenance solution prediction model; receives a maintenance solution prediction request carrying a damaged image to be detected from a request terminal; inputs the damaged image to be detected into the trained maintenance solution prediction model to perform a maintenance solution prediction operation to obtain a maintenance solution prediction result. Compared with the prior art, the present application optimizes the underlying convolutional layer structure of the network model backbone to more fully extract multi-scale feature map spatial information and realize cross-dimensional channel attention important feature interaction; according to the large number of body parts in the target task, and the close positions of the parts or the similar appearances of the parts are easy to cause mutual false detection, a new loss function is introduced by statistically analyzing the confusion category set of a single class, optimizing the false detection problem of body parts, and achieving more robust and reliable prediction of the damage assessment results of different parts of the whole case. BRIEF DESCRIPTION OF THE DRAWINGS

[0051] In order to more clearly illustrate the scheme in the present application, a brief introduction is given below to the drawings required for use in the description of the embodiments of the present application. Obviously, the drawings described below are some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0052] Figure 1 is an exemplary system architecture diagram to which the present application may be applied;

[0053] Figure 2 This is a flow chart of the implementation of the vehicle maintenance plan prediction method provided in Example 1 of the present application;

[0054] Figure 3 It is a structural schematic diagram of a vehicle maintenance plan prediction device provided in Example 2 of the present application;

[0055] Figure 4 It is a structural diagram of an embodiment of a computer device according to the present application. DETAILED DESCRIPTION

[0056] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as those commonly understood by technicians in the technical field of the present application; the terms used in the specification of the application herein are only for the purpose of describing specific embodiments and are not intended to limit the present application; the terms "including" and "having" and any variations thereof in the specification and claims of the present application and the above-mentioned drawings are intended to cover non-exclusive inclusions. The terms "first", "second", etc. in the specification and claims of the present application or the above-mentioned drawings are used to distinguish different objects, not to describe a specific order.

[0057] Reference to "embodiments" herein means that a particular feature, structure, or characteristic described in conjunction with the embodiments may be included in at least one embodiment of the present application. The appearance of the phrase in various locations in the specification does not necessarily refer to the same embodiment, nor is it an independent or alternative embodiment that is mutually exclusive with other embodiments. It is explicitly and implicitly understood by those skilled in the art that the embodiments described herein may be combined with other embodiments.

[0058] In order to enable those skilled in the art to better understand the solution of the present application, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the accompanying drawings.

[0059] like Figure 1 As shown, the system architecture 100 may include a terminal device 101, a network 102 and a server 103. The terminal device 101 may be a laptop 1011, a tablet computer 1012 or a mobile phone 1013. The network 102 is used to provide a medium for a communication link between the terminal device 101 and the server 103. The network 102 may include various connection types, such as wired, wireless communication links or optical fiber cables, etc.

[0060] The user can use the terminal device 101 to interact with the server 103 through the network 102 to receive or send messages, etc. Various communication client applications can be installed on the terminal device 101, such as web browser applications, shopping applications, search applications, instant messaging tools, email clients, social platform software, etc.

[0061] The terminal device 101 can be various electronic devices with a display screen and supporting web browsing. In addition to the laptop computer 1011, the tablet computer 1012 or the mobile phone 1013, the terminal device 101 can also be an e-book reader, an MP3 player (Moving Picture Experts Group Audio Layer III), an MP4 (Moving Picture Experts Group Audio Layer IV) player, a laptop computer, a desktop computer, etc.

[0062] The server 103 may be a server that provides various services, such as a background server that provides support for a web page displayed on the terminal device 101 .

[0063] It should be noted that the vehicle maintenance plan prediction method provided in the embodiment of the present application is generally executed by a server / terminal device, and accordingly, the vehicle maintenance plan prediction device is generally set in the server / terminal device.

[0064] It should be understood that Figure 1 The number of terminal devices, networks and servers in the embodiment is only for illustration. Any number of terminal devices, networks and servers may be provided according to implementation requirements.

[0065] Embodiment 1

[0066] Continue to refer Figure 2 , shows a flow chart of an embodiment of a vehicle maintenance plan prediction method according to the present application. The vehicle maintenance plan prediction method comprises: step S201, step S202, step S203, step S204, step S205, step S206 and step S207.

[0067] In step S201, a system database is read to obtain historical maintenance data from the system database, wherein the historical maintenance data includes historical damaged images, historical maintenance types, and historical maintenance plans.

[0068] In an embodiment of the present application, historical images of component repair plans are screened out from a system database of vehicle maintenance business scenarios, and component repair plan categories and labeling criteria are defined, including five types of component repair plans, namely, painting, small sheet metal, medium sheet metal, large sheet metal, and replacement, as well as repair plans at the damaged locations of corresponding damaged parts.

[0069] In step S202, the annotation information corresponding to the historical damaged image sent by the annotation terminal is received, wherein the annotation information includes the damaged position information and the damaged component category.

[0070] In an embodiment of the present application, according to the component repair plan form defined in step S201, historical pictures of the body component repair plan are manually annotated in batches, the component repair plan form appearing in each picture is annotated with a rectangular frame, and its component repair plan type, repair plan and body component name label are recorded.

[0071] In step S203, a model training set is constructed based on the historical maintenance data and annotation information.

[0072] In step S204, the initial yolov7 detection model is called, and the damaged location information, historical maintenance plan and damaged component category are used as the damaged location label, maintenance plan label and damaged component label when the initial yolov7 detection model calculates the loss, respectively. Among them, the convolution layer of the CBS module in the backbone feature extraction network of the initial yolov7 detection model is a pyramid squeeze attention module.

[0073] In an embodiment of the present application, the marked component repair plan location and repair plan are used as the location and category labels when the network calculates the loss, and the component category is used as a secondary label. The selected network model is used to learn and infer the damaged component location, repair plan and component category of the output image.

[0074] In an embodiment of the present application, yo l ov7 is selected as the basic network model, and the 3*3 convolution module of the backbone is replaced with the PSA module to fully extract the spatial information of the multi-scale feature map and realize the interaction of important features of cross-dimensional channel attention, effectively establish the long-term dependency relationship between multi-scale channel attention, improve the backbone module's multi-level feature acquisition of the vehicle, and assign different attention weights, so that the model pays attention to the vehicle parts with more serious parts maintenance solutions to improve the detection accuracy.

[0075] In step S205, the model training set is input into the initial yolov7 detection model, and the initial yolov7 detection model is trained according to the damaged location loss function, the maintenance plan loss function and the component category loss function to obtain a trained maintenance plan prediction model.

[0076] In step S206, a repair solution prediction request carrying a damaged image to be detected is received from a requesting terminal.

[0077] In step S207, the damaged image to be detected is input into the trained maintenance plan prediction model to perform a maintenance plan prediction operation to obtain a maintenance plan prediction result.

[0078] In an embodiment of the present application, a vehicle maintenance plan prediction method is provided, including: reading a system database, obtaining historical maintenance data in the system database, wherein the historical maintenance data includes historical damaged images, historical maintenance types, and historical maintenance plans; receiving annotation information corresponding to the historical damaged images sent by the annotation terminal, wherein the annotation information includes damaged location information and damaged component categories; constructing a model training set according to the historical maintenance data and the annotation information; calling an initial yolov7 detection model, and using the damaged location information, the historical maintenance plan, and the damaged component category as damaged location labels, maintenance plan labels, and damaged component labels when the initial yolov7 detection model calculates losses, respectively, wherein the convolution layer of the CBS module in the backbone feature extraction network of the initial yolov7 detection model is a pyramid squeezed attention module; inputting the model training set into the initial yolov7 detection model, and performing a model training operation on the initial yolov7 detection model according to a damaged location loss function, a maintenance plan loss function, and a component category loss function to obtain a trained maintenance plan prediction model; receiving a maintenance plan prediction request carrying a damaged image to be detected sent by a request terminal; inputting the damaged image to be detected into the trained maintenance plan prediction model to perform a maintenance plan prediction operation to obtain a maintenance plan prediction result. Compared with the prior art, the present application optimizes the underlying convolutional layer structure of the backbone of the network model to more fully extract the spatial information of the multi-scale feature map and realize the interaction of important features of cross-dimensional channel attention; since there are many body parts in the target task, and the parts are close in position or similar in appearance, it is easy to cause mutual false detection, and a new loss function is introduced by statistically analyzing the confusion category set of a single category to optimize the false detection problem of body parts and achieve more robust and reliable prediction of the damage assessment results of different parts of the whole case.

[0079] In some optional implementations of the embodiments of the present application, after step S201, the following steps are further included:

[0080] De-noising the historical damaged image according to the wavelet transform algorithm to obtain a de-noised damaged image;

[0081] Performing contrast enhancement processing on the denoised damaged image according to the histogram equalization algorithm to obtain an enhanced damaged image;

[0082] The enhanced damaged image is subjected to edge detection processing according to the Laplace operator to obtain a preprocessed historical damaged image.

[0083] In an embodiment of the present application, a batch of damaged vehicle images are acquired as a training data set based on a pre-established damaged vehicle image database.

[0084] In the embodiment of the present application, a wavelet transform algorithm is used to perform denoising processing on the acquired damaged vehicle image to remove high-frequency noise in the image and obtain a denoised damaged vehicle image.

[0085] In an embodiment of the present application, for the denoised damaged vehicle image, the image contrast is enhanced by a histogram equalization algorithm to improve the contrast and clarity of the image, thereby obtaining an enhanced damaged vehicle image.

[0086] In the embodiment of the present application, for the enhanced damaged vehicle image, the Laplace operator is used to perform edge detection on the image to highlight the edge contour of the damaged area of ​​the vehicle, thereby obtaining an edge-enhanced damaged vehicle image.

[0087] In some optional implementations of the embodiments of the present application, the model training set includes a confusion category training set, and the step S203 specifically includes the following steps:

[0088] Obtain the intersection-over-union ratio of the annotation boxes of the historical damaged image that carries the annotation information;

[0089] According to the intersection-and-union ratio of the annotation boxes, the intersection-and-union ratio overlap values ​​between the historical damaged images of different damaged component categories are calculated respectively;

[0090] The historical damaged images corresponding to the IoU overlap values ​​that meet the preset IoU overlap threshold form a confusion category training set, wherein the confusion category training set is annotated with confusion component categories;

[0091] The above step S204 specifically includes the following steps:

[0092] The confused part category is used as the confused label when the initial yolov7 detection model calculates the loss;

[0093] The above step S205 specifically includes the following steps:

[0094] The confusion category training set is input into the initial yolov7 detection model, and the initial yolov7 detection model is trained according to the confusion category loss function to obtain a trained maintenance plan prediction model.

[0095] In the embodiment of the present application, in addition to the location component maintenance plan function, classification loss function and confidence loss function of the Yo L OV7 model itself, the iou values ​​of the component maintenance plan component boxes labeled with different component categories in the training set are statistically analyzed, and potential mutually misdetected component categories with high iou overlap between different types of component labeling boxes are selected to form a set of confusion categories corresponding to a single category, and the set of confusion categories corresponding to a single category with all high iou of a single component category is recorded.

[0096] In the embodiment of the present application, the different types of markings with higher i ou include two situations:

[0097] (1) There is a door trim panel on the car door, and there may be a door trim strip component on the door trim panel. Both may be marked with component repair solutions at the same time. The two are located close to each other and may be mistakenly identified as one component during model detection.

[0098] (2) Parts with similar appearance may be misidentified with each other (this can be determined by analyzing the test set). Therefore, a single part category A may be a part pair with multiple parts of categories X, Y, and Z.

[0099] In an embodiment of the present application, a set of component categories with a high correlation in the target is analyzed according to target task statistics, and a penalty term is introduced into the loss function to optimize the problem of false detection of target components.

[0100] In some optional implementations of the embodiments of the present application, the above confusion category loss function is expressed as:

[0101]

[0102] Among them, f gt represents the predicted probability of the true category of the prediction box, f pred Represents the predicted probability of the set of confused categories corresponding to a single category of the true category of the prediction box.

[0103] In the embodiment of the present application, when calculating the loss function, the loss of all relevant potential mutually misdetected components of the target category is calculated based on the set of confusion categories corresponding to a single category of all components obtained through statistics and testing. where f gt is the predicted probability of the true category of the prediction box, f pred It is the predicted probability of the set of confused categories corresponding to a single category of the true category of the prediction box. Finally, the sum is added to the total loss function of the model for calculation.

[0104] In the embodiment of the present application, when the predicted probability a of the true category gt is equal to the predicted probability b of a certain category in the corresponding confusion category set, (ab)=1, then 1-(ab)=0, and the loss function value is 0.

[0105] In some optional implementations of the embodiments of the present application, after step S207, the following steps are further included:

[0106] Verify the rationality of the maintenance plan prediction results and obtain the maintenance plan verification results;

[0107] According to the maintenance plan verification results, the maintenance plan prediction results are optimized and adjusted to obtain the target maintenance plan.

[0108] In the embodiment of the present application, in order to ensure the feasibility and practicality of the prediction results, it is necessary to verify the rationality thereof. The rationality verification can use simulation technology to virtually verify the maintenance plan and evaluate the feasibility and effectiveness of the plan. Specifically:

[0109] 1. Virtual construction of maintenance plan:

[0110] (1) Data collection: First, it is necessary to collect detailed data of the maintenance object, including the technical parameters, structural drawings, maintenance history records, etc. These data are the basis for building a virtual model.

[0111] (2) Model building: Use 3D modeling software and other tools to build a virtual model of the maintenance object based on the collected data. The model should reflect the actual structure and performance of the equipment as accurately as possible.

[0112] (3) Scenario construction: Build a maintenance scenario in a virtual environment, including the location of the equipment, the layout of the surrounding environment, the operating space of the maintenance personnel, etc. Ensure that the scenario is consistent with the actual situation so that a realistic simulation test can be carried out.

[0113] 2. Virtual verification of maintenance solutions:

[0114] (1) Plan input: The maintenance plan to be verified is input into the simulation system, including maintenance steps, required tools, personnel configuration and other information.

[0115] (2) Simulation execution: Execute the maintenance plan in a virtual environment, simulating the operation process of the maintenance personnel. The system will judge the rationality and effectiveness of the maintenance plan based on the preset parameters and logic.

[0116] (3) Performance evaluation: Through simulation, the maintenance solution is evaluated in terms of:

[0117] ① Technical feasibility: Check whether the technical measures in the maintenance plan are feasible and whether there are any technical obstacles.

[0118] ② Economic rationality: Evaluate the cost-effectiveness of the maintenance plan, including maintenance costs, downtime losses, etc.

[0119] ③Safety compliance: Verify whether the maintenance plan complies with safety standards and regulatory requirements to ensure the safety of personnel and equipment during the maintenance process.

[0120] Time efficiency: Evaluate the execution time of the maintenance plan to ensure that the maintenance task is completed within the predetermined time window.

[0121] In the embodiment of the present application, the plan is optimized and adjusted according to the maintenance plan verification result to generate a final maintenance plan.

[0122] In the embodiment of the present application, the use of simulation technology to virtually verify the maintenance plan is a scientific and efficient method. It can help us discover problems and deficiencies in the maintenance plan in advance, and make targeted optimizations and adjustments, thereby ensuring the smooth progress of maintenance work and the normal operation of equipment.

[0123] In some optional implementations of the embodiments of the present application, after the above step S207, the following steps are further included:

[0124] The maintenance plan prediction results are displayed in a visual form, where the visualization includes a schematic diagram of the damaged location and instructions for the maintenance steps.

[0125] In the embodiment of the present application, the maintenance plan is presented to the maintenance personnel in a visual form, including etc., to assist the maintenance personnel in performing accurate and efficient maintenance operations, specifically:

[0126] 1. Schematic diagram of damaged location:

[0127] The damage location diagram is an indispensable part of the maintenance plan prediction results. It graphically displays the specific location of damage in the equipment or structure. This diagram usually contains the following information:

[0128] (1) Overall view of the equipment: First, provide an overall view of the equipment or structure to help the audience establish a holistic understanding of the maintenance object.

[0129] (2) Highlighting damaged areas: In the overall view, the damaged areas are highlighted using different colors, shadows, or borders to make them clear at a glance.

[0130] (3) Display of damage details: For areas with more serious damage, detailed drawings or cross-sectional drawings can be provided to show the specific form and extent of the damage and the possible scope of impact.

[0131] (4) Marking information: Mark the name, number, location coordinates and other information of the damaged part in the schematic diagram to facilitate accurate positioning of subsequent maintenance work.

[0132] 2. Repair steps:

[0133] The repair step description is another important visual element in the repair plan prediction results. It describes in detail the whole process from discovering the damage to completing the repair. In order to enhance readability and ease of understanding, the repair step description is usually visualized in the following form:

[0134] (1) Flowchart: Use a flowchart to show the logical sequence and process relationship of the maintenance steps. Each step is represented by a graphic block, and arrows are used to connect the steps to show the sequence and dependency relationship.

[0135] (2) Step Number and Title: Assign a unique number and a concise and clear title to each maintenance step so that the audience can quickly locate and understand the content of each step.

[0136] (3) Step description: Provide a detailed description of each step next to or below the flowchart, including the required tools, materials, staffing, key points of operation, etc. The description should be as concise and clear as possible, and avoid using too many professional terms.

[0137] (4) Graphics assistance: For some complex maintenance steps, graphics or animations can be used to assist in the explanation. Graphics can show key operations or equipment status changes during the maintenance process, while animations can simulate the dynamic effects of the maintenance process, allowing the audience to understand the maintenance steps more intuitively.

[0138] In the embodiment of the present application, the damaged location diagram and the repair step description are combined for comprehensive display, which can form a complete and intuitive visual report of the repair plan prediction results. This report can not only help technicians quickly understand the repair needs and steps, but also provide decision-making support for management to ensure the smooth progress of the repair work and the timely recovery of the equipment. In the comprehensive display, other visual elements such as timeline, progress bar, risk warning, etc. can also be considered to further enhance the information content and readability of the report.

[0139] In some optional implementations of the embodiments of the present application, the generated maintenance plan is added to the knowledge base to enrich the case data of the knowledge base and achieve continuous accumulation and optimization of knowledge.

[0140] The embodiments of the present application can acquire and process relevant data based on artificial intelligence technology. Artificial Intelligence (AI) is the theory, method, technology and application system that uses digital computers or machines controlled by digital computers to simulate, extend and expand human intelligence, perceive the environment, acquire knowledge and use knowledge to obtain the best results.

[0141] AI basic technologies generally include sensors, dedicated AI chips, cloud computing, distributed storage, big data processing technology, operation / interaction systems, mechatronics, etc. AI software technologies mainly include computer vision technology, robotics technology, biometrics technology, speech processing technology, natural language processing technology, and machine learning / deep learning.

[0142] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiments can be implemented by instructing the relevant hardware through computer-readable instructions, and the computer-readable instructions can be stored in a computer-readable storage medium. When the program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, the aforementioned storage medium can be a non-volatile storage medium such as a disk, an optical disk, a read-only memory (ROM), or a random access memory (RAM).

[0143] It should be understood that, although the steps in the flowchart of the accompanying drawings are displayed in sequence as indicated by the arrows, these steps are not necessarily executed in sequence in the order indicated by the arrows. Unless otherwise specified herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least a part of the steps in the flowchart of the accompanying drawings may include multiple sub-steps or multiple stages, and these sub-steps or stages are not necessarily executed at the same time, but can be executed at different times, and their execution order is not necessarily sequential, but can be executed in turn or alternately with other steps or at least a part of the sub-steps or stages of other steps.

[0144] Embodiment 2

[0145] Further references Figure 3 , as a response to the above Figure 2 The present application provides an embodiment of a vehicle maintenance plan prediction device, and the device embodiment is similar to Figure 2 Corresponding to the method embodiment shown, the device can be specifically applied to various electronic devices.

[0146] like Figure 3 As shown, the vehicle maintenance plan prediction device 200 of the embodiment of the present application includes:

[0147] The historical data acquisition module 210 is used to read the system database and acquire the historical maintenance data in the system database, wherein the historical maintenance data includes historical damaged images, historical maintenance types and historical maintenance plans;

[0148] The annotation information acquisition module 220 is used to receive the annotation information corresponding to the historical damaged image sent by the annotation terminal, wherein the annotation information includes the damaged position information and the damaged component category;

[0149] A training set construction module 230 is used to construct a model training set based on historical maintenance data and annotation information;

[0150] The loss label determination module 240 is used to call the initial yolov7 detection model, and use the damaged location information, the historical maintenance plan and the damaged component category as the damaged location label, the maintenance plan label and the damaged component label when the initial yolov7 detection model calculates the loss, wherein the convolution layer of the CBS module in the backbone feature extraction network of the initial yolov7 detection model is a pyramid squeeze attention module;

[0151] The model training module 250 is used to input the model training set into the initial yolov7 detection model, and perform a model training operation on the initial yolov7 detection model according to the damaged location loss function, the maintenance solution loss function and the component category loss function to obtain a trained maintenance solution prediction model;

[0152] The request receiving module 260 is used to receive a repair plan prediction request carrying a damaged image to be detected sent by a request terminal;

[0153] The solution prediction module 270 is used to input the damaged image to be detected into the trained maintenance solution prediction model to perform a maintenance solution prediction operation and obtain a maintenance solution prediction result.

[0154] In an embodiment of the present application, a vehicle maintenance plan prediction device 200 is provided, including: a historical data acquisition module 210, used to read a system database, and obtain historical maintenance data in the system database, wherein the historical maintenance data includes historical damaged images, historical maintenance types, and historical maintenance plans; an annotation information acquisition module 220, used to receive annotation information corresponding to historical damaged images sent by an annotation terminal, wherein the annotation information includes damaged location information and damaged component categories; a training set construction module 230, used to construct a model training set based on historical maintenance data and annotation information; a loss label determination module 240, used to call an initial yolov7 detection model, and use the damaged location information, historical maintenance plans, and damaged component categories as the initial yolov7 detection model for calculation. The damaged position label, maintenance plan label and damaged component label at the time of loss, wherein the convolution layer of the CBS module in the backbone feature extraction network of the initial yolov7 detection model is a pyramid squeezed attention module; the model training module 250 is used to input the model training set into the initial yolov7 detection model, and perform model training operations on the initial yolov7 detection model according to the damaged position loss function, the maintenance plan loss function and the component category loss function to obtain a trained maintenance plan prediction model; the request receiving module 260 is used to receive a maintenance plan prediction request carrying a damaged image to be detected sent by a requesting terminal; the plan prediction module 270 is used to input the damaged image to be detected into the trained maintenance plan prediction model to perform a maintenance plan prediction operation to obtain a maintenance plan prediction result. Compared with the prior art, the present application optimizes the underlying convolutional layer structure of the backbone of the network model to more fully extract the spatial information of the multi-scale feature map and realize the interaction of important features of cross-dimensional channel attention; since there are many body parts in the target task, and the parts are close in position or similar in appearance, it is easy to cause mutual false detection, and a new loss function is introduced by statistically analyzing the confusion category set of a single category to optimize the false detection problem of body parts and achieve more robust and reliable prediction of the damage assessment results of different parts of the whole case.

[0155] In some optional implementations of the embodiments of the present application, the vehicle maintenance plan prediction device 200 further includes:

[0156] A denoising module is used to perform denoising on the historical damaged image according to a wavelet transform algorithm to obtain a denoised damaged image;

[0157] A contrast enhancement module is used to perform contrast enhancement processing on the denoised damaged image according to a histogram equalization algorithm to obtain an enhanced damaged image;

[0158] The edge detection module is used to perform edge detection processing on the enhanced damaged image according to the Laplace operator to obtain the pre-processed historical damaged image.

[0159] To solve the above technical problems, the present application also provides a computer device. Figure 4 , Figure 4 This is a basic structural block diagram of a computer device according to an embodiment of the present application.

[0160] The computer device 300 includes a memory 310, a processor 320, and a network interface 330 that are interconnected and communicated through a system bus. It should be noted that the figure only shows a computer device 300 having components 310-330, but it should be understood that it is not required to implement all the components shown, and more or fewer components can be implemented instead. Among them, those skilled in the art can understand that the computer device here is a device that can automatically perform numerical calculations and / or information processing according to pre-set or stored instructions, and its hardware includes but is not limited to microprocessors, application specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), digital processors (DSPs), embedded devices, etc.

[0161] The computer device may be a computing device such as a desktop computer, a notebook, a PDA, a cloud server, etc. The computer device may interact with a user through a keyboard, a mouse, a remote controller, a touch pad, or a voice control device.

[0162] The memory 310 includes at least one type of readable storage medium, and the readable storage medium includes flash memory, hard disk, multimedia card, card-type memory (for example, SD or DX memory, etc.), random access memory (RAM), static random access memory (SRAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), programmable read-only memory (PROM), magnetic memory, magnetic disk, optical disk, etc. In some embodiments, the memory 310 can be an internal storage unit of the computer device 300, such as a hard disk or memory of the computer device 300. In other embodiments, the memory 310 can also be an external storage device of the computer device 300, such as a plug-in hard disk equipped on the computer device 300, a smart memory card (Smart Media Card, SMC), a secure digital (Secure Digital, SD) card, a flash card, etc. Of course, the memory 310 can also include both the internal storage unit of the computer device 300 and its external storage device. In the embodiment of the present application, the memory 310 is generally used to store the operating system and various application software installed on the computer device 300, such as computer-readable instructions of the vehicle maintenance plan prediction method, etc. In addition, the memory 310 can also be used to temporarily store various data that have been output or are to be output.

[0163] The processor 320 may be a central processing unit (CPU), a controller, a microcontroller, a microprocessor, or other data processing chip in some embodiments. The processor 320 is generally used to control the overall operation of the computer device 300. In the embodiment of the present application, the processor 320 is used to run the computer-readable instructions or process data stored in the memory 310, such as computer-readable instructions for running the vehicle maintenance plan prediction method.

[0164] The network interface 330 may include a wireless network interface or a wired network interface. The network interface 330 is generally used to establish a communication connection between the computer device 300 and other electronic devices.

[0165] The computer device provided in the present application optimizes the underlying convolutional layer structure of the backbone of the network model to more fully extract the spatial information of the multi-scale feature map and realize the interaction of important features of cross-dimensional channel attention; according to the large number of body parts in the target task, and the close positions of the parts or the similar appearance of the parts are easy to cause mutual false detection, a new loss function is introduced by statistically analyzing the confusion category set of a single category, so as to optimize the false detection problem of body parts and realize more robust and reliable prediction of the damage assessment results of different parts of the whole case.

[0166] The present application also provides another embodiment, namely, providing a computer-readable storage medium, wherein the computer-readable storage medium stores computer-readable instructions, and the computer-readable instructions can be executed by at least one processor to enable the at least one processor to perform the steps of the vehicle maintenance plan prediction method as described above.

[0167] The computer-readable storage medium provided in the present application optimizes the underlying convolutional layer structure of the backbone of the network model to more fully extract the spatial information of the multi-scale feature map and realize the interaction of important features of cross-dimensional channel attention; according to the large number of body parts in the target task, and the close positions of the parts or the similar appearances of the parts are easy to cause mutual false detection, a new loss function is introduced by statistically analyzing the confusion category set of a single category, so as to optimize the false detection problem of body parts and realize more robust and reliable prediction of the damage assessment results of different parts of the whole case.

[0168] Through the description of the above implementation methods, those skilled in the art can clearly understand that the above-mentioned embodiment methods can be implemented by means of software plus a necessary general hardware platform, and of course by hardware, but in many cases the former is a better implementation method. Based on such an understanding, the technical solution of the present application, or the part that contributes to the prior art, can be embodied in the form of a software product, which is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk), and includes a number of instructions for a terminal device (which can be a mobile phone, computer, server, air conditioner, or network device, etc.) to execute the methods described in each embodiment of the present application.

[0169] Obviously, the embodiments described above are only some embodiments of the present application, rather than all embodiments. The preferred embodiments of the present application are given in the accompanying drawings, but they do not limit the patent scope of the present application. The present application can be implemented in many different forms. On the contrary, the purpose of providing these embodiments is to make the understanding of the disclosure of the present application more thorough and comprehensive. Although the present application is described in detail with reference to the aforementioned embodiments, for those skilled in the art, it is still possible to modify the technical solutions recorded in the aforementioned specific implementation methods, or to perform equivalent replacement of some of the technical features therein. Any equivalent structure made using the contents of the specification and drawings of this application, directly or indirectly used in other related technical fields, is similarly within the scope of patent protection of this application.

Claims

1. A vehicle maintenance plan prediction method, characterized in that: The steps include: Reading a system database, and obtaining historical maintenance data in the system database, wherein the historical maintenance data includes historical damaged images, historical maintenance types, and historical maintenance plans; Receiving annotation information corresponding to the historical damaged image sent by the annotation terminal, wherein the annotation information includes damaged location information and damaged component category; Constructing a model training set according to the historical maintenance data and the annotation information; Calling the initial yolov7 detection model, and using the damaged location information, the historical maintenance plan, and the damaged component category as the damaged location label, the maintenance plan label, and the damaged component label when the initial yolov7 detection model calculates the loss, wherein the convolution layer of the CBS module in the backbone feature extraction network of the initial yolov7 detection model is a pyramid squeeze attention module; Input the model training set into the initial yolov7 detection model, and perform model training operation on the initial yolov7 detection model according to the damaged position loss function, the maintenance plan loss function and the component category loss function to obtain a trained maintenance plan prediction model; receiving a repair plan prediction request including a damaged image to be detected, sent by a requesting terminal; The damaged image to be detected is input into the trained maintenance plan prediction model to perform a maintenance plan prediction operation to obtain a maintenance plan prediction result.

2. The vehicle maintenance plan prediction method according to claim 1, characterized in that: After the step of reading the system database and obtaining the historical maintenance data in the system database, the following steps are also included: Performing denoising on the historical damaged image according to a wavelet transform algorithm to obtain a denoised damaged image; Performing contrast enhancement processing on the denoised damaged image according to a histogram equalization algorithm to obtain an enhanced damaged image; The enhanced damaged image is subjected to edge detection processing according to the Laplace operator to obtain a pre-processed historical damaged image.

3. The vehicle maintenance plan prediction method according to claim 1, characterized in that: The model training set includes a confusion category training set, and the step of constructing the model training set according to the historical maintenance data and the annotation information specifically includes the following steps: Obtaining an intersection-over-union ratio of a labeling box of a historical damaged image carrying the labeling information; According to the annotation box intersection-and-union ratio, respectively calculate the intersection-and-union ratio overlap values ​​between the historical damaged images of different damaged component categories; The historical damaged images corresponding to the IoU overlap values ​​that meet the preset IoU overlap threshold form the confusion category training set, wherein the confusion category training set is annotated with the confusion component category; The step of calling the initial yolov7 detection model and using the damaged location information, the historical maintenance plan and the damaged component category as the damaged location label, the maintenance plan label and the damaged component label when the initial yolov7 detection model calculates the loss specifically includes the following steps: Using the confusing component category as the confusing label when the initial yolov7 detection model calculates the loss; The step of inputting the model training set into the initial yolov7 detection model, and performing a model training operation on the initial yolov7 detection model according to the damaged location loss function, the maintenance solution loss function and the component category loss function to obtain a trained maintenance solution prediction model specifically includes the following steps: The confusion category training set is input into the initial yolov7 detection model, and the initial yolov7 detection model is subjected to model training operation according to the confusion category loss function to obtain the trained maintenance plan prediction model.

4. The vehicle maintenance plan prediction method according to claim 3, characterized in that: The confusion category loss function is expressed as: Among them, f gt represents the predicted probability of the true category of the prediction box, f pred Represents the predicted probability of the set of confused categories corresponding to a single category of the true category of the prediction box.

5. The vehicle maintenance plan prediction method according to claim 1, characterized in that: After the step of inputting the damaged image to be detected into the trained maintenance solution prediction model to perform maintenance solution prediction operation and obtain the maintenance solution prediction result, the following steps are also included: Performing rationality verification on the maintenance plan prediction result to obtain a maintenance plan verification result; The maintenance plan prediction result is optimized and adjusted according to the maintenance plan verification result to obtain a target maintenance plan.

6. The vehicle maintenance plan prediction method according to claim 1, characterized in that: After the step of inputting the damaged image to be detected into the trained maintenance solution prediction model to perform maintenance solution prediction operation and obtain the maintenance solution prediction result, the following steps are also included: The maintenance plan prediction result is displayed in a visual form, wherein the visualization includes a schematic diagram of the damaged location and a description of the maintenance steps.

7. A vehicle maintenance plan prediction device, characterized in that: include: A historical data acquisition module, used to read a system database and acquire historical maintenance data from the system database, wherein the historical maintenance data includes historical damaged images, historical maintenance types, and historical maintenance plans; A marking information acquisition module, used to receive the marking information corresponding to the historical damaged image sent by the marking terminal, wherein the marking information includes damaged location information and damaged component category; A training set construction module, used to construct a model training set according to the historical maintenance data and the annotation information; A loss label determination module is used to call the initial yolov7 detection model, and use the damaged location information, the historical maintenance plan and the damaged component category as the damaged location label, the maintenance plan label and the damaged component label when the initial yolov7 detection model calculates the loss, wherein the convolution layer of the CBS module in the backbone feature extraction network of the initial yolov7 detection model is a pyramid squeeze attention module; A model training module, used for inputting the model training set into the initial yolov7 detection model, and performing a model training operation on the initial yolov7 detection model according to a damaged location loss function, a maintenance solution loss function and a component category loss function, to obtain a trained maintenance solution prediction model; A request receiving module, used for receiving a repair plan prediction request carrying a damaged image to be detected, sent by a request terminal; The solution prediction module is used to input the damaged image to be detected into the trained maintenance solution prediction model to perform a maintenance solution prediction operation to obtain a maintenance solution prediction result.

8. The vehicle maintenance plan prediction device according to claim 7, characterized in that: The device also includes: A denoising module, used for denoising the historical damaged image according to a wavelet transform algorithm to obtain a denoised damaged image; A contrast enhancement module, used for performing contrast enhancement processing on the denoised damaged image according to a histogram equalization algorithm to obtain an enhanced damaged image; The edge detection module is used to perform edge detection processing on the enhanced damaged image according to the Laplace operator to obtain a pre-processed historical damaged image.

9. A computer device comprising a memory and a processor, characterized in that: The memory stores computer-readable instructions, and when the processor executes the computer-readable instructions, the steps of the vehicle maintenance plan prediction method according to any one of claims 1 to 6 are implemented.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer-readable instructions, and when the computer-readable instructions are executed by a processor, the steps of the vehicle maintenance plan prediction method according to any one of claims 1 to 6 are implemented.

Images