Interactive vehicle loss assessment method and device based on multi-model fusion and storage medium
By using multi-model fusion technology in vehicle loss determination system to process the characteristics of different types and dimensions in multimedia data, the problem of low loss accuracy in individual depth models in special scenarios is solved, and higher loss accuracy is achieved Sex and user experience.
Patent Information
- Application Number
- CN202411464775.4
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-10-18
- Publication Date
- 2025-02-18
AI Technical Summary
In the prior art, relying solely on the depth model cannot provide accurate vehicle damage measurement results in special scenarios (such as rain and snow, overexposure, occlusion, etc.), resulting in low accuracy of damage measurement.
Using an interactive vehicle loss determination method based on multi-model fusion, the vehicle loss determination judgment model including component maintenance machine learning model, damage morphology maintenance machine learning model and separate maintenance machine learning model , comprehensively process the characteristics of different types and dimensions to improve loss accuracy.
Through the multi-model fusion method, the accuracy and fault tolerance of vehicle damage determination are improved, the demand for manual intervention is reduced, and the user experience is improved.
Smart Images
Figure CN119477552A_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the fields of image processing and financial technology, and in particular to an interactive vehicle damage assessment method, device, computer equipment and computer-readable storage medium based on multi-model fusion. Background Art
[0002] Nowadays, with the development of computer vision and artificial intelligence technology, automatic damage assessment technology based on images and videos has become increasingly mature. The industry is looking forward to the ultimate realization of fully automated vehicle damage assessment. However, it is well known that artificial intelligence technology cannot achieve 100% accuracy. Algorithms will have poor performance in many special scenarios, such as rainy and snowy weather, overexposure, occlusion, etc. This is even more true for vehicle damage assessment scenarios. Some simple composite damages often cause relatively large interference to the model, such as the superposition of scratches and dents.
[0003] In order to solve the problem that deep models alone cannot give accurate results, the current solutions in the industry are mainly to divert these data through indicators such as confidence, damage area, and damage location of a single model, that is, the high-confidence and obvious damage assessment results generated by the system will be directly output, while low-confidence and unclear results need to be transferred to the traditional manual damage assessment method to generate results. This method can solve the above problems to a certain extent, but there is still a certain degree of inaccuracy. Therefore, how to improve the accuracy of vehicle damage assessment is a technical problem that needs to be solved urgently. Summary of the invention
[0004] The main purpose of this application is to provide an interactive vehicle damage assessment method, device, computer equipment and computer-readable storage medium based on multi-model fusion, aiming to avoid the technical problem of low accuracy of the output results of a single automatic damage assessment model, so as to improve the accuracy of vehicle damage assessment results.
[0005] To achieve the above objectives, the present application provides an interactive vehicle damage assessment method based on multi-model fusion, and the interactive vehicle damage assessment method based on multi-model fusion comprises the following steps:
[0006] Receiving multimedia data uploaded by a user through a terminal device, wherein the multimedia data is collected by the user through the terminal device and is used to assess damage to the accident vehicle when a traffic accident occurs;
[0007] Input the multimedia data into a trained vehicle damage assessment model, wherein the vehicle damage assessment model includes a component repair machine learning model, a damage morphology repair machine learning model, and a separate repair machine learning model. The component repair machine learning model is used to determine a repair plan according to the degree of damage to the components of the accident vehicle, the damage morphology repair machine learning model is used to determine a repair plan according to the damage morphology of the accident vehicle, and the separate repair machine learning model is used to directly determine a repair plan;
[0008] The accident vehicle damage is assessed based on the repair plan output by the component repair machine learning model, the damage morphology repair machine learning model and the individual repair machine learning model.
[0009] In addition, to achieve the above-mentioned purpose, the present application also provides an interactive vehicle damage assessment device based on multi-model fusion, and the interactive vehicle damage assessment device based on multi-model fusion includes:
[0010] A data acquisition unit, used to receive multimedia data uploaded by a user through a terminal device, wherein the multimedia data is collected by the user through the terminal device and is used to assess the damage of the accident vehicle when a traffic accident occurs;
[0011] a model operation unit, configured to input the multimedia data into a trained vehicle damage assessment model, wherein the vehicle damage assessment model includes a component repair machine learning model, a damage morphology repair machine learning model, and a separate repair machine learning model, wherein the component repair machine learning model is configured to determine a repair plan according to the degree of damage to the components of the accident vehicle, the damage morphology repair machine learning model is configured to determine a repair plan according to the damage morphology of the accident vehicle, and the separate repair machine learning model is configured to directly determine a repair plan;
[0012] A vehicle damage assessment unit is used to assess the damage of an accident vehicle based on the repair plan output by the component repair machine learning model, the damage morphology repair machine learning model and the individual repair machine learning model.
[0013] In addition, to achieve the above-mentioned purpose, the present application also provides a computer device, which includes a processor, a memory, and an interactive vehicle damage assessment program based on multi-model fusion stored on the memory and executable by the processor, wherein when the interactive vehicle damage assessment program based on multi-model fusion is executed by the processor, the steps of the interactive vehicle damage assessment method based on multi-model fusion as described above are implemented.
[0014] In addition, to achieve the above-mentioned purpose, the present application also provides a computer-readable storage medium, on which an interactive vehicle damage assessment program based on multi-model fusion is stored. When the interactive vehicle damage assessment program based on multi-model fusion is executed by a processor, the steps of the interactive vehicle damage assessment method based on multi-model fusion as described above are implemented.
[0015] The present application provides an interactive vehicle damage assessment method, apparatus, computer equipment and computer-readable storage medium based on multi-model fusion, which receives multimedia data uploaded by a user through a terminal device, the multimedia data is collected by the user through the terminal device, and the multimedia data is used to assess the damage of the accident vehicle when a traffic accident occurs; then the multimedia data is input into a trained vehicle damage assessment judgment model, the vehicle damage assessment judgment model includes a component repair machine learning model, a damage morphology repair machine learning model and a separate repair machine learning model, the component repair machine learning model is used to determine the repair plan according to the degree of damage to the components of the accident vehicle, the damage morphology repair machine learning model is used to determine the repair plan according to the damage morphology of the accident vehicle, and the separate repair machine learning model is used to directly determine the repair plan; according to the component repair machine learning model, the repair plan is determined according to the damage morphology of the accident vehicle. The machine learning model, the damage morphology repair machine learning model and the separate repair machine learning model output the repair plan to assess the damage of the accident vehicle; the input data of a single model is single and dedicated to the model, so the features extracted are limited. In the embodiment of the present application, the input data of the model is multimedia data, including different types of data, so features of different types and dimensions can be extracted, and the input is richer; secondly, the vehicle damage assessment model in the embodiment of the present application includes three models: a component repair machine learning model, a damage morphology repair machine learning model and a separate repair machine learning model, which can be used to process the features of different types and dimensions, and then assess the damage of the accident vehicle according to the repair plan output by the three models. Compared with the damage assessment method of a single model, it has higher fault tolerance and higher accuracy. BRIEF DESCRIPTION OF THE DRAWINGS
[0016] Figure 1 This is a flowchart of an interactive vehicle damage assessment method based on multi-model fusion in this application;
[0017] Figure 2 The embodiment of the present application provides a functional module diagram of an interactive vehicle damage assessment device based on multi-model fusion;
[0018] Figure 3 It is a schematic block diagram of the structure of a computer device provided in an embodiment of the present application.
[0019] The realization of the purpose, functional features and advantages of this application will be further explained in conjunction with embodiments and with reference to the accompanying drawings. DETAILED DESCRIPTION
[0020] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.
[0021] The flowcharts shown in the accompanying drawings are only examples and do not necessarily include all the contents and operations / steps, nor must they be executed in the order described. For example, some operations / steps may also be decomposed, combined or partially merged, so the actual execution order may change according to actual conditions.
[0022] It should be understood that the terms used in this application specification are only for the purpose of describing specific embodiments and are not intended to limit the application. As used in this application specification and the appended claims, unless the context clearly indicates otherwise, the singular forms "a", "an" and "the" are intended to include plural forms.
[0023] It should be further understood that the term “and / or” used in the specification and appended claims refers to any combination and all possible combinations of one or more of the associated listed items, and includes these combinations.
[0024] The interactive vehicle damage assessment method based on multi-model fusion involved in the embodiment of the present application is mainly applied to computer equipment. The interactive vehicle damage assessment device based on multi-model fusion can be a PC, a portable computer, a mobile terminal, or other device with display and processing functions.
[0025] In conjunction with the accompanying drawings, some embodiments of the present application are described in detail below. In the absence of conflict, the following embodiments and features in the embodiments can be combined with each other.
[0026] Reference Figure 1 , Figure 1 This is a flow chart of the interactive vehicle damage assessment method based on multi-model fusion in this application.
[0027] like Figure 1 As shown, an embodiment of the present application provides an interactive vehicle damage assessment method based on multi-model fusion, and the interactive vehicle damage assessment method based on multi-model fusion includes steps S10 to S30.
[0028] In this embodiment, the traffic switching method based on the gateway platform includes the following steps:
[0029] Step S10, receiving multimedia data uploaded by the user through the terminal device, wherein the multimedia data is collected by the user through the terminal device, and the multimedia data is used to assess the damage of the accident vehicle when a traffic accident occurs.
[0030] Multimedia data refers to a data set that contains two or more different types of information, which usually include text, images, audio, video, animation, etc. Since multimedia data contains multiple media forms, each of which has its own unique properties and processing methods, this leads to the complexity of multimedia data processing.
[0031] The current damage assessment model uses the confidence, damage area, damage location and other indicators of a single model to divert and process these data. Therefore, its input is mainly pictures or videos, which are taken and uploaded by users through their terminal devices. There may be the following problems: First, the user is not a professional claims adjuster, and the pictures or videos taken may not meet the specifications; second, the terminal devices held by different users are not the same, and there may even be extreme cases where there is no terminal device. Therefore, even if the shooting method is correct, there may still be pictures or videos that do not meet the requirements. For example, the terminal device is a mobile phone, tablet or driving recorder, and the pixels of old driving recorders or mobile phones are not high, resulting in insufficient clarity. Third, traffic accidents are often accompanied by extreme weather, so the pictures or videos taken may not meet the requirements. The above problems will cause the pictures or videos taken to still not meet the requirements.
[0032] The embodiment of the present application uses multimedia data, which may include the user's text or voice description of the on-site situation, pictures or videos taken by the user, collision audio recorded by a driving recorder, etc., and sound wave data recorded by the vehicle terminal radar. Therefore, even if a single data is not good, it will not affect the comprehensive judgment of the entire model.
[0033] Step S20, input the multimedia data into a trained vehicle damage assessment model, the vehicle damage assessment model including a component repair machine learning model, a damage morphology repair machine learning model and a separate repair machine learning model, the component repair machine learning model is used to determine a repair plan according to the degree of damage to the components of the accident vehicle, the damage morphology repair machine learning model is used to determine a repair plan according to the damage morphology of the accident vehicle, and the separate repair machine learning model is used to directly determine the repair plan.
[0034] Among them, the vehicle damage assessment model is usually a tool or program used by insurance companies, automobile repair organizations or third-party assessment organizations to assess the degree of damage to a vehicle after an accident and determine the cost of repair. The vehicle damage assessment model described in the embodiment of the present application is a method that uses artificial intelligence technology to identify vehicle damage characteristics. Different from the traditional vehicle damage assessment method, which relies on experienced appraisers, they will estimate the repair cost based on photos of the vehicle damage, on-site inspections, and damage assessment standards within the industry.
[0035] The vehicle damage assessment model integrates three machine learning models: component repair machine learning model, damage morphology repair machine learning model and separate repair machine learning model. The component repair machine learning model uses computer vision technology to extract the features of the damaged parts from the image, such as the location, size, shape, etc. of the damage, and then trains based on the features to identify different types of damage and recommend the most appropriate repair plan. The damage morphology repair machine learning model collects vehicle images containing various damage forms and their corresponding repair records. When used, it matches the multimedia data uploaded by the accident vehicle in the database to find the most suitable repair record and output the repair plan. The separate repair machine learning model directly judges the repair plan based on the uploaded multimedia data. Compared with the above two machine learning models, it does not have special requirements for data. It only judges based on the multimedia data uploaded by the user, and does not subdivide the data type characteristics. It is a general large model.
[0036] Step S30: assess the damage of the accident vehicle according to the repair plan output by the component repair machine learning model, the damage morphology repair machine learning model and the individual repair machine learning model.
[0037] After the maintenance plan is determined, the labor costs and parts costs can be determined based on the maintenance plan to achieve damage assessment of the accident vehicle. This process can be achieved manually or with the built-in functions of the model. Of course, the valuation can also be dynamically adjusted based on the latest market conditions, changes in parts prices and other factors.
[0038] In an embodiment of the present application, multimedia data uploaded by a user through a terminal device is received, and the multimedia data is collected by the user through the terminal device, and the multimedia data is used to assess the damage of the accident vehicle when a traffic accident occurs; then the multimedia data is input into a trained vehicle damage assessment model, and the vehicle damage assessment model includes a component repair machine learning model, a damage morphology repair machine learning model, and a separate repair machine learning model. The component repair machine learning model is used to determine a repair plan according to the degree of damage to the components of the accident vehicle, the damage morphology repair machine learning model is used to determine a repair plan according to the damage morphology of the accident vehicle, and the separate repair machine learning model is used to directly determine the repair plan; according to the component repair machine learning model, the damage morphology repair machine learning model The damage of the accident vehicle is assessed based on the repair plan output by the type and separate repair machine learning model; the input data of a single model is single and dedicated to the model, so the features extracted are limited. In the embodiment of the present application, the input data of the model is multimedia data, including different types of data, so features of different types and dimensions can be extracted, and the input is richer; secondly, the vehicle damage assessment model in the embodiment of the present application includes three models: a component repair machine learning model, a damage morphology repair machine learning model, and a separate repair machine learning model, which can be used to process the features of different types and dimensions, and then assess the damage of the accident vehicle based on the repair plan output by the three models. Compared with the damage assessment method of a single model, it has higher fault tolerance and higher accuracy.
[0039] Specifically, the damage assessment of the accident vehicle according to the maintenance plan output by the component maintenance machine learning model, the damage morphology maintenance machine learning model and the separate maintenance machine learning model includes:
[0040] If the repair plans determined by the component repair machine learning model, the damage morphology repair machine learning model and the separate repair machine learning model are the same, then the output result of the vehicle damage assessment model is judged to be completely credible, and the accident vehicle damage is assessed based on the repair plans determined by the component repair machine learning model, the damage morphology repair machine learning model and the separate repair machine learning model;
[0041] If the repair plans determined by two of the three models, namely, the component repair machine learning model, the damage morphology repair machine learning model and the separate repair machine learning model, are the same, then it is determined that the output result of the vehicle damage assessment model has a high credibility, and a weak manual intervention method is adopted to assess the damage of the accident vehicle;
[0042] If the repair plans determined by the component repair machine learning model, the damage morphology repair machine learning model and the individual repair machine learning model are all different, then the output result of the vehicle damage assessment model is judged to be completely unreliable, and strong manual intervention is required to conduct damage assessment of the accident vehicle.
[0043] Among them, the maintenance plan determined by the machine learning model of component maintenance is used in R A It means that the repair plan determined by the damage morphology repair machine learning model is represented by R B Indicates that the maintenance plan determined by the individual maintenance machine learning model is expressed in R C If R A =R B =R C , the result is considered completely reliable. Using this maintenance plan as the model output result, the task is automatically passed without human intervention; if R A , R B , R C If there are two identical results, then according to the majority theory, the result is considered to be more reliable and weak human intervention is sufficient. A , R B , R C If they are all different, the results are considered completely unreliable and strong human intervention is required.
[0044] The workflow of the automatic damage assessment diversion solution in the current prior art is usually: the user accesses the system, the damage is automatically assessed, and after failure, it is transferred to manual damage assessment. This process will, to a certain extent, make customers think that the automatic damage assessment process is redundant and useless, and then develop annoyance and resistance to the entire product. In the embodiment of the present application, strong manual intervention will only be taken when the result is completely unfeasible, and traditional manual damage assessment will be performed. When the feasibility of the result is high, excessive manual intervention is reduced, and weak manual damage assessment is performed. This also improves the existing technology in terms of user experience.
[0045] Specifically, the method of assessing the damage of the accident vehicle by adopting a weak manual intervention method includes:
[0046] Sending interactive questions to the user through the terminal device, wherein the interactive questions are used to inquire about the damage conditions of the damaged parts of the accident vehicle;
[0047] receiving, through the terminal device, a user's answer to the interactive question;
[0048] If the answer is yes, then the damage is assessed based on the damage to the damaged parts of the accident vehicle.
[0049] Furthermore, the interactive vehicle damage assessment method based on multi-model fusion also includes:
[0050] If the answer is no, instructing the user to retake pictures of the damaged parts of the accident vehicle through the terminal device;
[0051] The damage is assessed based on the re-photographed pictures of the damaged parts of the accident vehicle.
[0052] Specifically, the damage condition includes at least one of the following: cracking, wrinkling, missing, denting, and scratching, and the repair plan includes at least one of the following: replacement, sheet metal, and painting;
[0053] When the repair solution determined by two of the three models, namely, the component repair machine learning model, the damage morphology repair machine learning model and the separate repair machine learning model, is replacement, the interactive question is used to inquire whether the damaged part of the accident vehicle is cracked, wrinkled or missing;
[0054] When the repair solution determined by two of the three models, namely, the component repair machine learning model, the damage morphology repair machine learning model and the separate repair machine learning model, is sheet metal, the interactive question is used to inquire whether the damaged part of the accident vehicle has a dent;
[0055] When the repair solution determined by two of the three models, namely, the component repair machine learning model, the damage morphology repair machine learning model and the individual repair machine learning model, is painting, the interactive question is used to inquire whether the damaged part of the accident vehicle is only scratches.
[0056] In the embodiment of the present application, under weak manual intervention, the quality and recognition of the multimedia data uploaded by the customer are acceptable. The result may be uncertain due to the angle or slight blur of the picture. At this time, the uploaded multimedia data can still be used, so the user only needs to answer a small number of questions. Only when the user's answer is inconsistent with the result judged based on the multimedia data, the user will be guided to re-take pictures of the damaged parts of the accident vehicle to further confirm the result. Compared with traditional manual damage assessment, strong manual intervention without using this part of the multimedia data can reduce the discomfort caused to users by detailed inquiries.
[0057] Specifically, the strong manual intervention method for assessing the damage of the accident vehicle includes:
[0058] Sending a location acquisition request to the terminal device;
[0059] Receiving the accident location sent by the terminal device, where the accident location is edited by the user in response to the location acquisition request or obtained by opening the terminal device positioning system;
[0060] Matching a claims adjuster terminal that is idle at the current moment nearby according to the accident location;
[0061] Notify the claims adjuster corresponding to the claims adjuster terminal to go to the accident site to conduct on-site manual damage assessment.
[0062] In an embodiment of the present application, under a strong manual intervention mode, the location acquisition method can be obtained by the user actively opening the terminal device positioning system, which is faster and more accurate, and is conducive to quickly performing manual damage assessment for the user, and is more efficient; if the user pays attention to privacy, he can also edit and send it independently, and can also quickly locate and perform manual damage assessment.
[0063] Among them, the vehicle damage assessment model includes a shared encoder and a component repair decoder, a damage form repair decoder and a separate repair decoder; the shared encoder is used to perform feature fusion encoding on the multimedia data to obtain encoded data; the component repair decoder is used to decode the encoded data to obtain input for the component repair machine learning model; the damage form repair decoder is used to decode the encoded data to obtain input for the damage form repair machine learning model; the separate repair decoder is used to decode the encoded data to obtain input for the separate repair machine learning model.
[0064] As mentioned above, multimedia data has many dimensions and is complex. In this application, by sharing the same shared editor, it is possible to save the tedious process of separate encoding and improve efficiency, and to integrate multiple different types of features of different dimensions to increase the accuracy of each model. When used for different models, they are then decoded separately to obtain input data suitable for different models, which is beneficial for each model to obtain appropriate input data for accurate recognition.
[0065] See also Figure 2 , Figure 2 The present application provides a schematic diagram of the functional modules of an interactive vehicle damage assessment device based on multi-model fusion.
[0066] like Figure 2 As shown, the interactive vehicle damage assessment device 200 based on multi-model fusion includes:
[0067] The data acquisition unit 10 is used to receive multimedia data uploaded by the user through the terminal device, wherein the multimedia data is collected by the user through the terminal device and is used to assess the damage of the accident vehicle when a traffic accident occurs;
[0068] The model operation unit 20 is used to input the multimedia data into a trained vehicle damage assessment model, wherein the vehicle damage assessment model includes a component repair machine learning model, a damage morphology repair machine learning model and a separate repair machine learning model, wherein the component repair machine learning model is used to determine a repair plan according to the degree of damage to the components of the accident vehicle, the damage morphology repair machine learning model is used to determine a repair plan according to the damage morphology of the accident vehicle, and the separate repair machine learning model is used to directly determine a repair plan;
[0069] The vehicle damage assessment unit 30 is used to assess the damage of the accident vehicle based on the repair plan output by the component repair machine learning model, the damage morphology repair machine learning model and the individual repair machine learning model.
[0070] Specifically, in the aspect of assessing the damage of the accident vehicle according to the repair plan output by the component repair machine learning model, the damage morphology repair machine learning model and the separate repair machine learning model, the vehicle damage assessment unit 30 is specifically used to:
[0071] If the repair plans determined by the component repair machine learning model, the damage morphology repair machine learning model and the separate repair machine learning model are the same, then the output result of the vehicle damage assessment model is judged to be completely credible, and the accident vehicle damage is assessed based on the repair plans determined by the component repair machine learning model, the damage morphology repair machine learning model and the separate repair machine learning model;
[0072] If the repair plans determined by two of the three models, namely, the component repair machine learning model, the damage morphology repair machine learning model and the separate repair machine learning model, are the same, then it is determined that the output result of the vehicle damage assessment model has a high credibility, and a weak manual intervention method is adopted to assess the damage of the accident vehicle;
[0073] If the repair plans determined by the component repair machine learning model, the damage morphology repair machine learning model and the individual repair machine learning model are all different, then the output result of the vehicle damage assessment model is judged to be completely unreliable, and strong manual intervention is required to conduct damage assessment of the accident vehicle.
[0074] Specifically, in the aspect of assessing the damage of the accident vehicle by adopting the weak manual intervention method, the vehicle damage assessment unit 30 is used to:
[0075] Sending interactive questions to the user through the terminal device, wherein the interactive questions are used to inquire about the damage conditions of the damaged parts of the accident vehicle;
[0076] receiving, through the terminal device, a user's answer to the interactive question;
[0077] If the answer is yes, then the damage is assessed based on the damage to the damaged parts of the accident vehicle.
[0078] Furthermore, the vehicle damage assessment unit 30 is also used for:
[0079] If the answer is no, instructing the user to retake pictures of the damaged parts of the accident vehicle through the terminal device;
[0080] The damage is assessed based on the re-photographed pictures of the damaged parts of the accident vehicle.
[0081] Specifically, the damage condition includes at least one of the following: cracking, wrinkling, missing, denting, and scratching, and the repair plan includes at least one of the following: replacement, sheet metal, and painting;
[0082] When the repair solution determined by two of the three models, namely, the component repair machine learning model, the damage morphology repair machine learning model and the separate repair machine learning model, is replacement, the interactive question is used to inquire whether the damaged part of the accident vehicle is cracked, wrinkled or missing;
[0083] When the repair solution determined by two of the three models, namely, the component repair machine learning model, the damage morphology repair machine learning model and the separate repair machine learning model, is sheet metal, the interactive question is used to inquire whether the damaged part of the accident vehicle has a dent;
[0084] When the repair solution determined by two of the three models, namely, the component repair machine learning model, the damage morphology repair machine learning model and the individual repair machine learning model, is painting, the interactive question is used to inquire whether the damaged part of the accident vehicle is only scratches.
[0085] Specifically, the vehicle damage assessment unit 30 is used to:
[0086] Sending a location acquisition request to the terminal device;
[0087] Receiving the accident location sent by the terminal device, where the accident location is edited by the user in response to the location acquisition request or obtained by opening the terminal device positioning system;
[0088] Matching a claims adjuster terminal that is idle at the current moment nearby according to the accident location;
[0089] Notify the claims adjuster corresponding to the claims adjuster terminal to go to the accident site to conduct on-site manual damage assessment.
[0090] Among them, the vehicle damage assessment model includes a shared encoder and a component repair decoder, a damage form repair decoder and a separate repair decoder; the shared encoder is used to perform feature fusion encoding on the multimedia data to obtain encoded data; the component repair decoder is used to decode the encoded data to obtain input for the component repair machine learning model; the damage form repair decoder is used to decode the encoded data to obtain input for the damage form repair machine learning model; the separate repair decoder is used to decode the encoded data to obtain input for the separate repair machine learning model.
[0091] It should be noted that those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working process of the above-described device and each module can refer to the corresponding process in the aforementioned method embodiment, and will not be repeated here.
[0092] The above-mentioned device can be implemented in the form of a computer program. Figure 3 Runs on the computer device shown.
[0093] See also Figure 3 , Figure 3 1 is a schematic block diagram of a computer device provided in an embodiment of the present application. The computer device may be a server.
[0094] See also Figure 3 The computer device includes a processor, a memory and a network interface connected through a system bus, wherein the memory may include a non-volatile storage medium and an internal memory.
[0095] The non-volatile storage medium can store an operating system and a computer program. The computer program includes program instructions, and when the program instructions are executed, the processor can execute any interactive vehicle damage assessment method based on multi-model fusion.
[0096] The processor is used to provide computing and control capabilities and support the operation of the entire computer equipment.
[0097] The internal memory provides an environment for the operation of the computer program in the non-volatile storage medium. When the computer program is executed by the processor, the processor can execute any interactive vehicle damage assessment method based on multi-model fusion.
[0098] The network interface is used for network communication, such as sending assigned tasks, etc. Those skilled in the art will understand that Figure 3The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.
[0099] It should be understood that the processor may be a central processing unit (CPU), and the processor may also be other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. Among them, the general-purpose processor may be a microprocessor or the processor may also be any conventional processor, etc.
[0100] In one embodiment, the processor is used to run a computer program stored in the memory to implement the following steps:
[0101] Receiving multimedia data uploaded by a user through a terminal device, wherein the multimedia data is collected by the user through the terminal device and is used to assess damage to the accident vehicle when a traffic accident occurs;
[0102] Input the multimedia data into a trained vehicle damage assessment model, wherein the vehicle damage assessment model includes a component repair machine learning model, a damage morphology repair machine learning model, and a separate repair machine learning model. The component repair machine learning model is used to determine a repair plan according to the degree of damage to the components of the accident vehicle, the damage morphology repair machine learning model is used to determine a repair plan according to the damage morphology of the accident vehicle, and the separate repair machine learning model is used to directly determine a repair plan;
[0103] The accident vehicle damage is assessed based on the repair plan output by the component repair machine learning model, the damage morphology repair machine learning model and the individual repair machine learning model.
[0104] In one embodiment, in the aspect of assessing the damage of an accident vehicle according to the repair scheme output by the component repair machine learning model, the damage morphology repair machine learning model, and the separate repair machine learning model, the processor is used to run a computer program stored in a memory to implement the following steps:
[0105] If the repair plans determined by the component repair machine learning model, the damage morphology repair machine learning model and the separate repair machine learning model are the same, then the output result of the vehicle damage assessment model is judged to be completely credible, and the accident vehicle damage is assessed based on the repair plans determined by the component repair machine learning model, the damage morphology repair machine learning model and the separate repair machine learning model;
[0106] If the repair plans determined by two of the three models, namely, the component repair machine learning model, the damage morphology repair machine learning model and the separate repair machine learning model, are the same, then it is determined that the output result of the vehicle damage assessment model has a high credibility, and a weak manual intervention method is adopted to assess the damage of the accident vehicle;
[0107] If the repair plans determined by the component repair machine learning model, the damage morphology repair machine learning model and the individual repair machine learning model are all different, then the output result of the vehicle damage assessment model is judged to be completely unreliable, and strong manual intervention is required to conduct damage assessment of the accident vehicle.
[0108] In one embodiment, in the aspect of assessing the damage of the accident vehicle by adopting a weak manual intervention method, the processor is used to run a computer program stored in a memory to implement the following steps:
[0109] Sending interactive questions to the user through the terminal device, wherein the interactive questions are used to inquire about the damage conditions of the damaged parts of the accident vehicle;
[0110] receiving, through the terminal device, a user's answer to the interactive question;
[0111] If the answer is yes, then the damage is assessed based on the damage to the damaged parts of the accident vehicle.
[0112] In one embodiment, the processor is further configured to run a computer program stored in the memory to implement the following steps:
[0113] If the answer is no, instructing the user to retake pictures of the damaged parts of the accident vehicle through the terminal device;
[0114] The damage is assessed based on the re-photographed pictures of the damaged parts of the accident vehicle.
[0115] In one embodiment, the damage condition includes at least one of the following: cracking, wrinkling, missing, denting, and scratching, and the repair solution includes at least one of the following: replacement, sheet metal, and painting;
[0116] When the repair solution determined by two of the three models, namely, the component repair machine learning model, the damage morphology repair machine learning model and the separate repair machine learning model, is replacement, the interactive question is used to inquire whether the damaged part of the accident vehicle is cracked, wrinkled or missing;
[0117] When the repair solution determined by two of the three models, namely, the component repair machine learning model, the damage morphology repair machine learning model and the separate repair machine learning model, is sheet metal, the interactive question is used to inquire whether the damaged part of the accident vehicle has a dent;
[0118] When the repair solution determined by two of the three models, namely, the component repair machine learning model, the damage morphology repair machine learning model and the individual repair machine learning model, is painting, the interactive question is used to inquire whether the damaged part of the accident vehicle is only scratches.
[0119] In one embodiment, in the aspect of carrying out damage assessment of accident vehicles by strong manual intervention, the processor is used to run a computer program stored in a memory to implement the following steps:
[0120] Sending a location acquisition request to the terminal device;
[0121] Receiving the accident location sent by the terminal device, where the accident location is edited by the user in response to the location acquisition request or obtained by opening the terminal device positioning system;
[0122] Matching a claims adjuster terminal that is idle at the current moment nearby according to the accident location;
[0123] Notify the claims adjuster corresponding to the claims adjuster terminal to go to the accident site to conduct on-site manual damage assessment.
[0124] In one embodiment, the vehicle damage assessment model includes a shared encoder and a component repair decoder, a damage morphology repair decoder and a separate repair decoder; the shared encoder is used to perform feature fusion encoding on the multimedia data to obtain encoded data; the component repair decoder is used to decode the encoded data to obtain input for the component repair machine learning model; the damage morphology repair decoder is used to decode the encoded data to obtain input for the damage morphology repair machine learning model; the separate repair decoder is used to decode the encoded data to obtain input for the separate repair machine learning model.
[0125] A computer-readable storage medium is also provided in an embodiment of the present application, wherein the computer-readable storage medium stores a computer program, wherein the computer program includes program instructions, and the processor executes the program instructions to implement any one of the interactive vehicle damage assessment methods based on multi-model fusion provided in the embodiments of the present application.
[0126] The computer-readable storage medium may be an internal storage unit of the computer device described in the foregoing embodiment, such as a hard disk or memory of the computer device. The computer-readable storage medium may also be an external storage device of the computer device, such as a plug-in hard disk, a smart memory card (Smart Medi a Card, SMC), a secure digital (Secure Digital, SD) card, a flash memory card (Flash Card), etc., equipped on the computer device.
[0127] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any technician familiar with the technical field can easily think of various equivalent modifications or replacements within the technical scope disclosed in the present application, and these modifications or replacements should be included in the protection scope of the present application. Therefore, the protection scope of the present application shall be based on the protection scope of the claims.
Claims
1. An interactive vehicle damage assessment method based on multi-model fusion, characterized in that: The flow switching method comprises the following steps: Receiving multimedia data uploaded by a user through a terminal device, wherein the multimedia data is collected by the user through the terminal device and is used to assess damage to the accident vehicle when a traffic accident occurs; Input the multimedia data into a trained vehicle damage assessment model, wherein the vehicle damage assessment model includes a component repair machine learning model, a damage morphology repair machine learning model, and a separate repair machine learning model. The component repair machine learning model is used to determine a repair plan according to the degree of damage to the components of the accident vehicle, the damage morphology repair machine learning model is used to determine a repair plan according to the damage morphology of the accident vehicle, and the separate repair machine learning model is used to directly determine a repair plan; The accident vehicle damage is assessed based on the repair plan output by the component repair machine learning model, the damage morphology repair machine learning model and the individual repair machine learning model.
2. The interactive vehicle damage assessment method based on multi-model fusion according to claim 1, characterized in that: The performing damage assessment on the accident vehicle according to the repair plan output by the component repair machine learning model, the damage morphology repair machine learning model and the separate repair machine learning model comprises: If the repair plans determined by the component repair machine learning model, the damage morphology repair machine learning model and the separate repair machine learning model are the same, then the output result of the vehicle damage assessment model is judged to be completely credible, and the accident vehicle damage is assessed based on the repair plans determined by the component repair machine learning model, the damage morphology repair machine learning model and the separate repair machine learning model; If the repair plans determined by two of the three models, namely, the component repair machine learning model, the damage morphology repair machine learning model and the separate repair machine learning model, are the same, then it is determined that the output result of the vehicle damage assessment model has a high credibility, and a weak manual intervention method is adopted to assess the damage of the accident vehicle; If the repair plans determined by the component repair machine learning model, the damage morphology repair machine learning model and the individual repair machine learning model are all different, then the output result of the vehicle damage assessment model is judged to be completely unreliable, and strong manual intervention is required to conduct damage assessment of the accident vehicle.
3. The interactive vehicle damage assessment method based on multi-model fusion according to claim 2, characterized in that: The method of using weak manual intervention to assess the damage of the accident vehicle includes: Sending interactive questions to the user through the terminal device, wherein the interactive questions are used to inquire about the damage condition of the damaged parts of the accident vehicle; receiving, through the terminal device, a user's answer to the interactive question; If the answer is yes, then the damage is assessed based on the damage to the damaged parts of the accident vehicle.
4. The interactive vehicle damage assessment method based on multi-model fusion as claimed in claim 3, characterized in that: The interactive vehicle damage assessment method based on multi-model fusion also includes: If the answer is no, instructing the user to retake pictures of the damaged parts of the accident vehicle through the terminal device; The damage is assessed based on the re-photographed pictures of the damaged parts of the accident vehicle.
5. The interactive vehicle damage assessment method based on multi-model fusion according to claim 3 or 4, characterized in that: The damage condition includes at least one of the following: cracking, wrinkling, missing, denting, and scratching, and the repair plan includes at least one of the following: replacement, sheet metal, and painting; When the repair solution determined by two of the three models, namely, the component repair machine learning model, the damage morphology repair machine learning model and the separate repair machine learning model, is replacement, the interactive question is used to inquire whether the damaged part of the accident vehicle is cracked, wrinkled or missing; When the repair solution determined by two of the three models, namely, the component repair machine learning model, the damage morphology repair machine learning model and the separate repair machine learning model, is sheet metal, the interactive question is used to inquire whether the damaged part of the accident vehicle has a dent; When the repair solution determined by two of the three models, namely, the component repair machine learning model, the damage morphology repair machine learning model and the individual repair machine learning model, is painting, the interactive question is used to inquire whether the damaged part of the accident vehicle is only scratches.
6. The interactive vehicle damage assessment method based on multi-model fusion according to claim 2, characterized in that: The strong manual intervention method for assessing the damage of the accident vehicle includes: Sending a location acquisition request to the terminal device; Receiving the accident location sent by the terminal device, where the accident location is edited by the user in response to the location acquisition request or obtained by opening the terminal device positioning system; Matching a claims adjuster terminal that is idle at the current moment nearby according to the accident location; Notify the claims adjuster corresponding to the claims adjuster terminal to go to the accident site to conduct on-site manual damage assessment.
7. The interactive vehicle damage assessment method based on multi-model fusion according to claim 1, characterized in that: The vehicle damage assessment model includes a shared encoder and a component repair decoder, a damage morphology repair decoder and a separate repair decoder; the shared encoder is used to perform feature fusion encoding on the multimedia data to obtain encoded data; The component repair decoder is used to decode the encoded data to obtain the input for the component repair machine learning model; the damage morphology repair decoder is used to decode the encoded data to obtain the input for the damage morphology repair machine learning model; the individual repair decoder is used to decode the encoded data to obtain the input for the individual repair machine learning model.
8. An interactive vehicle damage assessment device based on multi-model fusion, characterized in that: The interactive vehicle damage assessment device based on multi-model fusion includes: A data acquisition unit, used to receive multimedia data uploaded by a user through a terminal device, wherein the multimedia data is collected by the user through the terminal device and is used to assess the damage of the accident vehicle when a traffic accident occurs; a model operation unit, configured to input the multimedia data into a trained vehicle damage assessment model, wherein the vehicle damage assessment model includes a component repair machine learning model, a damage morphology repair machine learning model, and a separate repair machine learning model, wherein the component repair machine learning model is configured to determine a repair plan according to the degree of damage to the components of the accident vehicle, the damage morphology repair machine learning model is configured to determine a repair plan according to the damage morphology of the accident vehicle, and the separate repair machine learning model is configured to directly determine a repair plan; A vehicle damage assessment unit is used to assess the damage of an accident vehicle based on the repair plan output by the component repair machine learning model, the damage morphology repair machine learning model and the individual repair machine learning model.
9. A computer device, characterized in that: The computer device includes a processor, a memory, and an interactive vehicle damage assessment program based on multi-model fusion stored in the memory and executable by the processor, wherein when the interactive vehicle damage assessment program based on multi-model fusion is executed by the processor, the steps of the interactive vehicle damage assessment method based on multi-model fusion as described in any one of claims 1 to 7 are implemented.
10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores an interactive vehicle damage assessment program based on multi-model fusion, wherein when the interactive vehicle damage assessment program based on multi-model fusion is executed by a processor, the steps of the interactive vehicle damage assessment method based on multi-model fusion as described in any one of claims 1 to 7 are implemented.