Method, system and device for training car insurance accident photo sorting model and storage medium

Abstract

The present application relates to the technical field of image processing and reinforcement learning, and specifically discloses a vehicle insurance accident photo sorting model training method, system, device and storage medium. The technical scheme of the present application inputs multiple photos of the same vehicle insurance case into a visual language model, outputs a result containing a predicted vehicle group and a predicted scene list, introduces a gating coefficient to filter illegal output and mutual exclusion attribution, calculates a sub-item score and weighted sum when the gating coefficient is non-zero, multiplies the total score by the gating coefficient to obtain a reward value and maximizes it to update the parameters, solves the defects that the training task form is inconsistent with the main business target, the sparse reward is insufficient for fine-grained grouping error correction, the lack of gating leads to illegal structure or serious mutual exclusion of the output still getting high scores, improves the accuracy of the division of the multiple vehicle photo set, the accuracy of the scene boundary determination, and the reliability of the key field output, and reduces the field illusion rate.

Description

Technical Field

[0001] This invention relates to the fields of image processing and reinforcement learning technology, and in particular to a method, system, device and storage medium for training a vehicle insurance accident photo sorting model. Background Technology

[0002] Intelligent sorting of car insurance accident photos is a crucial step in automating claims settlement. Existing technologies have significant shortcomings in addressing this problem: early methods solidified manual sorting experience into rules, relying on preset fields for photo classification. However, when faced with open scenarios such as multiple vehicles of the same color and brand, or mixed documents and panoramic views, the rule coverage is insufficient, and effective supervision cannot be provided for training large multimodal models. Single-image classification or similarity clustering methods process each photo independently, and the supervision signal cannot express the goal of grouping photos of the same vehicle into the same set. Full-text JSON mimicry supervision, using the full-text annotation as the cross-entropy target, can learn the output format, but lacks dense and separable optimization signals for key fields such as pairing and grouping relationships, scene boundary determination, and VIN codes, which are prone to illusions.

[0003] In reinforcement learning, existing solutions mostly use sparse whole-line right-or-wrong rewards, giving positive feedback only when the output is completely correct. The gradient is sparse, which is insufficient for correcting fine-grained grouping errors of multiple vehicles, scene boundary bias, and field illusion. Some multi-index weighted rewards do not have gating mechanisms, and each item is directly accumulated, which means that outputs with illegal structure or serious mutual exclusion may still get a high total score due to a high value of a certain index, making the training process unstable.

[0004] The fundamental flaw of the aforementioned methods lies in the inconsistency between the training task format and the primary business objective. The core business requirement is to achieve multi-vehicle grouping and document-based scenario segmentation, while existing training methods employ single-image classification or whole-segment imitation, resulting in a severe disconnect between the reward signal and the task objective. Furthermore, existing technologies lack dense, verifiable rewards aligned with the multi-vehicle set segmentation, leading to poor correction of grouping errors, scene boundary errors, and field illusions. In handling key fields such as VIN codes and license plates, encouraging model completion without evidence results in a high illusion rate, while lacking hierarchical constraints on format and consistency even with evidence.

[0005] Therefore, there is an urgent need to provide a technical solution to address the above problems. Summary of the Invention

[0006] To address the aforementioned technical problems, this invention provides a method, system, device, and storage medium for training a vehicle insurance accident photo sorting model.

[0007] Firstly, the present invention provides a method for training a vehicle insurance accident photo sorting model, the technical solution of which is as follows: Obtain multiple accident photos and corresponding supervision information for the same auto insurance case. The supervision information includes truth value grouping of the vehicle group to which the accident photos belong and scene-type photo annotations. The multiple accident photos are input into a visual language model to obtain the prediction results output by the visual language model. The prediction results include a list of predicted vehicle groups and a list of predicted scenes. Each predicted vehicle group contains a list of photo numbers belonging to the same accident vehicle. The gating coefficient is calculated based on the prediction result and the supervision information. When the prediction result cannot be parsed into legal structured data, the gating coefficient is set to zero. When the same photo number belongs to both the predicted vehicle group and the predicted scene list, the gating coefficient is set to a positive number less than 1. When the gating coefficient is not zero, multiple sub-scores are calculated based on the prediction results and the supervision information; The weighted sum of the multiple sub-scores is multiplied by the gating coefficient to obtain the total reward value. The parameters of the visual language model are then updated with the goal of maximizing the total reward value, so that the trained visual language model can be used to sort car insurance accident photos.

[0008] The beneficial effects of the vehicle insurance accident photo sorting model training method of the present invention are as follows: The method of this invention inputs multiple photos of the same car insurance case into a visual language model, outputting a list of predicted vehicle groups and predicted scenes. It introduces a gating coefficient to filter illegal outputs and mutually exclusive attributions. When the gating coefficient is non-zero, it calculates the scores of each item and performs a weighted sum. The total score is multiplied by the gating coefficient to obtain the reward value, which is maximized to update the parameters. This method solves the defects of inconsistent training task form with business objectives, insufficient correction of fine-grained grouping errors by sparse rewards, and high scores still obtained by outputs with illegal structures or severe mutually exclusive attributions due to lack of gating. It improves the accuracy of multi-vehicle photo set division, the precision of scene boundary determination, and the reliability of key field output, and reduces the field illusion rate.

[0009] Based on the above scheme, the training method for a vehicle insurance accident photo sorting model of the present invention can be further improved as follows.

[0010] In one alternative approach, before inputting the plurality of accident photos into the visual language model, the method further includes: The visual language model is fine-tuned under supervision using structured JSON that does not contain document-type fields as supervision labels. The supervision labels include the truth value groupings and the scene-type photo annotations.

[0011] The beneficial effects of adopting the above optional approach are as follows: further utilizing structured JSON supervision tags without document-type fields to perform supervised fine-tuning of the visual language model, enabling the model to learn output format specifications and grouping rules in advance, strengthening its cognitive ability of vehicle affiliation and scene category, and providing a foundation for subsequent reinforcement learning training with correct format and accurate fields.

[0012] In one alternative approach, the step of updating the parameters of the visual language model includes: For multiple accident photos of the same car insurance case, at least two candidate prediction results are sampled, and the total reward value corresponding to each candidate prediction result is calculated. The parameters of the visual language model are updated after comparing the relative size of the total reward values ​​between the at least two candidate prediction results.

[0013] The advantages of adopting the above optional method are: further updating the model parameters by sampling at least two candidate prediction results and comparing the relative size of the total reward value, enabling the model to identify the differences in output quality, learn a better grouping strategy from the comparison, avoid the bias caused by single sample gradient update, and enhance the stability of the training process and the optimization effect of the grouping strategy.

[0014] In one alternative approach, the multiple sub-scores include format validity score, coverage validity score, attribution mutual exclusion score, paired grouping score, document type complement score, scene attribution score, vehicle identification number score, license plate score, and vehicle count score.

[0015] The beneficial effects of adopting the above optional methods are as follows: the reward signal is further refined into nine scores: format, coverage, mutual exclusion, pairing, complement, scene, vehicle identification code, license plate and number of vehicles, so that the optimization goal is closely aligned with business needs, and the model can obtain quantifiable feedback on multiple dimensions at the same time, thereby improving the consistency between photo sorting results and supervision information.

[0016] In one alternative approach, the formula for calculating the total reward value is: in, The total reward value, The gating coefficient is... Score the legality of the format. The score for the legality of the coverage. The score for the mutually exclusive affiliation is given. The paired group scores are given. The score is the complement of the document type. Assign a score to the scenario. The vehicle identification number is scored. Score the license plate. Score the number of vehicles. , , , , , , , , These are the weights corresponding to the scores of each sub-item, with the weight of the paired grouped scores being the highest.

[0017] The beneficial effects of adopting the above optional method are: further assigning the highest weight to the paired group score when weighted summing, making the training objective focus on the accuracy of the matching relationship of photos of the same vehicle, strengthening the model's ability to judge the vehicle ownership boundary, and guiding the model to prioritize the correctness of photo grouping in multi-vehicle accident scenarios.

[0018] In one alternative approach, the step of calculating the pairwise grouping score includes: Based on the true value grouping in the supervision information, construct a true value set of photos of the same vehicle; based on the predicted vehicle grouping in the prediction results, construct a predicted set of photos of the same vehicle; compare the degree of overlap between the true value set of photos of the same vehicle and the predicted set of photos of the same vehicle; determine the score of the paired grouping based on the degree of overlap, and the higher the degree of overlap, the higher the score of the paired grouping. In this context, each true vehicle photo pair in the set of true vehicle photo pairs consists of two photos belonging to the same vehicle group, and each predicted vehicle photo pair in the set of predicted vehicle photo pairs consists of two photos within the same predicted vehicle group.

[0019] The beneficial effects of adopting the above optional method are as follows: by further comparing the degree of overlap between the true set of photos of the same vehicle and the predicted set of photos of the same vehicle, the pair grouping score is determined, the quality of set partitioning is transformed into a quantifiable pairing index, so that the model can obtain clear optimization guidance on the attribution relationship at the photo level and improve the accuracy of grouping judgment.

[0020] In one alternative approach, the step of calculating the scene attribution score includes: The consistency between the scene-type photo annotations in the supervision information and the predicted scene list is compared. The scene affiliation score is determined based on the consistency. Additional points are deducted when a photo is annotated as a scene-type photo in the supervision information and belongs to the predicted vehicle group. The scene affiliation score is obtained after the points are deducted. The steps for calculating the vehicle identification number score and the license plate score include: The predicted vehicle group is matched with the true vehicle group in the supervision information. The consistency of the vehicle identification number and license plate in the predicted vehicle group with the corresponding field in the supervision information is compared for each vehicle. The vehicle identification number score and the license plate score are determined according to the consistency. The score when the vehicle identification number or license plate in the predicted vehicle group is empty and the corresponding field in the supervision information is also empty is higher than the score when the vehicle identification number or license plate in the predicted vehicle group is not empty and the corresponding field in the supervision information is empty.

[0021] The beneficial effects of adopting the above optional methods are: to further impose additional penalties when scene-type photos are incorrectly classified into vehicle groups, to avoid confusion between scene and vehicle attribution, and to give higher rewards for leaving key fields blank when there is no evidence, to suppress the model from generating vehicle identification codes or license plates out of thin air, and to reduce the risk of field illusion.

[0022] Secondly, this invention provides a vehicle insurance accident photo sorting model training system, the technical solution of which is as follows: The acquisition module is used to acquire multiple accident photos and corresponding supervision information for the same car insurance case. The supervision information includes the truth grouping of the vehicle group to which the accident photos belong and the scene-type photo annotation. The prediction module is used to input the multiple accident photos into the visual language model and obtain the prediction results output by the visual language model. The prediction results include a predicted vehicle group and a predicted scene list. Each predicted vehicle group contains a list of photo numbers belonging to the same accident vehicle. The processing module is used to calculate the gate coefficient based on the prediction result and the supervision information. When the prediction result cannot be parsed into legal structured data, the gate coefficient is set to zero. When there are photos with the same sequence number that belong to both the predicted vehicle group and the predicted scene list, the gate coefficient is set to a positive number less than 1. The calculation module is used to calculate multiple sub-scores based on the prediction result and the supervision information when the gating coefficient is not zero; The update module is used to multiply the weighted sum of the multiple sub-scores by the gating coefficient to obtain the total reward value, and update the parameters of the visual language model with the goal of maximizing the total reward value, so that the trained visual language model can be used to sort car insurance accident photos.

[0023] The beneficial effects of the vehicle insurance accident photo sorting model training system of the present invention are as follows: The system of this invention inputs multiple photos of the same car insurance case into a visual language model and outputs a list of predicted vehicle groups and predicted scenes. It introduces a gating coefficient to filter illegal outputs and mutually exclusive attributions. When the gating coefficient is non-zero, it calculates the scores of each item and performs a weighted sum. The total score is multiplied by the gating coefficient to obtain the reward value, which is maximized to update the parameters. This solves the defects of inconsistent training task form with business objectives, insufficient correction of fine-grained grouping errors by sparse rewards, and high scores still obtained by outputs with illegal structures or severe mutually exclusive attributions due to lack of gating. It improves the accuracy of multi-vehicle photo set division, the precision of scene boundary determination, and the reliability of key field output, and reduces the field illusion rate.

[0024] Thirdly, the technical solution of an electronic device according to the present invention is as follows: The system includes a memory, a processor, and a program stored in the memory and running on the processor, wherein the processor executes the program to implement the steps of the vehicle insurance accident photo sorting model training method of the present invention.

[0025] Fourthly, the technical solution of a computer-readable storage medium provided by the present invention is as follows: The computer-readable storage medium stores instructions that, when read, cause the computer-readable storage medium to perform the steps of the vehicle insurance accident photo sorting model training method of the present invention.

[0026] The above description is merely an overview of the technical solution of the present invention. In order to better understand the technical means of the present invention, it can be implemented according to the contents of the specification. In order to make the above and other objects, features and advantages of the present invention more obvious and understandable, specific embodiments of the present invention are described below. Attached Figure Description

[0027] The accompanying drawings are for illustrative purposes only and are not intended to limit the invention. Furthermore, the same reference numerals denote the same parts throughout the drawings. In the drawings: Figure 1 This is a flowchart illustrating an embodiment of a vehicle insurance accident photo sorting model training method according to the present invention. Figure 2 This is a schematic diagram of an embodiment of a vehicle insurance accident photo sorting model training system according to the present invention; Figure 3 This is a schematic diagram of an embodiment of an electronic device according to the present invention. Detailed Implementation

[0028] Exemplary embodiments of the invention will now be described in more detail with reference to the accompanying drawings. While exemplary embodiments of the invention are shown in the drawings, it should be understood that the invention can be implemented in various forms and should not be limited to the embodiments set forth herein.

[0029] Figure 1 This diagram illustrates a flowchart of an embodiment of a vehicle insurance accident photo sorting model training method provided by the present invention. This method can be executed by electronic devices such as terminal devices or servers. The terminal device can be any fixed or mobile terminal, such as user equipment (UE), mobile device, user terminal, terminal, cellular phone, cordless phone, personal digital assistant (PDA), handheld device, computing device, in-vehicle device, or wearable device. The server can be a single server or a server cluster consisting of multiple servers. Any electronic device can implement the vehicle insurance accident photo sorting model training method by having its processor call computer-readable instructions stored in its memory. Figure 1 As shown, it includes the following steps: S1. Obtain multiple accident photos and corresponding supervision information for the same vehicle insurance case. The supervision information includes the truth value grouping of the vehicle group to which the accident photos belong and the scene-type photo annotation.

[0030] Accident photos refer to digital image files taken after a traffic accident to record vehicle damage, the accident scene environment, and relevant document information. For example, a car insurance case on August 1, 2026, includes four accident photos: Photo 1 is a panoramic view of the accident scene, Photo 2 shows the damage to the left front side of a white sedan, Photo 3 shows the damage to the right front side of a white sedan, and Photo 4 shows the damage to the left side of a black SUV. Supervisory information refers to pre-labeled ground truth data for multiple accident photos of the same car insurance case, used to provide a standard answer during model training to calculate the accuracy of the prediction results. For example, for the aforementioned four accident photos, the supervisory information includes ground truth grouping of the vehicle group to which each accident photo belongs and scene-type photo annotations.

[0031] The truth grouping of vehicle groups refers to the set of photo numbers in the surveillance information that belong to the same accident vehicle, indicating which photos depict the same vehicle. For example, if the surveillance information indicates that photos 2 and 3 belong to the same white sedan, then the truth grouping of vehicle group V1 includes photo numbers 2 and 3; if the surveillance information indicates that photo 4 belongs to a black SUV, then the truth grouping of vehicle group V2 includes photo number 4. Scene-type photo annotation refers to the annotation data in the surveillance information that marks photos capturing a panoramic view of the accident scene or the overall environment as scene-type photos, indicating that the photo should not belong to any vehicle group. For example, if the surveillance information labels photo 1 as a scene-type photo, it means that photo 1 is a panoramic view of the accident scene and does not belong to any vehicle group.

[0032] S2. Input the multiple accident photos into the visual language model and obtain the prediction results output by the visual language model. The prediction results include a predicted vehicle group and a predicted scene list. Each predicted vehicle group contains a list of photo numbers belonging to the same accident vehicle.

[0033] Among them, the visual language model refers to an artificial intelligence model that can simultaneously understand image content and text semantics and generate structured text output, which serves as a car insurance accident photo sorting model to be trained; for example, after the aforementioned four accident photos are input into the visual language model, the model outputs a structured prediction result containing vehicle group information and scene information.

[0034] The predicted vehicle group refers to the set of photo numbers predicted by the visual language model based on the input accident photos, indicating which photos belong to the same accident vehicle. For example, in the prediction results output by the visual language model for the aforementioned four accident photos, predicted vehicle group V1 includes photo numbers 1, 2, and 3, and predicted vehicle group V2 includes photo number 4. The predicted scene list refers to the set of photo numbers predicted by the visual language model based on the input accident photos, indicating which photos represent a panoramic view of the accident scene or the overall environmental conditions. For example, in the prediction results output by the visual language model for the aforementioned four accident photos, the predicted scene list includes photo number 1.

[0035] The photo sequence number list refers to a list of unique numerical numbers assigned to each accident photo in the same vehicle insurance case, used to identify and reference each accident photo in the structured prediction results. For example, the aforementioned four accident photos have photo sequence numbers 1, 2, 3 and 4 respectively according to the input order. The photo sequence number list of the predicted vehicle group V1 is 1, 2 and 3, indicating that the model determines that photos 1, 2 and 3 belong to the same accident vehicle.

[0036] S3. Calculate the gating coefficient based on the prediction result and the supervision information. When the prediction result cannot be parsed into legal structured data, set the gating coefficient to zero. When there are photos with the same serial number that belong to both the predicted vehicle group and the predicted scene list, set the gating coefficient to a positive number less than 1.

[0037] The gating coefficient is a value calculated based on the structural legality and attribution exclusivity of the prediction results. It is used to penalize prediction results with illegal structures or attribution exclusivity by applying the total reward value. For example, in the prediction results output by the visual language model, the predicted vehicle group V1 contains photo numbers 1, 2, and 3, and the predicted scene list contains photo number 1. Since photo number 1 belongs to both the predicted vehicle group V1 and the predicted scene list, there is attribution exclusivity, so the gating coefficient is set to 0.4. If there is no exclusivity and the structure is legal, the gating coefficient is set to 1.

[0038] Among them, legal structured data refers to: the prediction results can be correctly parsed into structured JSON data that meets the preset format requirements, and contains key fields such as the predicted vehicle group and the predicted scene list; for example, the prediction results output by the visual language model are in the format of a "vehicles" list and a "scene" list, such as {"vehicles":[{"vehicle_id":"V1","photo_indices":[1,2,3]},{"vehicle_id":"V2","photo_indices":[4]}],"scene":[1]}, this result can be parsed normally and contains a vehicle list and a scene list, which is legal structured data.

[0039] Specifically, when the same photo number belongs to both the predicted vehicle group and the predicted scene list, the gating coefficient is determined based on the ratio of the number of conflict photos to the total number of accident photos. The higher the proportion of conflict photos to the total number of accident photos, the lower the gating coefficient. The gating coefficient is a positive number between 0 and 1.

[0040] S4. When the gating coefficient is not zero, calculate multiple sub-scores based on the prediction results and the supervision information.

[0041] Among them, the multiple sub-scores refer to the evaluation scores of multiple dimensions calculated based on the prediction results and supervision information when the gating coefficient is not zero. These scores are used to quantitatively evaluate different aspects of the prediction results. For example, in the prediction results output by the visual language model, the following nine sub-scores are calculated: format legality score 0.9, coverage legality score 1.0, membership mutual exclusion score 0.6, paired grouping score 0.85, document class complement score 0.9, scene membership score 0.8, vehicle identification code score 0.7, license plate score 0.7, and vehicle count score 1.0.

[0042] S5. The weighted sum of the multiple sub-scores is multiplied by the gating coefficient to obtain the total reward value. The parameters of the visual language model are updated with the goal of maximizing the total reward value, so that the trained visual language model can be used to sort car insurance accident photos.

[0043] The total reward value refers to the final scalar value obtained by multiplying the weighted sum of multiple sub-scores by the gating coefficient. It is used to measure the overall accuracy of the prediction results and serves as the optimization target for updating model parameters. For example, the median value of 0.82 is obtained by weighting the nine sub-scores according to their respective weights. After multiplying it by the gating coefficient of 0.4, the total reward value is 0.328. This value is used to guide the direction of updating model parameters.

[0044] Among them, car insurance accident photos refer to digital images taken during the process of motor vehicle insurance accident claims to record the accident scene, the damage to the vehicles involved, and relevant document information; for example, in a motor vehicle insurance accident claim case, the surveyor used a mobile device to take a panoramic photo of the accident scene, a photo of the damage to the left front side of a white sedan, a photo of the damage to the right front side of a white sedan, and a photo of the damage to the left side of a black SUV. These four digital images are all car insurance accident photos.

[0045] The technical solution of this embodiment inputs multiple photos of the same car insurance case into a visual language model, outputs a list of predicted vehicle groups and predicted scenes, introduces a gating coefficient to filter illegal outputs and mutually exclusive attributions, calculates the sub-scores when the gating coefficient is non-zero and sums them by weight, multiplies the total score by the gating coefficient to obtain the reward value and maximizes it to update the parameters, solves the defects of inconsistent training task form with business main objectives, insufficient correction of fine-grained grouping errors by sparse rewards, and high scores still obtained by outputs with illegal structure or severe mutually exclusive attributions due to lack of gating, improves the accuracy of multi-vehicle photo set division, the precision of scene boundary determination and the reliability of key field output, and reduces field illusion rate.

[0046] In one alternative approach, before inputting the plurality of accident photos into the visual language model, the method further includes: The visual language model is fine-tuned under supervision using structured JSON that does not contain document-type fields as supervision labels. The supervision labels include the truth value groupings and the scene-type photo annotations.

[0047] Structured JSON refers to text data organized according to the JavaScript object notation format, with a clear hierarchical structure and field names, used by visual language models to output structured prediction results that are easy for computers to parse; for example, the structured JSON output by the visual language model is {"vehicles":[{"vehicle_id":"V1","photo_indices":[1,2,3]}],"scene":[1]}, and the computer program can directly parse the text and extract vehicle group information and scene information.

[0048] Among them, the supervision label refers to the structured JSON data provided to the visual language model as the standard answer during the supervision fine-tuning stage, which is used to train the model to learn to output prediction results that meet the contract requirements; for example, the supervision label used in the supervision fine-tuning stage is {"vehicles":[{"vehicle_id":"V1","photo_indices":[2,3]}],"scene":[1]}, which does not contain document-type fields, but only contains vehicle group information and scene information.

[0049] In the above optional methods, structured JSON supervision tags without document-type fields are further used to supervise and fine-tune the visual language model, enabling the model to learn output format specifications and grouping rules in advance, strengthening its ability to recognize vehicle ownership and scene categories, and providing a foundation for correct format and accurate fields for subsequent reinforcement learning training.

[0050] In one alternative approach, the step of updating the parameters of the visual language model includes: For multiple accident photos of the same car insurance case, at least two candidate prediction results are sampled, and the total reward value corresponding to each candidate prediction result is calculated. The parameters of the visual language model are updated after comparing the relative size of the total reward values ​​between the at least two candidate prediction results.

[0051] Here, candidate prediction results refer to the multiple structured prediction outputs generated by the visual language model through sampling for multiple accident photos of the same car insurance case during reinforcement learning training. These outputs are used to compare relative merits within a group and update model parameters. For example, the visual language model samples and generates two candidate prediction results for the aforementioned four accident photos. The predicted vehicle group V1 of the first candidate prediction result includes photo numbers 2 and 3, and the predicted vehicle group V1 of the second candidate prediction result includes photo numbers 1, 2, and 3. The total reward value of the two candidate prediction results is calculated and then compared within the group.

[0052] In the above-mentioned optional methods, the model parameters are further updated by sampling at least two candidate prediction results and comparing the relative size of the total reward value. This enables the model to identify the differences in output quality, learn a better grouping strategy from the comparison, avoid the bias caused by single sample gradient updates, and enhance the stability of the training process and the optimization effect of the grouping strategy.

[0053] In one alternative approach, the multiple sub-scores include format validity score, coverage validity score, attribution mutual exclusion score, paired grouping score, document type complement score, scene attribution score, vehicle identification number score, license plate score, and vehicle count score.

[0054] The format validity score refers to a sub-score calculated based on factors such as whether the prediction result can be correctly parsed into structured JSON, whether it contains key fields such as the predicted vehicle group and the predicted scene list, and whether the data types of each field are valid. The format validity score is calculated by checking N preset format rules, counting the number of rules that the prediction result passes, and the ratio of the number of passed rules to N is the format validity score. For example, if the prediction result output by the visual language model can be correctly parsed and contains a vehicle list and a scene list, and the photo number list in each vehicle group is a list of integers, the format validity score is 0.9. The coverage legitimacy score refers to a sub-score calculated based on factors such as whether the photo numbers included in the predicted vehicle group and predicted scenario list in the prediction results exceed the total number of accident photos for the same auto insurance case, and whether there are duplicate photo numbers. The calculation method for the coverage legitimacy score is as follows: count all photo numbers that appear in the prediction results, and calculate the ratio of the number of legitimate photo numbers (within the range of 1 to the total number of accident photos and without duplication) to the total number of photo numbers in the prediction results. This ratio is the coverage legitimacy score. For example, if there are 4 accident photos for the same auto insurance case, and all photo numbers in the prediction results are within the range of 1 to 4 and without duplication, the coverage legitimacy score is 1.0.

[0055] The mutual exclusion score refers to the sub-score calculated based on whether the same photo number appears simultaneously in both the predicted vehicle group and the predicted scene list. It is used to penalize mutual exclusion errors that classify the same photo as belonging to both the vehicle group and the scene category. For example, in the prediction result, photo number 1 appears in both the predicted vehicle group V1 and the predicted scene list, so the mutual exclusion score is 0.6. The pairwise grouping score refers to the sub-score obtained by comparing the degree of overlap between the ground truth set of photos of the same vehicle constructed based on the supervised information and the predicted set of photos of the same vehicle constructed based on the prediction result. It is used to measure the correctness of the prediction result in grouping multiple vehicle photos. For example, the ground truth set of photos of the same vehicle contains a pair (2,3), and the predicted set of photos of the same vehicle contains three pairs (1,2), (1,3), and (2,3). The overlap between the two sets is (2,3), and the pairwise grouping score is calculated as 0.85 based on the degree of overlap.

[0056] The document supplement score refers to the score obtained by comparing the set of photos remaining after removing the photos included in the predicted vehicle group and the predicted scene list from the accident photos, and then comparing this predicted document supplement with the document photo annotations in the supervision information. For example, if the document photo annotation in the supervision information is "none", the predicted vehicle group includes photos 1, 2, and 3, the predicted scene list includes photo 1, and the remaining photo 4 after removing the predicted vehicle group and the predicted scene list is the predicted document supplement, and the predicted document supplement matches the document photo annotations in the supervision information, the document supplement score is 0.9. Scene attribution score refers to the sub-score calculated based on the consistency between the scene-class photo annotations in the supervision information and the predicted scene list. It is used to measure the correctness of the prediction results in determining scene-class photos, and additional points are deducted for the behavior of incorrectly attributing scene-class photos to vehicle groups. For example, if photo 1 is labeled as a scene class in the supervision information, and the predicted scene list contains photo 1, but the predicted vehicle group V1 also contains photo 1, the basic consistency score is 1.0. Because photo 1 is assigned to the predicted vehicle group, an additional point of 0.2 is deducted, and the scene attribution score is 0.8.

[0057] The vehicle identification number (VIN) score refers to the score calculated by comparing the consistency of the VIN field in the predicted vehicle group with the corresponding field in the supervision information after matching the predicted vehicle group with the true vehicle group in the supervision information, and then calculating the score based on the comparison results. For example, if the VIN in the predicted vehicle group V1 is empty and the corresponding field in the supervision information is also empty, the vehicle will get full marks. If the VIN in the predicted vehicle group V2 is a non-empty value "VIN-B" and the corresponding field in the supervision information is empty, the vehicle will get low marks. The average score for each vehicle is 0.7. The license plate score refers to the score calculated by comparing the license plate field in the predicted vehicle group with the ground truth vehicle group in the supervision information, and then comparing the consistency between the predicted vehicle group and the corresponding field in the supervision information for each vehicle. For example, if the license plate in predicted vehicle group V1 is empty and the corresponding field in the supervision information is also empty, the vehicle receives full marks. If the license plate in predicted vehicle group V2 is a non-empty value "PLATE-B" and the corresponding field in the supervision information is also non-empty and consistent, the vehicle receives full marks. The average license plate score for each vehicle is 0.7. The vehicle count score refers to the score calculated based on the closeness between the number of predicted vehicle groups in the prediction results and the number of ground truth vehicle groups in the supervision information. The vehicle count score is calculated as follows: ; Score the number of vehicles. The number of the predicted vehicle group. The number of true vehicle groups is given. For example, the number of true vehicle groups in the supervision information is 2, the number of predicted vehicle groups in the prediction result is 2, and the vehicle count score is 1.0.

[0058] Among the above-mentioned optional methods, the reward signal is further refined into nine scores: format, coverage, mutual exclusion, pairing, complement, scene, vehicle identification code, license plate and number of vehicles. This makes the optimization goal closely correspond to business needs, and the model obtains quantifiable feedback on multiple dimensions at the same time, thereby improving the consistency between photo sorting results and supervision information.

[0059] In one alternative approach, the formula for calculating the total reward value is: in, The total reward value, The gating coefficient is... Score the legality of the format. The score for the legality of the coverage. The score for the mutually exclusive affiliation is given. The paired group scores are given. The score is the complement of the document type. Assign a score to the scenario. The vehicle identification number is scored. Score the license plate. Score the number of vehicles. , , , , , , , , These are the weights corresponding to the scores of each sub-item, with the weight of the paired grouped scores being the highest.

[0060] Among the above optional methods, pairwise grouping scores are further assigned the highest weight during weighted summation, so that the training objective focuses on the accuracy of the pairing relationship of photos of the same vehicle, strengthens the model's ability to judge the vehicle ownership boundary, and guides the model to prioritize the correctness of photo grouping in multi-vehicle accident scenarios.

[0061] In one alternative approach, the step of calculating the pairwise grouping score includes: Based on the true value grouping in the supervision information, construct a set of true vehicle photos; based on the predicted vehicle grouping in the prediction results, construct a set of predicted vehicle photos; compare the degree of overlap between the true vehicle photos and the predicted vehicle photos; determine the score of the paired grouping based on the degree of overlap, with a higher degree of overlap resulting in a higher score for the paired grouping.

[0062] In this context, each true vehicle photo pair in the set of true vehicle photo pairs consists of two photos belonging to the same vehicle group, and each predicted vehicle photo pair in the set of predicted vehicle photo pairs consists of two photos within the same predicted vehicle group.

[0063] The set of true vehicle photos refers to the set of all unordered photo pairs consisting of two accident photos belonging to the same vehicle group, constructed based on the true grouping in the supervision information. This set serves as a standard reference for measuring the accuracy of the prediction grouping. For example, in the true grouping, vehicle group V1 contains photos 2 and 3, and vehicle group V2 contains photo 4. Photos 2 and 3 in vehicle group V1 constitute a true vehicle photo pair (2,3), and the set of true vehicle photos pairs is {(2,3)}.

[0064] The predicted vehicle photo pair set refers to the set of all unordered photo pairs consisting of two accident photos from the same predicted vehicle group, constructed based on the predicted vehicle group in the prediction results. This set is used to compare with the true vehicle photo pair set to evaluate the accuracy of the prediction grouping. For example, the predicted vehicle group V1 in the prediction results contains photos 1, 2, and 3, which can form predicted vehicle photo pairs (1,2), (1,3), and (2,3). The predicted vehicle photo pair set is {(1,2),(1,3),(2,3)}.

[0065] The degree of overlap refers to the number of identical photo pairs commonly contained in the true and predicted sets of photo pairs in the same vehicle, and their relative proportions. This is used to quantify the consistency between the predicted and true groups. For example, the true set of photo pairs in the same vehicle is {(2,3)}, and the predicted set of photo pairs in the same vehicle is {(1,2),(1,3),(2,3)}. The two sets contain 1 identical photo pair (2,3). The degree of overlap is calculated based on the ratio of this number of identical pairs to the total number of identical pairs in the true set of photo pairs in the same vehicle.

[0066] In the above-mentioned optional methods, the pairing score is further determined by comparing the degree of overlap between the true set of photos of the same vehicle and the predicted set of photos of the same vehicle. This transforms the quality of set partitioning into a quantifiable pairing index, giving the model clear optimization guidance on the attribution relationship at the photo level and improving the accuracy of grouping judgment.

[0067] In one alternative approach, the step of calculating the scene attribution score includes: The consistency between the scene-type photo annotations in the supervision information and the predicted scene list is compared. The scene attribution score is determined based on the consistency. Additional points are deducted when a photo is annotated as a scene-type photo in the supervision information and belongs to the predicted vehicle group. The scene attribution score is obtained after the point deduction.

[0068] Among them, scene-type photo annotation refers to the annotation data in the supervision information that marks photos of the overall view of the accident scene or the overall environmental conditions as scene categories, which is used to indicate that the photo should belong to the scene list and not to any vehicle group; for example, if photo 1 is labeled as a scene category in the supervision information, it means that photo 1 is a panoramic photo of the accident scene, and the photo does not belong to any vehicle group in the truth grouping.

[0069] The steps for calculating the vehicle identification number score and the license plate score include: The predicted vehicle group is matched with the true vehicle group in the supervision information. The consistency of the vehicle identification number and license plate in the predicted vehicle group with the corresponding field in the supervision information is compared for each vehicle. The vehicle identification number score and the license plate score are determined according to the consistency. The score when the vehicle identification number or license plate in the predicted vehicle group is empty and the corresponding field in the supervision information is also empty is higher than the score when the vehicle identification number or license plate in the predicted vehicle group is not empty and the corresponding field in the supervision information is empty.

[0070] The Vehicle Identification Number (VIN) is a unique 17-digit alphanumeric code assigned to each motor vehicle by the manufacturer. It is used to uniquely identify the vehicle involved in an accident during the car insurance claims process. For example, the VIN of a white sedan is "VIN-A," and the VIN of a black SUV is "VIN-B." This code is printed on the lower left corner of the vehicle's windshield and can be used to verify the vehicle's identity in the claims photos. The license plate is the registration number recorded on the legal license plates issued by the traffic management department and affixed to the front and rear of the motor vehicle. It is used to identify and confirm the registration information of the vehicle involved in an accident during the car insurance claims process. For example, the license plate of a white sedan is "PLATE-A," and the license plate of a black SUV is "PLATE-B." This license plate is recorded on the license plates affixed to the front and rear of the vehicle and can be used to confirm the vehicle's identity in the accident photos.

[0071] Among the above-mentioned optional methods, additional penalties are imposed when scene-type photos are incorrectly classified into vehicle groups to avoid confusion between scene and vehicle attribution. At the same time, higher rewards are given for leaving key fields blank when there is no evidence, which suppresses the model from generating vehicle identification codes or license plates out of thin air and reduces the risk of field illusion.

[0072] This embodiment describes the training process of a visual language model for sorting car insurance accident photos. The training objective and output contract remain consistent. The rewards for the reinforcement learning phase can be automatically and reproducibly constructed from the prediction results and ground truth data, with the sub-items of multiple vehicles grouped in pairs serving as the main signals.

[0073] For multiple photos of the same accident case, annotations are created based on vehicle instances, scene classes, and other classes to construct ground truth data sets. The other classes include photos of identification documents. When writing the model's supervision objective, only the vehicle group, scene, and summary fields are retained; the identification document field is not output. Structured JSON data is used as supervision labels to perform supervised fine-tuning of the multimodal model, ensuring the model outputs vehicle groupings and scene attributions according to the contract. In the verifiable reward-driven policy optimization phase, multiple candidate outputs are sampled from multiple photo inputs of the same case. The total reward value is obtained by weighting the inputs according to the gating coefficient and the sub-items. The relative merits of the candidate outputs are compared within each group, and the policy is updated accordingly. The training effect is monitored using the verifiable total reward value and the weighted grouping of multiple vehicles. An example of the expected output is shown below. { "vehicles": [ {"vehicle_id":"V1","vin":"","plate_no":"","photo_indices": [1,3,5]}], "scene": [2,7,8] } The task of partitioning multi-vehicle accident photo sets formalizes sorting training into a structured partitioning of the photo sequence set. Vehicle groups refer to photos of vehicle bodies, damage, or partial components whose attribution can be determined, which are then entered into the corresponding vehicle's photo sequence list. Scenes refer to panoramic photos, long-range photos, and case-level environmental photos; panoramic photos are prioritized for the scene list and not for vehicle groups. The vehicle identification number (VIN) and license plate number for each accident vehicle can be empty; leaving them blank is preferable to random entries when there is no evidence. Other categories include document photos, photos whose attribution cannot be determined, and partial component photos whose attribution cannot be determined. These other categories of photos are only used to construct ground truth data and calculate rewards and are not required to be explicitly output by the model.

[0074] The definition of paired positive samples is core to the task: only two photos belonging to the same accident vehicle constitute a positive sample image pair. Photos labeled as other classes or scene classes are not considered as vehicle-to-vehicle pairs. This definition is used to construct grouped rewards, making the optimization object a set partitioning relationship, rather than classifying individual photos with independent labels.

[0075] The purpose of the supervised fine-tuning phase is to ensure the model consistently outputs data in a valid JSON format and acquires basic vehicle grouping and scene attribution capabilities, providing a resolvable starting point for subsequent verifiable reward optimization. The supervised fine-tuning steps include organizing multimodal samples, fine-tuning the visual language model, and monitoring the training process with output resolvability and verifiable rewards. Labels are structured JSON data without other class fields. Fine-grained grouping refinement, scene boundary refinement, and field illusion suppression are completed in the verifiable reward phase.

[0076] In the policy optimization process based on verifiable rewards, at least two candidate outputs are sampled from multiple photo inputs for the same case. The verifiable total reward value is calculated for each candidate output, and then the relative merits of the candidate outputs are compared to update the policy. The total reward value only reflects the degree of correctness or incorrectness of that output relative to the true value.

[0077] For a single candidate output, the prediction result in JSON format is extracted and parsed. The list of vehicle identification numbers, license plates, and photo numbers is normalized, the gating coefficient is calculated, and then the scores for each sub-item are calculated. Each sub-item score is normalized to the range of 0 to 1, and may include penalties before cropping. The scores are then weighted and summed, and multiplied by the gating coefficient to obtain the total reward value.

[0078] The total reward value is in its primary form as the gating coefficient multiplied by the weighted sum of all components. The formula for calculating the total reward value is: Where G is the gating coefficient.

[0079] The format and structure score is used to check whether the prediction results can be correctly parsed, whether there is a vehicle list and a scene list, and whether the field types of each vehicle are valid. Scores are given according to the proportion of items that pass.

[0080] Coverage and legality scores are used to check whether the photo numbers in the prediction results exceed the total number of case photos and whether there are duplicate photo numbers in the scene list. The more out-of-bounds or duplicates there are, the lower the score will be, which is used to suppress the occurrence of missing scores and illegal numbers.

[0081] The mutual exclusion score is used to prevent the same photo number from falling into both the vehicle group union and the scene list simultaneously. A perfect score is awarded when there is no conflict. When a conflict occurs, the score is reduced according to the proportion of conflicting photos to the total number of photos, and a discount is applied to the gating coefficient.

[0082] The main task is to score the paired grouping of vehicles. This involves constructing sets of paired vehicle photos from the ground truth photos and sets of paired vehicle photos from the predicted photos. The score is obtained by comparing the degree of overlap between the two sets. Higher overlap and fewer missed or incorrect pairings result in higher scores. Photos labeled as "other" or "scene" are excluded from the ground truth pairing, ensuring the main signal focuses on whether the accident vehicles are paired. A perfect score is awarded when there are no paired pairs on either side. A score of zero is awarded when there is a paired pair only on the predicted side and no paired pair on the ground truth side.

[0083] In the "Other Categories" section, the model does not output any "Other Category" fields. The remaining photos after removing those classified as vehicles and scenes from the prediction results are used as the complement set for predicted document categories. This complement set is then compared to the document category photos from the labeled side. The closer the complement set is to the labeled set and the fewer document categories are missed, the higher the "Other Categories" score will be. This score is part of the reward scoring and is not part of the post-processing of the model output for "Other Categories".

[0084] Scene attribution score is used to compare the consistency between the ground truth scene list and the predicted scene list, and the scene attribution score is obtained based on the consistency. For cases where a photo that should be a scene in the ground truth is only placed in a vehicle group in the prediction result but not in the scene list, additional points are deducted as a scene boundary penalty. Cases where the same photo is included in both the vehicle group and the scene list are handled by the attribution mutual exclusion score and the gating coefficient.

[0085] The vehicle identification number (VIN) score is calculated as follows: First, the predicted vehicle group is matched with the ground truth vehicle group based on overlap. Then, each vehicle is scored individually, and the average score is taken. A high score is awarded when the VIN in the predicted vehicle group is empty and the corresponding field in the ground truth group is also empty, indicating a valid empty field. A high score is awarded when the VIN content is an exact match. A low score is awarded when the VIN format is correct but the content is incorrect. The lowest score is awarded when a VIN is entered randomly without evidence and in an illegal format. Illegal VINs on unmatched predicted vehicles further lower the score. Field scores are still awarded even if the grouping is incorrect, but grouping scores are deducted as usual; grouping scores and field scores are calculated separately.

[0086] The license plate score is calculated by matching the normalized license plate with the true value, and the scoring logic is similar to the valid empty strategy for the vehicle identification number (VIN) score. If multiple predicted vehicles have the same normalized license plate, additional points are deducted as a penalty for not merging identical plates.

[0087] In the vehicle count score, the closer the predicted number of vehicle groups is to the actual number of vehicle groups, the better. Additional points are deducted if the proportion of predicted vehicle groups containing only one image is too high, to prevent excessive splitting of vehicle groups.

[0088] The gating coefficient is determined before or simultaneously with the individual scoring. When the prediction result cannot be parsed into a valid JSON format, or when key structures such as the vehicle list or scene list are missing to the point of being unusable, the gating coefficient is set to 0, resulting in an extremely low total reward value. When there is a prohibited conflict where the same photo number enters both the vehicle group and the scene list, the gating coefficient is discounted; in this example, it is set to 0.4, meaning that even if the paired group scores happen to be higher, a full score cannot be obtained. Otherwise, the gating coefficient is set to 1. The gating coefficient ensures that outputs with illegal structures or severe mutual exclusions are at a disadvantage in intra-group comparisons, forcing the strategy to satisfy the output contract before optimizing the grouping.

[0089] The design principles for the reward system include: rewards only evaluate the correctness of the output results. If hard constraints are not met, the total reward value is discounted or reset to zero. The main signal is the score for paired vehicle groups, which has the highest weight; in this example, the weight ranges from 0.35 to 0.45. Vehicle grouping, document category complements, and scene attribution are scored separately. Document categories are obtained from the predicted implicit complements and do not depend on other category fields output by the model. Vehicle identification numbers and license plates are allowed to be legally empty; fabricated entries without evidence will be severely punished.

[0090] In the weighted example of the embodiment, the total weight of each item is approximately 1.0, where the format score has a weight of 0.07, the coverage score has a weight of 0.07, the home exclusivity score has a weight of 0.12, the paired group score has a weight of 0.40 as the main signal, the document type complement score has a weight of 0.10, the scene home score has a weight of 0.10, the vehicle identification code score has a weight of 0.06, the license plate score has a weight of 0.05, and the number of vehicles score has a weight of 0.03.

[0091] In another embodiment of the vehicle insurance accident photo sorting model training method of the present invention, it includes: S10. Obtain at least two accident photos of the same car insurance case and the shooting timestamp corresponding to each accident photo. Generate a time tag based on the shooting timestamp. The time tag is used to represent the order of the accident photos in the time series.

[0092] S20. Input at least two accident photos into the visual language model and obtain the structured prediction results output by the visual language model. The structured prediction results include a list of predicted vehicle groups and a list of predicted scenes. Each predicted vehicle group contains a list of photo numbers belonging to the same accident vehicle.

[0093] S30. Calculate the gating coefficient based on the prediction results and supervision information. When the prediction results cannot be parsed into legal structured data, set the gating coefficient to zero. When there are photos with the same sequence number that belong to both the predicted vehicle group and the predicted scene list, set the gating coefficient to a positive number less than 1 based on the ratio of the number of conflict photos to the total number of accident photos.

[0094] S40. When the gating coefficient is not zero, construct a set of true vehicle-to-vehicle photo pairs based on the true value grouping in the supervision information, construct a set of predicted vehicle-to-vehicle photo pairs based on the predicted vehicle grouping in the prediction results, and calculate the pairwise grouping score; when both the true vehicle-to-vehicle photo pairing set and the predicted vehicle-to-vehicle photo pairing set are empty, the pairwise grouping score is set to 1; otherwise, the pairwise grouping score is calculated according to the following formula: in, To score in pairs, For the set of true value photos taken in the same vehicle, To predict the set of photos taken in the same vehicle, Indicates by photo number and photo number The disordered pairs of photographs constituted Photo number The timestamps of the corresponding accident photos, Photo number The timestamps of the corresponding accident photos, The preset positive time decay coefficient, Let be the size of the union of the set of true photos of people in the same vehicle and the set of predicted photos of people in the same vehicle.

[0095] It should be noted that the pairwise grouping score is calculated by summing the time decay weights corresponding to each photo pair in the intersection of the true set of photos of the same vehicle and the predicted set of photos of the same vehicle, and then dividing by the size of the union of the two sets. This quantifies the accuracy of the grouping while introducing a time consistency constraint to modulate the grouping results. The time decay weight adopts an exponential function form, which monotonically decreases as the time interval between the two photos of the same vehicle increases. The decay coefficient is positive, and the time decay coefficient determines the decay rate. This means that even if two photos with large differences in shooting time are assigned to the same vehicle group by the model, the weights they receive will approach zero, thus lowering the pairwise grouping score. This design utilizes the physical law that photos of the same accident vehicle taken from different perspectives are usually taken consecutively within a short period of time, guiding the model to prioritize grouping photos with close time proximity into the same group. The scores for paired grouping range between 0 and 1. When both sets are empty, the score is 1 because there are no photos of the same vehicle for comparison in this case, and the grouping task is considered to produce no error on this sample. When the union of the two sets is not empty, the numerator is the sum of the time decay weights of each photo pair in the intersection, and the denominator is the size of the union. The sum of the weights cannot exceed the size of the union, so the score will not exceed 1, and all items are non-negative, so the score is not lower than 0.

[0096] S50. The weighted sum of the paired group scores and other sub-scores is multiplied by the gating coefficient to obtain the total reward value. With the goal of maximizing the total reward value, the policy gradient is calculated based on the difference between the prediction results and the supervision information. The parameters of the visual language model are updated according to the policy gradient so that the trained visual language model can be used to sort car insurance accident photos.

[0097] The technical solution of this embodiment introduces a time tag constructed based on the timestamp of the accident photos and embeds the time decay weight into the calculation process of the group score. This enables the model to modulate the grouping results by utilizing the proximity of the photos of the same vehicle in terms of shooting time during reinforcement learning training. It guides the model to consider both visual features and temporal consistency when grouping. Combined with the overall suppression of illegal structure and mutually exclusive outputs by the gating coefficient and the hierarchical reward of assigning high scores to legal empty fields, it solves the technical problems of the lack of time dimension constraints in the group supervision signal, the difficulty in effectively distinguishing illegal structure outputs, and the forced filling in of key fields when there is no evidence in the existing training methods. It improves the accuracy of photo set division and the rationality of grouping decisions in multi-vehicle accident scenarios, and reduces the probability of erroneous merging of vehicle groups and field illusion.

[0098] Figure 2This diagram illustrates the structure of an embodiment of a vehicle insurance accident photo sorting model training system 200 provided by the present invention. Figure 2 As shown, the car insurance accident photo sorting model training system 200 includes: The acquisition module 201 is used to acquire multiple accident photos and corresponding supervision information for the same car insurance case. The supervision information includes the truth grouping of the vehicle group to which the accident photos belong and the scene-type photo annotation. Prediction module 202 is used to input the multiple accident photos into a visual language model and obtain the prediction results output by the visual language model. The prediction results include a predicted vehicle group and a predicted scene list. Each predicted vehicle group contains a list of photo numbers belonging to the same accident vehicle. Processing module 203 is used to calculate a gate coefficient based on the prediction result and the supervision information. When the prediction result cannot be parsed into legal structured data, the gate coefficient is set to zero. When there are photos with the same sequence number that belong to both the predicted vehicle group and the predicted scene list, the gate coefficient is set to a positive number less than 1. Calculation module 204 is used to calculate multiple sub-scores based on the prediction result and the supervision information when the gating coefficient is not zero; The update module 205 is used to multiply the weighted sum of the multiple sub-scores by the gating coefficient to obtain the total reward value, and update the parameters of the visual language model with the goal of maximizing the total reward value, so that the trained visual language model can be used to sort car insurance accident photos.

[0099] In an alternative embodiment, the prediction module 202 is further configured to: The visual language model is fine-tuned under supervision using structured JSON that does not contain document-type fields as supervision labels. The supervision labels include the truth value groupings and the scene-type photo annotations.

[0100] In an alternative embodiment, the update module 205 is specifically used for: For multiple accident photos of the same car insurance case, at least two candidate prediction results are sampled, and the total reward value corresponding to each candidate prediction result is calculated. The parameters of the visual language model are updated after comparing the relative size of the total reward values ​​between the at least two candidate prediction results.

[0101] In one alternative approach, the multiple sub-scores include format validity score, coverage validity score, attribution mutual exclusion score, paired grouping score, document type complement score, scene attribution score, vehicle identification number score, license plate score, and vehicle count score.

[0102] In one alternative approach, the formula for calculating the total reward value is: in, The total reward value, The gating coefficient is... Score the legality of the format. The score for the legality of the coverage. The score for the mutually exclusive affiliation is given. The paired group scores are given. The score is the complement of the document type. Assign a score to the scenario. The vehicle identification number is scored. Score the license plate. Score the number of vehicles. , , , , , , , , These are the weights corresponding to the scores of each sub-item, with the weight of the paired grouped scores being the highest.

[0103] In an alternative embodiment, the computing module 204 is specifically used for: Based on the true value grouping in the supervision information, construct a true value set of photos of the same vehicle; based on the predicted vehicle grouping in the prediction results, construct a predicted set of photos of the same vehicle; compare the degree of overlap between the true value set of photos of the same vehicle and the predicted set of photos of the same vehicle; determine the score of the paired grouping based on the degree of overlap, and the higher the degree of overlap, the higher the score of the paired grouping. In this context, each true vehicle photo pair in the set of true vehicle photo pairs consists of two photos belonging to the same vehicle group, and each predicted vehicle photo pair in the set of predicted vehicle photo pairs consists of two photos within the same predicted vehicle group.

[0104] In an alternative embodiment, the computing module 204 is specifically used for: The consistency between the scene-type photo annotations in the supervision information and the predicted scene list is compared. The scene affiliation score is determined based on the consistency. Additional points are deducted when a photo is annotated as a scene-type photo in the supervision information and belongs to the predicted vehicle group. The scene affiliation score is obtained after the points are deducted. The predicted vehicle group is matched with the true vehicle group in the supervision information. The consistency of the vehicle identification number and license plate in the predicted vehicle group with the corresponding field in the supervision information is compared for each vehicle. The vehicle identification number score and the license plate score are determined according to the consistency. The score when the vehicle identification number or license plate in the predicted vehicle group is empty and the corresponding field in the supervision information is also empty is higher than the score when the vehicle identification number or license plate in the predicted vehicle group is not empty and the corresponding field in the supervision information is empty.

[0105] It should be noted that the beneficial effects of the vehicle insurance accident photo sorting model training system 200 provided in the above embodiments are the same as those of the above vehicle insurance accident photo sorting model training method, and will not be repeated here. Furthermore, the system provided in the above embodiments is only illustrated by the division of the above functional modules. In practical applications, the above functions can be assigned to different functional modules as needed, that is, the system can be divided into different functional modules according to the actual situation to complete all or part of the functions described above. In addition, the system and method embodiments provided in the above embodiments belong to the same concept, and their specific implementation process is detailed in the method embodiments, and will not be repeated here.

[0106] The vehicle insurance accident photo sorting model training system 200 of the present invention can be a computer program (including program code) running on a computer device. For example, the vehicle insurance accident photo sorting model training system 200 of the present invention is an application software that can be used to execute the corresponding steps in the vehicle insurance accident photo sorting model training method of the present invention.

[0107] In some embodiments, the vehicle insurance accident photo sorting model training system 200 of the present invention can be implemented in a combination of hardware and software. As an example, the vehicle insurance accident photo sorting model training system 200 of the present invention can be a processor in the form of a hardware decoding processor, which is programmed to execute the vehicle insurance accident photo sorting model training method of the present invention. For example, the processor in the form of a hardware decoding processor can be one or more application specific integrated circuits (ASICs), DSPs, programmable logic devices (PLDs), complex programmable logic devices (CPLDs), field-programmable gate arrays (FPGAs), or other electronic components.

[0108] The modules described in the embodiments of this invention can be implemented in software or hardware. The names of the modules are not, in some cases, limiting the scope of the module itself.

[0109] An electronic device according to an embodiment of the present invention includes a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, it implements any of the above-mentioned methods for training a vehicle insurance accident photo sorting model. That is, an electronic device according to an embodiment of the present invention may include, but is not limited to: a processor and a memory; the memory is used to store the computer program; the processor is used to execute the vehicle insurance accident photo sorting model training method shown in any embodiment of the present invention by calling the computer program.

[0110] In one alternative embodiment, an electronic device is provided, such as Figure 3 As shown, Figure 3 The illustrated electronic device 4000 includes a processor 4001 and a memory 4003. The processor 4001 and the memory 4003 are connected, for example, via a bus 4002. Optionally, the electronic device 4000 may further include a transceiver 4004, which can be used for data interaction between the electronic device and other electronic devices, such as sending and / or receiving data. It should be noted that in practical applications, the transceiver 4004 is not limited to one type, and the structure of the electronic device 4000 does not constitute a limitation on the embodiments of the present invention.

[0111] Processor 4001 may be a CPU (Central Processing Unit), a general-purpose processor, a DSP (Digital Signal Processor), an ASIC (Application Specific Integrated Circuit), an FPGA (Field Programmable Gate Array), or other programmable logic devices, transistor logic devices, hardware components, or any combination thereof. It can implement or execute the various exemplary logic blocks, modules, and circuits described in conjunction with the disclosure of this invention. Processor 4001 may also be a combination that implements computational functions, such as including one or more microprocessor combinations, a combination of a DSP and a microprocessor, etc.

[0112] Bus 4002 may include a path for transmitting information between the aforementioned components. Bus 4002 may be a PCI (Peripheral Component Interconnect) bus or an EISA (Extended Industry Standard Architecture) bus, etc. Bus 4002 can be divided into address bus, data bus, control bus, etc. For ease of representation, Figure 3 The bus 4002 is represented by only one thick line, but this does not mean that there is only one bus or one type of bus.

[0113] The memory 4003 may be ROM (Read Only Memory) or other types of static storage devices capable of storing static information and instructions, RAM (Random Access Memory) or other types of dynamic storage devices capable of storing information and instructions, or EEPROM (Electrically Erasable Programmable Read Only Memory), CD-ROM (Compact Disc Read Only Memory) or other optical disc storage, optical disc storage (including compressed optical discs, laser discs, optical discs, digital universal optical discs, Blu-ray discs, etc.), magnetic disk storage media or other magnetic storage devices, or any other medium capable of carrying or storing desired program code in the form of instructions or data structures and accessible by a computer, but not limited thereto.

[0114] The memory 4003 stores application code (computer program) for executing the present invention, and its execution is controlled by the processor 4001. The processor 4001 executes the application code stored in the memory 4003 to implement the content shown in the foregoing method embodiments.

[0115] Among them, electronic devices can also be terminal devices. A terminal device can be any terminal device that can install applications and access web pages through applications, including at least one of smartphones, tablets, laptops, desktop computers, smart speakers, smartwatches, smart TVs, and smart in-vehicle devices.

[0116] It should be noted that, Figure 3 The electronic device shown is merely an example and should not be construed as limiting the functionality and scope of the embodiments of the present invention.

[0117] An embodiment of the present invention provides a computer-readable storage medium storing a computer program, which, when executed by a processor, implements any of the above-described methods for training a vehicle insurance accident photo sorting model.

[0118] Alternatively, the computer-readable storage medium may be a read-only memory (ROM), a random access memory (RAM), a compact disc read-only memory (CD-ROM), magnetic tape, a floppy disk, and an optical data storage device, etc.

[0119] In an exemplary embodiment, a computer program product or computer program is also provided, which includes computer instructions stored in a computer-readable storage medium. A processor of an electronic device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the electronic device to perform the aforementioned vehicle insurance accident photo sorting model training method.

[0120] Computer program code for performing the operations of this invention can be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, and C++, and conventional procedural programming languages ​​such as C or similar languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving remote computers, the remote computer can be connected to the user's computer via any type of network—including a local area network (LAN) or a wide area network (WAN)—or can be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0121] It should be understood that the flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of methods and computer program products according to various embodiments of the present invention. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing the specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, may be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0122] The computer-readable storage medium provided in this invention can be, but is not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of a computer-readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination thereof. In this invention, a computer-readable storage medium can be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, apparatus, or device.

[0123] The aforementioned computer-readable storage medium carries one or more programs, which, when executed by the electronic device, cause the electronic device to perform the method shown in the above embodiments.

[0124] The above description is merely a preferred embodiment of the present invention and an explanation of the technical principles employed. Those skilled in the art should understand that the scope of disclosure in this invention is not limited to technical solutions formed by specific combinations of the above-described technical features, but should also cover other technical solutions formed by arbitrary combinations of the above-described technical features or their equivalents without departing from the above-disclosed concept. For example, technical solutions formed by substituting the above features with (but not limited to) technical features with similar functions disclosed in this invention.

[0125] It should be noted that the terms "first," "second," etc., used in the specification and claims of this application are used to distinguish similar objects and represent a limitation on a specific order or sequence. Where appropriate, the order of use for similar objects can be interchanged so that the embodiments of this application described herein can be implemented in an order other than that shown or described.

[0126] Those skilled in the art will recognize that this invention can be implemented as a system, method, or computer program product. Therefore, this invention can be specifically implemented in the following forms: it can be entirely hardware, entirely software (including firmware, resident software, microcode, etc.), or a combination of hardware and software, generally referred to herein as a "circuit," "module," or "system." Furthermore, in some embodiments, this invention can also be implemented as a computer program product contained in one or more computer-readable media, which includes computer-readable program code.

[0127] Although embodiments of the present invention have been shown and described above, it is understood that the above embodiments are exemplary and should not be construed as limiting the present invention. Those skilled in the art can make changes, modifications, substitutions and variations to the above embodiments within the scope of the present invention.

Claims

1. A method for training a vehicle insurance accident photo sorting model, characterized in that, include: Obtain multiple accident photos and corresponding supervision information for the same auto insurance case. The supervision information includes truth value grouping of the vehicle group to which the accident photos belong and scene-type photo annotations. The multiple accident photos are input into a visual language model to obtain the prediction results output by the visual language model. The prediction results include a list of predicted vehicle groups and a list of predicted scenes. Each predicted vehicle group contains a list of photo numbers belonging to the same accident vehicle. The gating coefficient is calculated based on the prediction result and the supervision information. When the prediction result cannot be parsed into legal structured data, the gating coefficient is set to zero. When the same photo number belongs to both the predicted vehicle group and the predicted scene list, the gating coefficient is set to a positive number less than 1. When the gating coefficient is not zero, multiple sub-scores are calculated based on the prediction results and the supervision information; The weighted sum of the multiple sub-scores is multiplied by the gating coefficient to obtain the total reward value. The parameters of the visual language model are then updated with the goal of maximizing the total reward value, so that the trained visual language model can be used to sort car insurance accident photos.

2. The method for training a vehicle insurance accident photo sorting model according to claim 1, characterized in that, Before inputting the multiple accident photos into the visual language model, the process also includes: The visual language model is fine-tuned under supervision using structured JSON that does not contain document-type fields as supervision labels. The supervision labels include the truth value groupings and the scene-type photo annotations.

3. The method for training a vehicle insurance accident photo sorting model according to claim 2, characterized in that, The steps for updating the parameters of the visual language model include: For multiple accident photos of the same car insurance case, at least two candidate prediction results are sampled, and the total reward value corresponding to each candidate prediction result is calculated. The parameters of the visual language model are updated after comparing the relative size of the total reward values ​​between the at least two candidate prediction results.

4. The method for training a vehicle insurance accident photo sorting model according to claim 1, characterized in that, The multiple sub-scores include format legality score, coverage legality score, attribution mutual exclusion score, paired grouping score, document type complement score, scene attribution score, vehicle identification number score, license plate score, and vehicle count score.

5. The method for training a vehicle insurance accident photo sorting model according to claim 4, characterized in that, The formula for calculating the total reward value is as follows: in, The total reward value, The gating coefficient is... Score the validity of the format. The score for the legality of the coverage. The score for the mutually exclusive affiliation is given. The paired group scores are given. The score is the complement of the document type. Assign a score to the scenario. The vehicle identification number is scored. Score the license plate. Score the number of vehicles. , , , , , , , , These are the weights corresponding to the scores of each sub-item, with the weight of the paired grouped scores being the highest.

6. The method for training a vehicle insurance accident photo sorting model according to claim 5, characterized in that, The steps for calculating the pairwise group scores include: Based on the true value grouping in the supervision information, construct a true value set of photos of the same vehicle; based on the predicted vehicle grouping in the prediction results, construct a predicted set of photos of the same vehicle; compare the degree of overlap between the true value set of photos of the same vehicle and the predicted set of photos of the same vehicle; determine the score of the paired grouping based on the degree of overlap, and the higher the degree of overlap, the higher the score of the paired grouping. In this context, each true vehicle photo pair in the set of true vehicle photo pairs consists of two photos belonging to the same vehicle group, and each predicted vehicle photo pair in the set of predicted vehicle photo pairs consists of two photos within the same predicted vehicle group.

7. The method for training a vehicle insurance accident photo sorting model according to claim 5, characterized in that, The steps for calculating the scene attribution score include: The consistency between the scene-type photo annotations in the supervision information and the predicted scene list is compared. The scene affiliation score is determined based on the consistency. Additional points are deducted when a photo is annotated as a scene-type photo in the supervision information and belongs to the predicted vehicle group. The scene affiliation score is obtained after the points are deducted. The steps for calculating the vehicle identification number score and the license plate score include: The predicted vehicle group is matched with the true vehicle group in the supervision information. The consistency of the vehicle identification number and license plate in the predicted vehicle group with the corresponding field in the supervision information is compared for each vehicle. The vehicle identification number score and the license plate score are determined according to the consistency. The score when the vehicle identification number or license plate in the predicted vehicle group is empty and the corresponding field in the supervision information is also empty is higher than the score when the vehicle identification number or license plate in the predicted vehicle group is not empty and the corresponding field in the supervision information is empty.

8. A training system for a vehicle insurance accident photo sorting model, characterized in that, include: The acquisition module is used to acquire multiple accident photos and corresponding supervision information for the same car insurance case. The supervision information includes the truth grouping of the vehicle group to which the accident photos belong and the scene-type photo annotation. The prediction module is used to input the multiple accident photos into the visual language model and obtain the prediction results output by the visual language model. The prediction results include a predicted vehicle group and a predicted scene list. Each predicted vehicle group contains a list of photo numbers belonging to the same accident vehicle. The processing module is used to calculate the gate coefficient based on the prediction result and the supervision information. When the prediction result cannot be parsed into legal structured data, the gate coefficient is set to zero. When there are photos with the same sequence number that belong to both the predicted vehicle group and the predicted scene list, the gate coefficient is set to a positive number less than 1. The calculation module is used to calculate multiple sub-scores based on the prediction result and the supervision information when the gating coefficient is not zero; The update module is used to multiply the weighted sum of the multiple sub-scores by the gating coefficient to obtain the total reward value, and update the parameters of the visual language model with the goal of maximizing the total reward value, so that the trained visual language model can be used to sort car insurance accident photos.

9. An electronic device, characterized in that, The electronic device includes a processor coupled to a memory, the memory storing at least one computer program, which is loaded and executed by the processor to enable the electronic device to implement the vehicle insurance accident photo sorting model training method as described in any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores at least one computer program, which, when executed by a processor, implements the vehicle insurance accident photo sorting model training method as described in any one of claims 1 to 7.

Images