Vehicle component detection method and device, terminal and storage medium

Abstract

The application provides a vehicle component detection method, device, terminal and storage medium. The method comprises: acquiring a vehicle component image to be detected of a vehicle; performing scene detection on the vehicle component image to be detected based on a preset orientation scene matching network to obtain a vehicle orientation scene corresponding to the vehicle component image to be detected; performing component detection on the vehicle component image to be detected based on a preset vehicle component detection model to obtain an initial component category detection result corresponding to the vehicle component image to be detected; and correcting the initial component category detection result according to the vehicle orientation scene to obtain a final component category detection result corresponding to the vehicle component image to be detected. The application can solve the problem of detection error or missed detection and false detection of similar features in left-right category and front-back category in vehicle multi-component detection, thereby improving the accuracy and effect of vehicle component detection, and helping to improve the intelligence and marketization degree of automobile business based on vehicle component detection.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of intelligent transportation, and in particular to a vehicle component detection method and device, a terminal and a storage medium. BACKGROUND

[0002] With the gradual increase of vehicles in today's society, vehicle component detection has become an important part of various automobile businesses. For example, in the automobile insurance claim business, after a vehicle collision accident occurs, a series of processes such as alarm, responsibility identification, insurance company personnel on-site investigation and photography are generally required to carry out insurance claim settlement. Among them, whether the vehicle pictures obtained by the loss assessor of the insurance company personally shooting pictures in the risk control environment or the vehicle pictures obtained by the user based on the intelligent loss settlement system need to identify the vehicle components, thereby providing an important basis for judging the damage condition of the vehicle and carrying out vehicle insurance claim settlement.

[0003] Vehicle component detection is the basis of various automobile businesses, and whether the vehicle components can be quickly identified and segmented in the image determines the intelligent degree and market application rate of related applications of various automobile businesses. At present, there are problems such as inaccurate data labeling, poor applicability, low accuracy, and poor robustness in the application of vehicle component detection. However, a relatively perfect vehicle component detection and segmentation method should be able to adapt to complex scenes such as low visibility at night, visual deformation of vehicle components caused by changes in the position of the shot, and blurred pictures caused by device shaking during shooting. Deep learning methods have been widely concerned by domestic and foreign researchers since they were proposed, and using deep learning methods to locate and detect the hardware dataset has the advantages of less influence of hyperparameters on results, stronger feature extraction ability and better anti-interference ability.

[0004] However, the current vehicle component detection method based on deep learning is a relatively simple application of target detection model and instance segmentation model, which can achieve good detection effect when the vehicle damage and component categories are few. However, when the component and damage categories increase and the features of different categories are extremely similar, it is easy to make detection errors or miss detection, false detection and other problems, the detection accuracy is not enough, and the vehicle component detection task cannot be completed well. SUMMARY

[0005] The embodiments of the present application provide a vehicle component detection method, device, terminal and storage medium to solve the problem that the current vehicle component detection method is prone to detection errors or miss detection, false detection, and has insufficient detection accuracy, and cannot complete the vehicle component detection task well.

[0006] In a first aspect, the embodiments of the present application provide a vehicle component detection method, comprising:

[0007] obtain a vehicle orientation scene corresponding to the vehicle part to be detected based on a preset orientation scene matching network;

[0008] obtain a vehicle orientation scene corresponding to the vehicle part to be detected based on a preset orientation scene matching network;

[0009] obtain an initial part class detection result corresponding to the vehicle part to be detected based on a preset vehicle part detection model;

[0010] correct the initial part class detection result according to the vehicle orientation scene, and obtain a final part class detection result corresponding to the vehicle part to be detected.

[0011] In a possible implementation manner, the initial part class detection result includes an initial part class and an initial part confidence score, and the final part class detection result includes a final part class and a final part confidence score.

[0012] The correcting the initial part class detection result according to the vehicle orientation scene, and obtaining a final part class detection result corresponding to the vehicle part to be detected, includes:

[0013] convert a target part class that does not match the standard part class in the initial part class according to the standard part class corresponding to the vehicle orientation scene, and obtain a converted and corrected part class;

[0014] record an initial part confidence score corresponding to the target part class as a target initial part confidence score, perform hierarchical screening on the target initial part confidence score according to a low-dimensional confidence threshold and a high-dimensional confidence threshold, and obtain a final part class and a final part confidence score of the target part class based on a screening result and the corrected part class.

[0015] In a possible implementation manner, the performing hierarchical screening on the target initial part confidence score according to a low-dimensional confidence threshold and a high-dimensional confidence threshold, and obtaining a final part class and a final part confidence score of the target part class based on a screening result and the corrected part class, includes:

[0016] determine whether the target initial part confidence score is greater than or equal to the low-dimensional confidence threshold;

[0017] if the target initial part confidence score is greater than or equal to the low-dimensional confidence threshold, adjust the target initial part confidence score according to a preset confidence adjustment parameter, and obtain a corrected part confidence score corresponding to the target part class;

[0018] determining whether the modified component confidence score is greater than or equal to the high-dimensional confidence threshold value;

[0019] if the modified component confidence score is greater than or equal to the high-dimensional confidence threshold value, obtaining a final component confidence score and a final component category of the target component category according to the modified component confidence score and the modified component category corresponding to the modified component confidence score.

[0020] In a possible implementation, after determining whether the target initial component confidence score is greater than or equal to the low-dimensional confidence threshold value, the method further includes:

[0021] if the target initial component confidence score is less than the low-dimensional confidence threshold value, discarding the target component category, the target initial component confidence score, and the modified component category corresponding to the target component category;

[0022] Alternatively, after determining whether the modified component confidence score is greater than or equal to the high-dimensional confidence threshold value, the method further includes:

[0023] if the modified component confidence score is less than the high-dimensional confidence threshold value, discarding the target component category, the target initial component confidence score, the modified component category corresponding to the target component category, and the modified component confidence score.

[0024] In a possible implementation, the method further includes:

[0025] obtaining the modified component confidence score corresponding to the target component category according to the target initial component confidence score and the preset confidence regulation parameter. scene = [(Scores + conf) | Labels scene ], Scores ≥ α, to obtain the modified component confidence score corresponding to the target component category;

[0026] wherein Scores scene is the modified component confidence score corresponding to the target component category, Scores is the target initial component confidence score, conf is the preset confidence regulation parameter, and the value range of conf is [0.1, 0.5], Labels scene is the modified component category converted from the target component category, and α is the low-dimensional confidence threshold value, and the value range of α is [0, 0.1], [A | B] represents that the occurrence of event A is on the premise of event B, and the result of event A is a target output.

[0027] In a possible implementation, the obtaining of the final part confidence score and the final part category of the target part category according to the modified part confidence score and the modified part category corresponding to the modified part confidence score comprises the following steps.

[0028] According to obtaining the final part confidence score and the final part category of the target part category;

[0029] wherein Scores final is the final part confidence score of the target part category, Scores scene is the modified part confidence score corresponding to the target part category, score_thr is the high-dimensional confidence threshold, Labels final is the final part category of the target part category, Labels scene is the modified part category converted from the target part category corresponding to the modified part confidence score, and [A|B] indicates that the occurrence of event A is on the premise of event B, wherein the result of event A is a target output.

[0030] In a possible implementation, the training process of the preset orientation scene matching network comprises the following steps.

[0031] obtaining a training set constituted by vehicle orientation scene graphs, wherein each vehicle orientation scene graph is a vehicle graph captured from a certain orientation of a vehicle;

[0032] training an initial orientation scene matching network according to each vehicle orientation scene graph in the training set and a vehicle orientation scene label corresponding to the vehicle orientation scene graph, to obtain a preset orientation scene matching network.

[0033] In a second aspect, an embodiment of the present application provides a vehicle part detection device, comprising:

[0034] an input module configured to obtain a vehicle part graph to be detected of a vehicle;

[0035] a first processing module configured to perform scene detection on the vehicle part graph to be detected based on a preset orientation scene matching network, to obtain a vehicle orientation scene corresponding to the vehicle part graph to be detected;

[0036] a second processing module configured to perform part detection on the vehicle part graph to be detected based on a preset vehicle part detection model, to obtain an initial part category detection result corresponding to the vehicle part graph to be detected;

[0037] a third processing module configured to correct the initial part category detection result according to the vehicle orientation scene, to obtain a final part category detection result corresponding to the vehicle part graph to be detected.

[0038] In a third aspect, an embodiment of the present application provides a terminal, comprising a memory and a processor, the memory is configured to store a computer program, and the processor is configured to invoke and run the computer program stored in the memory to execute the steps of the method according to the first aspect or any possible implementation manner of the first aspect.

[0039] In a fourth aspect, an embodiment of the present application provides a computer readable storage medium, which stores a computer program, and the computer program is executed by a processor to implement the steps of the method according to the first aspect or any possible implementation manner of the first aspect.

[0040] The embodiment of the present application provides a vehicle part detection method, device, terminal and storage medium. The vehicle part detection method comprises the following steps: acquiring a to-be-detected vehicle part image of a vehicle; performing scene detection on the to-be-detected vehicle part image based on a preset orientation scene matching network to obtain a vehicle orientation scene corresponding to the to-be-detected vehicle part image; performing part detection on the to-be-detected vehicle part image based on a preset vehicle part detection model to obtain an initial part category detection result corresponding to the to-be-detected vehicle part image; and correcting the initial part category detection result according to the vehicle orientation scene to obtain a final part category detection result corresponding to the to-be-detected vehicle part image. Since the concept of the vehicle orientation scene is introduced in the embodiment, the accurate orientation scene of the to-be-detected vehicle part image can be obtained based on the preset orientation scene matching network. On this basis, the initial part category detection result corresponding to the to-be-detected vehicle part image is corrected according to the accurate orientation scene of the to-be-detected vehicle part image, so that the problem of detection error or missed detection of similar features in left and right categories and front and back categories in vehicle multi-part detection can be solved, so that the task of vehicle part detection can be better completed, the accuracy and effect of vehicle part detection are improved, and the intelligentization and marketization degree of intelligent loss assessment, risk control and other automobile businesses based on vehicle part detection are improved. BRIEF DESCRIPTION OF DRAWINGS

[0041] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings needed in the embodiments or prior art description will be briefly introduced. Obviously, the drawings in the following description are only some embodiments of the present application, and other drawings can be obtained by those skilled in the art without creative labor.

[0042] Figure 1 is the implementation flowchart of the vehicle part detection method provided by the embodiment of the present application;

[0043] Figure 2 is the 8+1 orientation scene schematic diagram of the vehicle part provided by the embodiment of the present application;

[0044] Figure 3 is a general network architecture diagram of a vehicle component detection method provided by an embodiment of the present application;

[0045] Figure 4 is a schematic diagram of converting a target component category provided by an embodiment of the present application;

[0046] Figure 5 is a schematic diagram of an initial component category detection result provided by an embodiment of the present application;

[0047] Figure 6 is a schematic diagram of a final component category detection result provided by an embodiment of the present application;

[0048] Figure 7 is a structural schematic diagram of a vehicle component detection device provided by an embodiment of the present application;

[0049] Figure 8 is a schematic diagram of a terminal provided by an embodiment of the present application. DETAILED DESCRIPTION

[0050] In the following description, specific details are set forth in order to provide a thorough understanding of embodiments of the application. However, persons of ordinary skill in the art will readily recognize that embodiments of the application can be practiced without these specific details. In other instances, well-known structures, devices, circuits, and processes have not been described in detail so as not to unnecessarily obscure aspects of the application.

[0051] In order to make the objectives, technical solutions and advantages of the present application clearer, the following will be described by specific embodiments in conjunction with the accompanying drawings.

[0052] Referring to Figure 1 which shows an implementation flowchart of a vehicle component detection method provided by an embodiment of the present application, and is described in detail as follows:

[0053] In step 101, a vehicle component to be detected image of a vehicle is acquired.

[0054] The vehicle component to be detected image can be a vehicle picture acquired by a loss adjuster in a risk control environment, can be a vehicle picture uploaded by a user in intelligent loss adjustment, or can be a vehicle picture acquired in other automobile business processes, such as a vehicle picture acquired when taking and returning a vehicle in vehicle rental, etc. The present embodiment does not limit the acquisition manner of the vehicle component to be detected image, and as long as a scene requiring vehicle component detection can apply the vehicle component detection method provided by the present embodiment.

[0055] In step 102, a scene detection is performed on the vehicle component to be detected image based on a preset orientation scene matching network, to obtain a vehicle orientation scene corresponding to the vehicle component to be detected image.

[0056] In this embodiment, the preset orientation scene matching network is a network that can distinguish vehicle pictures that are acquired relative to which orientation scene of the vehicle. Since the vehicle parts contained in the vehicle pictures taken from different orientation scenes of the vehicle are different. For example, when a vehicle picture is taken from an orientation scene of the right front or right side of the vehicle, the vehicle picture is more likely to contain a rearview mirror (right), a front door (right), etc., and when a vehicle picture is taken from an orientation scene of the left front or left side of the vehicle, the vehicle picture is more likely to contain a rearview mirror (left), a front door (left). Therefore, the orientation scene rule of the vehicle parts is defined to perform scene detection on the vehicle part picture to be detected based on the preset orientation scene matching network to obtain the vehicle orientation scene corresponding to the vehicle part picture to be detected, which is helpful to accurately detect the left and right categories, front and back categories, etc. of similar features in vehicle part detection.

[0057] For example, as shown in FIG. 8, 8+1 orientation scenes of vehicle parts can be defined. Among them, Scene1-Scene8 (i.e., Scene1-Scene8) are orientation scenes relative to the front, right front, right, right rear, rear, left rear, left, and left front of the vehicle, and the remaining one is the background. Defining 8+1 orientation scenes of vehicle parts determines the standard vehicle part set corresponding to each orientation scene, thereby helping to improve the accuracy and effect of subsequent vehicle part detection. Figure 2 Among them, defining 8+1 orientation scenes of vehicle parts helps to quickly and accurately obtain the vehicle orientation scene corresponding to the vehicle part picture to be detected. On this basis, in order to improve the scene detection speed or accuracy, fewer or more orientation scenes can also be defined.

[0058] Optionally, the training process of the preset orientation scene matching network can include:

[0059] A training set composed of vehicle orientation scene pictures is obtained, and the vehicle orientation scene picture is a vehicle picture taken from a certain orientation of the vehicle.

[0060] According to each vehicle orientation scene picture in the training set and the vehicle orientation scene label corresponding to the vehicle orientation scene picture, the initial orientation scene matching network is trained to obtain the preset orientation scene matching network.

[0061]

[0062] ​Exemplarily, 59 types of vehicle parts can be selected as experimental objects, and a vehicle part small data set Part_data1 required for training the preset orientation scene matching network can be constructed by referring to the construction method of the COCO data set. In the Part_data1 data set, the sample images of the training set and the test set can be set to 39058 and 4335 respectively, and the quantity ratio is 9:1, which contains 8 types of vehicle orientation scenes and one type of background, a total of nine types. The vehicle orientation scene graph is similar to the vehicle part graph, but it is not a mutual inclusion relationship. Part of them is cross, but most of them are independent of each other. The vehicle pictures taken from the perspective of long shot, medium shot and close-up can be selected as the vehicle orientation scene graph.

[0063] In the training process, the parameters can be set as follows: the learning rate lr is 0.0001, the batch size is 16, and the epoch is set to 10. The preset orientation scene matching network trained by the embodiment can better identify the corresponding orientation scene of the vehicle, and provides a certain basis for vehicle part detection based on the vehicle orientation scene.

[0064] In step 103, the part detection of the vehicle part graph to be detected is performed based on the preset vehicle part detection model, and the initial part category detection result corresponding to the vehicle part graph to be detected is obtained.

[0065] In the training process of the preset vehicle part detection model, the training process of the preset vehicle part detection model is similar to the training process of the preset orientation scene matching network. Exemplarily, 59 types of vehicle parts can be selected as experimental objects, and a vehicle part large data set Part_data2 required for training the preset vehicle part detection model can be constructed by referring to the construction method of the COCO data set. In the Part_data2 data set, the sample images of the training set and the test set can be set to 45503 and 11376 respectively, and the quantity ratio is 3:1, which contains 287331 and 71712 category objects respectively, a total of 59 categories. In the Part_data2 data set, the vehicle parts Figure 1 are generally taken by loss assessors in risk control scenes, involving three perspectives of long shot, medium shot and close-up, so the Part_data2 data set is more massive.

[0066] Exemplarily, a Maskrcnn model can be used as an initial vehicle component detection model for training. During the model training process, an NVIDIA 1080Ti professional acceleration card can be used for training and testing, an Ubuntu 16.04.6 LTS operating system is used, and CUDA 11.1 is used for accelerating training. The computer language used can be Python 3.8, and the network development framework can be Pytorch. During the training stage, the batchsize can be set to 1, and the stochastic gradient descent (SGD) algorithm is used, the initial learning rate can be set to 0.001, the learning rate can be reduced to 10% of the original after 9 epochs, and the maximum iteration training is 30 epochs. By using GPUs, this method can be easily extended to a distributed system.

[0067] In step 104, the initial component class detection result is corrected according to the vehicle orientation scene to obtain a final component class detection result corresponding to the to-be-detected vehicle component image.

[0068] In this embodiment, the preset vehicle component detection model in step 103 is used for component detection, which can achieve certain detection effect. However, there are many vehicle component categories, for example, 59 component categories, and the front and rear categories and the left and right categories are distinguished, such as'rearview mirror (right)' (Outer_mirror (right)),'rearview mirror (left)' (Outer_mirror (left)),'rear door glass (right)' (Rear_door_glass (right)), 'front door glass (right)' (Front_door_glass (right)) and the like. The characteristics of the categories are extremely similar, which can easily lead to misjudgment categories. It should be noted that the English names of each category are used during training and detection.

[0069] However, the deep learning framework is difficult to solve the misjudgment problem caused by the similar characteristics between different categories, so the vehicle component detection method provided in this embodiment uses a multifunctional module: a scene understanding module. For example, Figure 3As shown, first, the vehicle part graph to be detected is input into a preset orientation scene matching network similar to Resnet101 for feature extraction, and after passing through a fully connected layer, scene detection is performed; at the same time, the vehicle part graph to be detected is subjected to feature extraction through Backbone and FPN, and then a base feature graph (Base Feature) is obtained using a region proposal network (RPN) and a region of interest pooling unit (RoI Pooling), and the base feature graph is imported into a series of fully connected layers to generate part labels (Labels), part confidence scores (Scores), target frame positioning and mask, as the initial part class detection result corresponding to the vehicle part graph to be detected. Finally, the vehicle orientation scene detected by the scene understanding module is fused with the part class, part confidence score, target frame positioning and mask obtained by part detection, so as to correct the initial part class detection result according to the vehicle orientation scene, thereby obtaining the final part class detection result which can improve the detection effect of the vehicle part.

[0070] Optionally, the initial part class detection result includes an initial part class and an initial part confidence score, and the final part class detection result includes a final part class and a final part confidence score.

[0071] According to the vehicle orientation scene, the initial part class detection result is corrected to obtain the final part class detection result corresponding to the vehicle part graph to be detected, which can include:

[0072] According to the standard part class corresponding to the vehicle orientation scene, the target part class in the initial part class which does not match the standard part class is converted to obtain a converted modified part class.

[0073] The initial part confidence score corresponding to the target part class is recorded as a target initial part confidence score, the target initial part confidence score is classified and screened according to a low-dimensional confidence threshold and a high-dimensional confidence threshold, and based on the screening result and the modified part class, the final part class and the final part confidence score of the target part class are obtained.

[0074] In this embodiment, the scene understanding module performs part replacement under the scene driving on the Labels corresponding to the vehicle orientation scene which are detected by the preset vehicle part detection model and do not match the standard part class (that is, the target part class in the initial part class which does not match the standard part class is converted), as shown in the following formula:

[0075] Labels scene = scene[Labels];

[0076] wherein scene[.] represents the introduction of the vehicle orientation scene. By way of example, component replacement under Scene 2 (Scene2) is shown as Figure 4 To improve the accuracy of the modified component class after conversion, the initial component confidence score corresponding to the target component class for component replacement is further graded and screened according to the low-dimensional confidence threshold and the high-dimensional confidence threshold, so as to determine the final component class and the final component confidence score of the target component class.

[0077] Optionally, the grading and screening of the target initial component confidence score according to the low-dimensional confidence threshold and the high-dimensional confidence threshold, and obtaining the final component class and the final component confidence score of the target component class based on the screening result and the modified component class, can include:

[0078] determining whether the target initial component confidence score is greater than or equal to the low-dimensional confidence threshold.

[0079] If the target initial component confidence score is greater than or equal to the low-dimensional confidence threshold, the target initial component confidence score is regulated according to the pre-set confidence regulation parameter to obtain the modified component confidence score corresponding to the target component class.

[0080] determining whether the modified component confidence score is greater than or equal to the high-dimensional confidence threshold.

[0081] If the modified component confidence score is greater than or equal to the high-dimensional confidence threshold, the final component confidence score and the final component class of the target component class are obtained according to the modified component confidence score and the modified component class corresponding to the modified component confidence score.

[0082] Optionally, the regulation of the target initial component confidence score according to the pre-set confidence regulation parameter to obtain the modified component confidence score corresponding to the target component class can include:

[0083] obtaining the modified component confidence score corresponding to the target component class according to Scores scene = [(Scores + conf) | Labels scene ], Scores ≥ a.

[0084] wherein Scores scene is the modified component confidence score corresponding to the target component class, Scores is the target initial component confidence score, conf is the pre-set confidence regulation parameter, and the value range is [0.1, 0.5], and Labels sceneThe corrected component category is the target component category after conversion. α is the low-dimensional confidence threshold, with a value range of [0, 0.1]. [A|B] indicates that event A occurs under the premise of event B, where the result of event A is the target output.

[0085] Optionally, the final component confidence score and final component category of the target component category are obtained based on the corrected component confidence score and the corrected component category corresponding to the corrected component confidence score, including:

[0086] according to Obtain the final part confidence score and final part category for the target part category.

[0087] Among them, Scores final Scores are the final part confidence scores for the target part category. scene The corrected component confidence score corresponds to the target component category, where score_thr is the high-dimensional confidence threshold, and Labels are... final Labels are the final part categories for the target part category. scene To correct the target component category corresponding to the confidence score of the corrected component, the corrected component category is [A|B], which means that event A occurs under the premise of event B, where the result of event A is the target output.

[0088] In this embodiment, the scene understanding module adjusts the confidence level of the scores corresponding to Labels that do not match the standard component categories corresponding to the vehicle's orientation scene, detected by the preset vehicle component detection model. First, based on the low-dimensional confidence threshold α (i.e., the first-level low-dimensional parameter of the hierarchical target selection mechanism), according to the scores... scene =[(Scores+conf)|Labels scene The first level of the hierarchical screening mechanism, where Scores ≥ α, performs low-dimensional screening to select target initial component confidence scores (Scores) that are greater than or equal to the low-dimensional confidence threshold α. These initial component confidence scores are then adjusted using a pre-set confidence adjustment parameter (conf). The pre-set confidence adjustment parameter (conf) can be adaptively adjusted based on the target initial component confidence scores; when the target initial component confidence scores are low, the pre-set confidence adjustment parameter can be appropriately increased to achieve a balanced confidence score adjustment.

[0089] Based on this, after the first level of low-dimensional screening, a second level of high-dimensional screening is required. The specific steps can be determined based on Scores. final =Scores scene Scores sceneThe second level of the hierarchical screening mechanism, ≥score_thr, selects corrected component confidence scores (Scores) that are greater than or equal to the high-dimensional confidence threshold score_thr. scene In correcting the component confidence scores scene If the score is greater than or equal to the high-dimensional confidence threshold score_thr, then adjust the component confidence score. scene As the final component confidence score, the component confidence score will be adjusted. scene The corresponding target component category converted and modified component category Labels scene As the final component category.

[0090] Based on this, the scene understanding module can correct the target box localization and the corresponding target box mask in the initial component category detection results by modifying the initial component category and initial component confidence score.

[0091] Optionally, after determining whether the initial confidence score of the target component is greater than or equal to the low-dimensional confidence threshold, the following may also be included:

[0092] If the confidence score of the initial target component is less than the low-dimensional confidence threshold, then the target component category, the initial target component confidence score, and the corrected component category corresponding to the target component category are removed.

[0093] Alternatively, after determining whether the confidence score of the corrected component is greater than or equal to the high-dimensional confidence threshold, it may also include:

[0094] If the confidence score of the corrected component is less than the high-dimensional confidence threshold, then the target component category, the confidence score of the initial target component, the corrected component category corresponding to the target component category, and the corrected component confidence score are removed.

[0095] In this embodiment, cases where the initial confidence score of the target component is less than the low-dimensional confidence threshold, or cases where the confidence score of the corrected component is less than the high-dimensional confidence threshold, are eliminated to improve the accuracy of component detection.

[0096] Combination Figure 5 and Figure 6 As shown, because the scene understanding module in this embodiment fuses the vehicle orientation scene with the initial component category detection results, the misclassification problem of left / right and front / back categories caused by feature similarity between multiple categories is alleviated. Simultaneously, the confidence adjustment and hierarchical screening target mechanism in the scene understanding module can appropriately increase the confidence of component categories with originally low confidence, thereby exceeding the confidence threshold and being detected, thus mitigating the missed detection problem. For example, regarding the missed detection problem, in Figure 5The Front_window_glass, Bottom_edge(right), and Fog_lamp(right) that were missed during the process are... Figure 6 All were detected. Regarding false detections, Outer_mirror(left) was corrected to Outer_mirror(right), Car_left_door was corrected to Car_right_door, Front_fender(left) was corrected to Front_fender(right), and Head_lamp(left) was corrected to Head_lamp(right). This demonstrates that the vehicle component detection method provided in this embodiment achieves good detection results for multi-component vehicle detection, and the problems of missed detections and false detections are resolved to a certain extent.

[0097] This invention involves acquiring an image of a vehicle component to be inspected; firstly, scene detection is performed on the image based on a preset orientation scene matching network to obtain the corresponding vehicle orientation scene; then, component detection is performed on the image based on a preset vehicle component detection model to obtain the initial component category detection result; finally, the initial component category detection result is corrected based on the vehicle orientation scene to obtain the final component category detection result. Because this embodiment introduces the concept of a vehicle orientation scene, the accurate orientation scene of the image can be obtained first based on the preset orientation scene matching network. Based on this, the initial component category detection result is corrected through component category conversion, confidence adjustment, and a hierarchical target selection mechanism under the orientation scene. This solves the problem of incorrect or missed detections in the detection of left-right or front-rear categories of similar features when detecting multiple vehicle components, better completing the vehicle component detection task, improving the accuracy and effectiveness of vehicle component detection, and thus contributing to the improvement of the intelligence and marketization of automotive businesses such as intelligent damage assessment and risk control based on vehicle component detection.

[0098] It should be understood that the sequence number of each step in the above embodiments does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present invention.

[0099] The following are device embodiments of the present invention. For details not described in detail, please refer to the corresponding method embodiments described above.

[0100] Figure 7 A schematic diagram of the vehicle component detection device provided in an embodiment of the present invention is shown. For ease of explanation, only the parts related to the embodiment of the present invention are shown, and are described in detail below:

[0101] As shown in Figure 7 , the vehicle component detection device comprises an input module 71, a first processing module 72, a second processing module 73 and a third processing module 74.

[0102] The input module 71 is configured to obtain a vehicle component image to be detected of a vehicle;

[0103] The first processing module 72 is configured to perform scene detection on the vehicle component image to be detected based on a preset orientation scene matching network, to obtain a vehicle orientation scene corresponding to the vehicle component image to be detected;

[0104] The second processing module 73 is configured to perform component detection on the vehicle component image to be detected based on a preset vehicle component detection model, to obtain an initial component category detection result corresponding to the vehicle component image to be detected;

[0105] The third processing module 74 is configured to correct the initial component category detection result according to the vehicle orientation scene, to obtain a final component category detection result corresponding to the vehicle component image to be detected.

[0106] The embodiment of the present application obtains a vehicle component image to be detected of a vehicle; and first performs scene detection on the vehicle component image to be detected based on a preset orientation scene matching network, to obtain a vehicle orientation scene corresponding to the vehicle component image to be detected; then performs component detection on the vehicle component image to be detected based on a preset vehicle component detection model, to obtain an initial component category detection result corresponding to the vehicle component image to be detected; and finally corrects the initial component category detection result according to the vehicle orientation scene, to obtain a final component category detection result corresponding to the vehicle component image to be detected. Since the embodiment introduces the concept of vehicle orientation scene, the accurate orientation scene of the vehicle component image to be detected can be obtained based on the preset orientation scene matching network, and on this basis, the initial component category detection result corresponding to the vehicle component image to be detected is corrected through component category conversion under the orientation scene, confidence regulation and hierarchical screening target mechanism, so that the problems of detection error or missed detection and false detection of similar features in left and right categories and front and back categories in vehicle multi-component detection can be solved, the task of vehicle component detection can be better completed, the accuracy and effect of vehicle component detection are improved, and the intelligentization and marketization degree of intelligent loss assessment, risk control and other automobile businesses based on vehicle component detection are improved.

[0107] In a possible implementation, the initial component category detection result includes an initial component category and an initial component confidence score, and the final component category detection result includes a final component category and a final component confidence score; the third processing module 74 can be configured to convert a target component category that does not match a standard component category corresponding to the vehicle orientation scene, from the initial component category, to obtain a modified component category after conversion;

[0108] The initial component confidence score corresponding to the target component category is denoted as a target initial component confidence score, the target initial component confidence score is classified and filtered according to a low-dimensional confidence threshold and a high-dimensional confidence threshold, and based on a filtering result and the modified component category, a final component category and a final component confidence score of the target component category are obtained.

[0109] In a possible implementation, the third processing module 74 can be configured to determine whether the target initial component confidence score is greater than or equal to the low-dimensional confidence threshold;

[0110] If the target initial component confidence score is greater than or equal to the low-dimensional confidence threshold, the target initial component confidence score is regulated according to a preset confidence regulation parameter, to obtain a modified component confidence score corresponding to the target component category;

[0111] It is determined whether the modified component confidence score is greater than or equal to the high-dimensional confidence threshold;

[0112] If the modified component confidence score is greater than or equal to the high-dimensional confidence threshold, a final component confidence score and a final component category of the target component category are obtained according to the modified component confidence score and the modified component category corresponding to the modified component confidence score.

[0113] In a possible implementation, the third processing module 74 can be further configured to, if the target initial component confidence score is less than the low-dimensional confidence threshold, eliminate the target component category, the target initial component confidence score, and a modified component category corresponding to the target component category;

[0114] Alternatively, if the modified component confidence score is less than the high-dimensional confidence threshold, the target component category, the target initial component confidence score, the modified component category corresponding to the target component category, and the modified component confidence score are eliminated.

[0115] In a possible implementation, the third processing module 74 can be configured to obtain the final component category and the final component confidence score of the target component category according to Scores scene = [(Scores + conf) | Labels sceneIf Scores≥α, obtain the corrected component confidence score corresponding to the target component category;

[0116] Among them, Scores scene The corrected component confidence score corresponds to the target component category, Scores is the initial target component confidence score, conf is the preset confidence adjustment parameter with a value range of [0.1, 0.5], and Labels is the component confidence score. scene The target component category is the corrected component category after conversion. α is the low-dimensional confidence threshold, with a value range of [0, 0.1]. [A|B] indicates that event A occurs under the premise of event B, where the result of event A is the target output.

[0117] In one possible implementation, the third processing module 74 can be used to... Obtain the final component confidence score and final component category for the target component category;

[0118] Among them, Scores final The final component confidence score for the target component category, Scores scene The corrected component confidence score is the score corresponding to the target component category, where score_thr is the high-dimensional confidence threshold, and Labels are... final Labels are the final component categories for the target component category. scene The corrected component category is the target component category converted to the confidence score of the corrected component. [A|B] indicates that event A occurs under the premise of event B, where the result of event A is the target output.

[0119] In one possible implementation, the training process of the pre-defined orientation scene matching network includes:

[0120] A training set consisting of vehicle orientation scene images is obtained, wherein the vehicle orientation scene images are vehicle images taken from a certain position of the vehicle.

[0121] Based on each vehicle orientation scene image in the training set and the corresponding vehicle orientation scene label, the initial orientation scene matching network is trained to obtain the preset orientation scene matching network.

[0122] Figure 8 This is a schematic diagram of a terminal provided in an embodiment of the present invention. Figure 8 As shown, the terminal 8 in this embodiment includes a processor 80, a memory 81, and a computer program 82 stored in the memory 81 and executable on the processor 80. When the processor 80 executes the computer program 82, it implements the steps in the various vehicle component detection method embodiments described above, for example... Figure 1The steps 101-104 are shown. Alternatively, the processor 80 implements the functions of the modules / units in the above-mentioned device embodiments when executing the computer program 82, for example Figure 7 The functions of the modules / units 71-74 are shown.

[0123] The computer program 82 can be segmented into one or more modules / units, which are stored in the memory 81 and executed by the processor 80 to complete the present application. One or more modules / units can be a series of computer program instruction segments capable of completing a specific function, which are used to describe the execution process of the computer program 82 in the terminal 8. For example, the computer program 82 can be segmented into Figure 7 The modules / units 71-74 are shown.

[0124] The terminal 8 can be a desktop computer, a notebook, a palm computer, a cloud server, and the like. The terminal 8 can include, but is not limited to, the processor 80 and the memory 81. Those skilled in the art can understand that Figure 8 The terminal 8 is only an example and does not constitute a limitation on the terminal 8, which can include more or fewer components than shown, or combine certain components, or different components, for example, the terminal can also include an input / output device, a network access device, a bus, and the like.

[0125] The processor 80 can be a central processing unit (CPU), and can also be other general-purpose processors, digital signal processors (DSP), application specific integrated circuits (ASIC), field programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, or the like. The general-purpose processor can be a microprocessor or the processor can also be any conventional processor.

[0126] The memory 81 can be an internal storage unit of the terminal 8, such as a hard disk or a memory of the terminal 8. The memory 81 can also be an external storage device of the terminal 8, such as a plug-in hard disk, a smart media card (SMC), a secure digital (SD) card, a flash card, or the like equipped on the terminal 8. Further, the memory 81 can also include both the internal storage unit and the external storage device of the terminal 8. The memory 81 is used to store computer programs and other programs and data required by the terminal. The memory 81 can also be used to temporarily store data that has been output or will be output.

[0127] It can be clearly understood by those skilled in the art that, for the convenience and brevity of description, only the above-mentioned division of each functional unit and module is exemplified, and in actual application, the above-mentioned functions can be completed by different functional units and modules according to needs, that is, the internal structure of the device is divided into different functional units or modules to complete all or part of the functions described above. Each functional unit and module in the embodiment can be integrated in one processing unit, or each unit can be physically present separately, or two or more units can be integrated in one unit. The integrated unit can be realized in the form of hardware or in the form of a software functional unit. In addition, the specific names of each functional unit and module are only for the convenience of mutual distinction, and do not limit the protection scope of the present application. The specific working process of the units and modules in the above system can refer to the corresponding process in the foregoing method embodiments, which will not be described here.

[0128] In the above embodiments, the description of each embodiment has its own emphasis, and the parts not described or recorded in detail in a certain embodiment can be referred to the relevant description of other embodiments.

[0129] Those of ordinary skill in the art can appreciate that the units and algorithm steps of each example described in combination with the embodiments disclosed herein can be realized in electronic hardware or a combination of computer software and electronic hardware. Whether the functions are executed in hardware or software depends on the specific application and design constraints of the technical solution. A person skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of the present application.

[0130] In the embodiments of the present application, it should be understood that the disclosed apparatus / terminal and method can be implemented in other manners. For example, the embodiments of the apparatus / terminal described above are merely schematic, and the division of the modules or units can be different, for example, a plurality of units or components can be combined or integrated into another system, or some features can be ignored or not executed. In addition, the displayed or discussed mutual couplings or direct couplings or communication connections can be indirect couplings or communication connections through some interfaces, devices or units, and can be in electrical, mechanical or other forms.

[0131] The units described as separate components can or can not be physically separate, and the components displayed as units can or can not be physical units, i.e., can be located in one place, or can be distributed on a plurality of network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the embodiments.

[0132] In addition, each functional unit in the various embodiments of the present application can be integrated into a processing unit, or each unit can be a physically independent unit, or two or more units can be integrated into a unit. The integrated unit can be implemented in the form of hardware or in the form of a software functional unit.

[0133] If the integrated module / unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer readable storage medium. Based on this understanding, all or part of the flow of the above-mentioned embodiment methods can be implemented by a computer program instructing related hardware to complete, and the computer program can be stored in a computer readable storage medium. When the processor executes the computer program, the steps of each vehicle component detection method embodiment described above can be implemented. The computer program includes computer program code, which can be in the form of source code, object code, executable file or some intermediate form. The computer readable medium can include any entity or device capable of carrying the computer program code, recording medium, U disk, mobile hard disk, magnetic disk, optical disk, computer memory, read-only memory (ROM), random access memory (RAM), electrical carrier signal, telecommunication signal and software distribution medium, etc.

[0134] The above examples are only used to illustrate the technical solutions of the present application, but not limit the present application; although the present application has been described in detail with reference to the foregoing examples, those ordinarily skilled in the art should understand: the technical solutions recorded in the foregoing examples can still be modified, or some technical features can be replaced equivalently; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present application, and should be included in the protection scope of the present application.

Claims

1. A method for inspecting vehicle components, characterized in that, include: Obtain images of the vehicle components to be inspected; Based on a preset orientation scene matching network, scene detection is performed on the vehicle component image to be detected to obtain the vehicle orientation scene corresponding to the vehicle component image to be detected. Based on a preset vehicle component detection model, component detection is performed on the vehicle component image to be detected to obtain the initial component category detection result corresponding to the vehicle component image to be detected. The initial component category detection result is corrected based on the vehicle orientation scene to obtain the final component category detection result corresponding to the vehicle component image to be detected; The initial component category detection result includes the initial component category and the initial component confidence score, and the final component category detection result includes the final component category and the final component confidence score; The step of correcting the initial component category detection result based on the vehicle orientation scene to obtain the final component category detection result corresponding to the vehicle component image to be detected includes: Based on the standard component category corresponding to the vehicle orientation scene, the target component category that does not match the standard component category in the initial component category is converted to obtain the converted corrected component category; The initial component confidence score corresponding to the target component category is recorded as the target initial component confidence score. The target initial component confidence score is then graded and filtered according to the low-dimensional confidence threshold and the high-dimensional confidence threshold. Based on the filtering results and the corrected component category, the final component category and the final component confidence score of the target component category are obtained. The step of classifying and filtering the initial confidence scores of the target components based on low-dimensional and high-dimensional confidence thresholds, and obtaining the final component category and final component confidence score based on the filtering results and the corrected component category, includes: Determine whether the confidence score of the target initial component is greater than or equal to the low-dimensional confidence threshold; If the confidence score of the target initial component is greater than or equal to the low-dimensional confidence threshold, the confidence score of the target initial component is adjusted according to the preset confidence adjustment parameters to obtain the corrected component confidence score corresponding to the target component category; Determine whether the confidence score of the corrected component is greater than or equal to the high-dimensional confidence threshold; If the confidence score of the corrected component is greater than or equal to the high-dimensional confidence threshold, the final component confidence score and final component category of the target component category are obtained based on the confidence score of the corrected component and the corrected component category corresponding to the confidence score of the corrected component.

2. The vehicle component testing method according to claim 1, characterized in that, After determining whether the confidence score of the target initial component is greater than or equal to the low-dimensional confidence threshold, the method further includes: If the confidence score of the target initial component is less than the low-dimensional confidence threshold, then the target component category, the target initial component confidence score, and the modified component category corresponding to the target component category are removed. Alternatively, after determining whether the confidence score of the corrected component is greater than or equal to the high-dimensional confidence threshold, the method further includes: If the confidence score of the corrected component is less than the high-dimensional confidence threshold, then the target component category, the confidence score of the initial target component, the corrected component category corresponding to the target component category, and the corrected component confidence score are removed.

3. The vehicle component testing method according to claim 1, characterized in that, The step of adjusting the initial confidence score of the target component according to a preset confidence adjustment parameter to obtain the corrected component confidence score corresponding to the target component category includes: according to Obtain the corrected component confidence score corresponding to the target component category; in, The corrected component confidence score corresponding to the target component category. The initial confidence score for the target component. The preset reliability adjustment parameter has a value range of [range missing]. , The corrected component category is the one converted from the target component category. The low-dimensional confidence threshold is defined as having a value range of [value range missing]. , express The incident occurred in Given the context of the event, among which, The result of the event is the target output.

4. The vehicle component testing method according to claim 1, characterized in that, The step of obtaining the final component confidence score and final component category of the target component category based on the modified component confidence score and the modified component category corresponding to the modified component confidence score includes: according to The final component confidence score and final component category of the target component category are obtained. in, The final component confidence score for the target component category. The corrected component confidence score corresponding to the target component category. The high-dimensional confidence threshold is... The final component category of the target component category. The corrected component category is the target component category converted to the corrected component confidence score. express The incident occurred in Given the context of the event, among which, The result of the event is the target output.

5. The vehicle component testing method according to any one of claims 1-4, characterized in that, The training process of the preset orientation scene matching network includes: A training set consisting of vehicle orientation scene images is obtained, wherein the vehicle orientation scene images are vehicle images taken from a certain position of the vehicle. Based on each vehicle orientation scene image in the training set and the corresponding vehicle orientation scene label, the initial orientation scene matching network is trained to obtain the preset orientation scene matching network.

6. A vehicle component testing device, characterized in that, include: The input module is used to obtain images of the vehicle components to be inspected. The first processing module is used to perform scene detection on the vehicle component image to be detected based on a preset orientation scene matching network, and obtain the vehicle orientation scene corresponding to the vehicle component image to be detected. The second processing module is used to perform component detection on the vehicle component image to be detected based on a preset vehicle component detection model, and obtain the initial component category detection result corresponding to the vehicle component image to be detected. The third processing module is used to correct the initial component category detection result according to the vehicle orientation scene, and obtain the final component category detection result corresponding to the vehicle component image to be detected. The initial component category detection result includes the initial component category and the initial component confidence score, and the final component category detection result includes the final component category and the final component confidence score; The third processing module is specifically used for: Based on the standard component category corresponding to the vehicle orientation scene, the target component category that does not match the standard component category in the initial component category is converted to obtain the converted corrected component category; The initial component confidence score corresponding to the target component category is recorded as the target initial component confidence score. The target initial component confidence score is then graded and filtered according to the low-dimensional confidence threshold and the high-dimensional confidence threshold. Based on the filtering results and the corrected component category, the final component category and the final component confidence score of the target component category are obtained. The third processing module is specifically used for: Determine whether the confidence score of the target initial component is greater than or equal to the low-dimensional confidence threshold; If the confidence score of the target initial component is greater than or equal to the low-dimensional confidence threshold, the confidence score of the target initial component is adjusted according to the preset confidence adjustment parameters to obtain the corrected component confidence score corresponding to the target component category; Determine whether the confidence score of the corrected component is greater than or equal to the high-dimensional confidence threshold; If the confidence score of the corrected component is greater than or equal to the high-dimensional confidence threshold, the final component confidence score and final component category of the target component category are obtained based on the confidence score of the corrected component and the corrected component category corresponding to the confidence score of the corrected component.

7. A terminal, characterized in that, It includes a memory and a processor, the memory being used to store a computer program, and the processor being used to call and run the computer program stored in the memory to perform the method as described in any one of claims 1 to 5.

8. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by a processor, it implements the steps of the method as described in any one of claims 1 to 5 above.

Images