Vehicle appearance recess detection method and system based on visual identification

Abstract

The invention provides a vehicle appearance depression detection method and system based on visual identification, relates to the technical field of visual identification, and aims to fully utilize the vehicle surface water drop flowing behavior in actual scenes such as spraying before vehicle cleaning or maintenance to extract a potential abnormal mode in a fluid track as a depression sensing clue so as to detect the depression. A unified vehicle body structure unfolding expression mode is introduced, a detection result is accurately mapped to the position of a specific part of a vehicle, conversion from point-like identification to part-level labeling is achieved, and readability and maintenance directivity of the detection result are improved. The whole scheme emphasizes natural fusion of the sensing process and the service process, so that the detection action and the cleaning process are highly coordinated, the detection result and the maintenance process are closely connected, and the system method has the advantages of simple deployment, strong adaptability, structured output, multi-scene support and the like. The method can be widely applied to actual business scenes such as vehicle cleaning stations, maintenance stations, lease handover and vehicle condition evaluation.

Description

Technical Field

[0001] The present invention relates to the technical field of visual recognition, and in particular to a method and system for detecting vehicle exterior dents based on visual recognition. Background Art

[0002] With increasing urban traffic density and vehicle usage frequency, dents caused by minor collisions and scrapes have become one of the most common types of damage in daily vehicle use. While these dents do not affect vehicle performance, they can significantly impact vehicle aesthetics, value retention, and subsequent safety. Consequently, there is a strong demand for automated and precise detection of vehicle exterior dents in a variety of applications, including vehicle repairs, used car appraisals, rental deliveries, and insurance claims.

[0003] Current mainstream inspection methods include manual visual inspection, laser scanning, structured light imaging, and deep learning-based image recognition. However, manual visual inspection is subject to significant subjective judgment and suffers from poor stability. While laser or structured light solutions offer high accuracy, they are expensive and require complex data processing, making them difficult to implement in regular car washes or repair shops. Image recognition, while low-cost and easy to deploy, significantly reduces accuracy in situations with uneven lighting, strong reflective paint, or sparse texture features. Furthermore, inspection results are often presented in the form of pixel coordinates or heat maps, lacking structured semantic support and making them difficult to integrate with repair processes. Existing methods are particularly susceptible to environmental noise for shallow, small dents, resulting in high false detection and missed detection rates. Furthermore, most inspection systems struggle to integrate naturally with vehicle owner processes, service scenarios, or workstation configurations, often acting as an "additional step" that burdens users and limits their practical application value.

[0004] Therefore, there is an urgent need for a systematic approach that does not rely on high-cost equipment, can adapt to complex lighting environments, has the ability to identify tiny depressions, can output structured detection results and be embedded in existing service processes to meet the urgent needs of the industry. Summary of the Invention

[0005] To address the above issues, the present invention proposes a method and system for visually identifying vehicle exterior dents that combines physical response behavior with structured expression. This method can achieve high-precision and explainable dent detection without introducing expensive equipment or changing user operating habits.

[0006] To achieve the above object, the technical solution adopted by the present invention is:

[0007] In a first aspect of the present invention, a method for detecting dents in a vehicle's exterior based on visual recognition is provided, comprising the following steps:

[0008] S1. Collect an image sequence of the water droplet flow process on the vehicle body surface during the car wash process, and extract the movement trajectory of each water droplet in the continuous image frames through visual methods;

[0009] S2. Build a dynamic analysis model to perform anomaly analysis on the water droplet trajectory during the car wash process and obtain a set of candidate concave regions;

[0010] S3. Performing an image-level geometric consistency analysis and deformation scoring on the candidate concave region set to obtain a corresponding confidence score set, which is used to indicate the confidence level that the region may be a real concave region and to determine whether a concave region exists. The confidence scores with a confidence level greater than a preset threshold are selected as a high-confidence candidate region set.

[0011] S4. Mapping the high-confidence candidate regions and their corresponding confidence score sets to the vehicle structure expansion diagram to complete the structural semantic annotation of the dent detection results;

[0012] S5. Generate a structured report of the vehicle dent detection result based on the structure annotation set and the vehicle structure diagram image, and push the structured report to the user, maintenance platform, App client or backend management system.

[0013] Preferably, in S1, a high-definition industrial camera is used to capture images of the flow process of water droplets on the vehicle body surface during the car wash process, and the feature points on the image are extracted by the Shi-Tomasi corner detection algorithm. Subsequently, the PyramidalLucas-Kanade optical flow algorithm is used to track the images frame by frame to obtain the position evolution path of the water droplets in the image sequence.

[0014] Preferably, in S2, the velocity and direction of the moving trajectory of the water droplet are dynamically modeled, and a weighted velocity mutation is introduced to determine whether the trajectory of the water droplet in the concave area is abnormal. The specific calculation formula is as follows:

[0015]

[0016] Among them, ∈ i represents the abnormal trajectory score of water droplet i, n i represents the number of frames in which the water droplet i is successfully tracked in the continuous image frames, k represents the kth frame of the image, and θ k Indicates the direction change angle of the trajectory in the kth frame, represents the velocity of water droplet i in the corresponding segment, and δ is a small constant introduced to prevent division by zero.

[0017] More optimally, after mapping the end point positions of all abnormal trajectories to the image plane, a density-based clustering algorithm is used for cluster analysis. Regions in which the number of abnormal trajectories exceeds a certain proportion in the cluster will be defined as candidate concave regions. The circumscribed rectangle of each cluster is used as the region boundary to form the final set of candidate regions.

[0018] Preferably, in said S3, the specific operation steps of the geometric consistency analysis include: cutting out the image blocks corresponding to the suspicious sunken areas obtained by trajectory clustering from the original image sequence, normalizing the brightness channels of the area, removing the overall illumination offset, and then using a local second-order polynomial function to perform surface fitting on the brightness image, constructing the expected brightness distribution model, and then calculating the residual map between the actual brightness and the fitted brightness; and then performing statistical analysis on the residual map of each area.

[0019] Preferably, in S3, the specific calculation formula for the deformation score is as follows:

[0020]

[0021] Among them, s j represents the deformation score, α and β are the fusion weights of trajectory behavior score and image residual score, ∈ is the stabilization factor to avoid zero division, c j represents the number of abnormal water droplet trajectories in the corresponding area, r j Represents the deformation strength score at the image level, and max represents the selection of confidence scores greater than the preset threshold as the high-confidence candidate region set. A collection representing the number of abnormal trajectories.

[0022] Preferably, in S4, when any high-confidence candidate area is larger than a preset threshold after mapping, it is considered that the component has recognizable deformation, and is added to the final annotation set, and the boundaries of all components in the final annotation set are marked with highlighted colors on the vehicle structure diagram according to the final annotation set.

[0023] In a second aspect of the present invention, a vehicle exterior dent detection system based on visual recognition is provided, comprising:

[0024] The data acquisition and processing module is used to collect image sequences of the water droplet flow process on the vehicle body surface during the car wash process, and extract the movement trajectory of each water droplet in the continuous image frames through visual methods;

[0025] The abnormal trajectory analysis and extraction module is used to build a dynamic analysis model to perform abnormal analysis on the water droplet trajectory during the car wash process and obtain a set of candidate concave areas through extraction;

[0026] The concave assessment module is used to perform geometric consistency analysis on the image level and perform deformation scoring on the set of candidate concave regions to obtain a corresponding confidence score set, which is used to indicate the credibility of the region as a real concave and to judge whether a concave exists based on this confidence score.

[0027] The abnormal region mapping module is used to map high-confidence candidate regions and their corresponding confidence score sets to the vehicle structure expansion diagram and complete the structural semantic annotation of the dent detection results;

[0028] The structured report generation module is used to generate a structured report of the vehicle dent detection results based on the structure annotation set and the vehicle structure diagram image, and push the structured report to the user, maintenance platform, App client or backend management system.

[0029] The beneficial effects of the present invention are as follows: the present invention proposes a method and system for visual recognition of vehicle exterior dents that combines physical response behavior with structured expression, which can achieve high-precision and explainable dent detection without introducing expensive equipment or changing user operating habits. This system makes full use of the flow behavior of water droplets on the vehicle surface in actual scenarios such as vehicle cleaning or spraying before maintenance, and extracts potential abnormal patterns in the fluid trajectory as dent perception clues to enhance the detection robustness of low-texture and strong reflective areas. At the same time, a unified vehicle body structure expansion expression method is introduced to accurately map the detection results to the specific component positions of the vehicle, realizing the transformation from "point recognition" to "component-level labeling", and improving the readability and maintenance direction of the detection results. The entire solution emphasizes the natural integration of the perception process and the service process, so that the detection action and the cleaning process are highly coordinated, and the detection results are closely connected with the maintenance process. It not only improves the detection efficiency and accuracy, but also can directly generate maintenance suggestions and work order dispatch parameters, helping enterprises to form a complete intelligent service closed loop. This system approach has the advantages of easy deployment, strong adaptability, structured output, and support for multiple scenarios. It can be widely used in actual business scenarios such as vehicle cleaning stations, maintenance stations, lease handovers, and vehicle condition assessments. BRIEF DESCRIPTION OF THE DRAWINGS

[0030] Figure 1 This is a flow chart of a vehicle appearance dent detection method based on visual recognition according to the present invention.

[0031] Figure 2 This is a structural block diagram of a vehicle appearance dent detection system based on visual recognition according to the present invention. DETAILED DESCRIPTION

[0032] See also Figure 1 As shown, in a first aspect of the present invention, a method for detecting dents in a vehicle's exterior based on visual recognition is provided, comprising the following steps:

[0033] S1. Collect an image sequence of the water droplet flow process on the vehicle body surface during the car wash process, and extract the movement trajectory of each water droplet in the continuous image frames through visual methods;

[0034] The core objective of step S1 is to capture an image sequence of water droplet flow on the vehicle body during the wash cycle and, using visual methods, extract the trajectory of each droplet within consecutive image frames. These trajectories serve as the physical behavior input for subsequent dent detection. This method relies on the natural flow of water during a real-world car wash, making it seamlessly integrated into existing car washes without introducing additional operational burdens, ensuring engineering feasibility and economical deployment.

[0035] To ensure stable data acquisition, standard high-definition industrial cameras (such as the Basler ac A1920-40uc) are used as video cameras. These cameras are mounted on the top or side of the car wash at an angle of approximately 30°, covering the left and right exterior areas of the vehicle. To ensure image clarity and consistency, the cameras use a 1920×1080 resolution and a frame rate of 30 fps. An LED ring light source ensures uniform reflections on the vehicle surface and minimizes shadows. After the vehicle enters the wash station, an automatic spray system evenly sprays deionized water at a controlled flow rate of 1.5–2.0 L / min, ensuring that water droplets roll off naturally and avoid forming a continuous film. For example, a typical family sedan (4.5 meters long) generates approximately 20–40 individual water droplet paths on the door panel during spraying.

[0036] The video data collected by the camera will be stored in the form of image sequences, denoted as Each frame I t represents an image of the vehicle's exterior captured at a specific point in the cleaning process. To extract a stable water droplet path that can be used for physical trajectory analysis, the Shi-Tomasi corner detection algorithm is first used in the first frame, I1, to extract feature points. These feature points are concentrated at the edges of water droplets, reflective spots, or surface micro-protrusions. After feature point extraction, the Pyramidal Lucas-Kanade optical flow algorithm is used for frame-by-frame tracking to obtain the positional evolution path of the water droplet in the image sequence.

[0037] Each water droplet trajectory T i Expressed as:

[0038] T i ={(x1,y1),(x2,y2),...,(x n ,y n )} (1)

[0039] Among them, (x k ,y k ) indicates that the water droplets are in the image frame I kThe pixel coordinates in are calculated by the image tracking algorithm; n represents the number of consecutive image frames in which the water droplet was successfully tracked. If tracking fails in any frame due to occlusion or reflection, the track is truncated in that frame and discarded as an "incomplete track." The track extraction process sets a maximum tolerance of 2.0 pixels / frame; exceeding this threshold will result in tracking failure.

[0040] In addition, to eliminate minor jitter caused by sensor noise, the trajectory data is spatially smoothed using an average filter with a sliding window length of 3. Furthermore, to filter out unstable water droplets, such as atomized streams and surface residual water oscillations, only trajectories with a length of 5 frames or longer are retained. In actual testing, approximately 30–60 water droplet corner points can be extracted from one frame of image, of which valid trajectories account for approximately 40–60%. Finally, a set of 20–30 stable trajectories is obtained, denoted as

[0041] To facilitate subsequent trajectory anomaly analysis, the frame length of each trajectory must also be recorded and defined as a set where n i Represents the trajectory T i The frame length is a positive integer, which represents the number of frames in which the water drop is continuously visible in the image sequence.

[0042] The innovation of this step lies in converting the behavior of water droplets into trackable visual trajectories, which are used to perceive subtle geometric changes on the vehicle surface. This avoids the reliance on texture features and sensitivity to lighting and color in traditional image processing. Compared with static image texture analysis methods, this method is more robust on smooth, highly reflective paint surfaces. Furthermore, the use of sparse optical flow tracking avoids intensive full-image computation, reducing computing power requirements and improving the practicality of the overall system deployment.

[0043] Finally, step S1 outputs a trajectory set and the corresponding trajectory length set These two variables will be fully passed to the next step for trajectory anomaly detection and candidate concave area extraction, forming the data basis of the entire patent detection process.

[0044] S2. Build a dynamic analysis model to perform anomaly analysis on the water droplet trajectory during the car wash process and obtain a set of candidate concave regions;

[0045] This step generates the water droplet trajectory set in step S1. and the corresponding trajectory length set Based on this, we identify those trajectories that have local behavioral anomalies due to geometric depressions on the vehicle body surface, and extract candidate depression areas accordingly.

[0046] This step uses refined trajectory behavior modeling to identify common phenomena such as sudden velocity drops, sudden changes in direction, and abnormal trajectory terminations in concave areas. These phenomena are difficult to detect using traditional concave detection methods based on texture or shape analysis, but are particularly sensitive in this scenario, thus forming the unique innovation of this step.

[0047] When a vehicle is sprayed in a car wash, water droplets naturally flow along the surface due to factors such as gravity, surface tension, and the paint's inclination. In areas without indentations, the droplet's trajectory is continuous, smooth, and stable. However, when passing through indentations, the droplet's speed and direction may suddenly change, or even cease, due to changes in local curvature, water film retention, or altered sliding paths. This localized perturbation in the trajectory is crucial for identifying indentations.

[0048] This step first performs dynamic modeling of the trajectory in terms of speed and direction. i The continuous point pairs (x k ,y k ) and (x k+1 ,y k+1 ), define the local velocity between frames as:

[0049]

[0050] in, represents the image displacement from the kth frame to the k+1th frame. This velocity sequence will be used to determine whether there is a significant deceleration phenomenon. In order to highlight the "non-physical sudden stagnation" problem caused by the concave area, we introduce a weighted velocity mutation metric that comprehensively considers local velocity and trajectory stability:

[0051]

[0052] Among them, θ k is the direction change angle of the trajectory in the kth frame, that is, the direction change angle of the trajectory in the kth frame is the direction change angle of the trajectory in the kth frame. k-1 ,y k-1 )、(x k ,y k ) and (x k+1 ,y k+1 ). cos(θ k ) measures the smoothness of the direction, and a value close to 1 indicates a stable direction. is the velocity of the corresponding segment. δ is a small constant introduced to prevent division by zero. This formula couples sudden changes in direction with decreases in velocity, highlighting the abnormal trajectory behavior of "sudden changes in flow path + rapid decrease in flow velocity." This is an innovative criterion specifically designed for the "depression identification based on water droplet trajectory" scenario described in this patent.

[0053] Anomaly score ∈i After normalization, combined with the trajectory length n i The displacement trend of the endpoint (the direction of the dense end points) is used to determine whether it is an abnormal trajectory.

[0054] Select all ∈ i Trajectories greater than the abnormal threshold τ constitute the abnormal trajectory set The threshold can be determined dynamically by calculating the upper quartile of the entire ∈ distribution.

[0055] The end points of all abnormal trajectories After mapping to the image plane, a density-based clustering algorithm (DBSCAN) is used for cluster analysis. The area where the number of abnormal trajectories in the cluster exceeds a certain proportion (for example, the number of cluster points exceeds 30% of the total number of trajectory points within the cluster radius) will be defined as a candidate concave area. The circumscribed rectangle of each cluster is used as the region boundary to form the final candidate region set. Each R j is a rectangular box on the image plane.

[0056] The innovation of this step is not only reflected in the joint detection model of speed and direction mutation, but also in its modeling and clustering of “trajectory termination behavior”. Traditional sag recognition relies on fixed geometric models, while this step introduces a dynamic behavior regularization term (∈ i ) characterizes the local geometric response, significantly improving the sensitivity to shallow depressions or areas with strong reflectiveness, and is practical in engineering environments.

[0057] The output includes: (1) candidate concave region set Used for subsequent image enhancement and evaluation analysis; (2) Quantitative statistical collection of abnormal trajectories within the region where c j Represents the number of abnormal trajectories within the jth region, used for subsequent confidence quantification. These two output variables are passed in their entirety to the next step as input and constitute the core candidate space generation mechanism in this patented "Behavior-Driven Concavity Recognition" system.

[0058] S3. Performing an image-level geometric consistency analysis and deformation scoring on the candidate concave region set to obtain a corresponding confidence score set, which is used to indicate the confidence level that the region may be a real concave region and to determine whether a concave region exists. The confidence scores with a confidence level greater than a preset threshold are selected as a high-confidence candidate region set.

[0059] This step S3 is the set of candidate concave regions output in step S2 And the corresponding abnormal trajectory statistics On this basis, we further perform geometric consistency analysis and confidence enhancement scoring on each region at the image level.

[0060] The goal of step S3 is to leverage the physical consistency between the image data and the trajectory data to further confirm the presence of real, microscopic dents. Compared to traditional dent detection methods based on static image grayscale distribution or edge mutations, this step innovatively proposes a cross-modal confidence enhancement strategy that combines the behavioral perturbation characteristics of water droplet trajectories with an image brightness residual model for scoring. This significantly improves dent detection capabilities on highly reflective and weakly textured paint surfaces, making it particularly suitable for the complex lighting conditions found in real-world car washes and repairs.

[0061] The input variables include: (a set of candidate rectangular regions in the image), where each R j represents the suspicious concave area obtained by trajectory clustering; and where c j Represents the number of abnormal water droplet trajectories in the corresponding area. All areas are derived from the results selected in step S3 based on clustering and trajectory perturbation scores and are preliminary suspicious areas with physical behavior indicators.

[0062] In order to verify whether these regions actually correspond to the geometric deformation of the vehicle body surface, in each candidate region R j The specific operation is to cut out the image sequence that matches R from the original image sequence. j The corresponding image block I j , and normalize the brightness channel of the region to remove the overall illumination offset. Subsequently, a local second-order polynomial function is used to perform surface fitting on the brightness image to construct the expected brightness distribution model f j (x, y), and then calculate the residual map E between the actual brightness and the fitted brightness j (x,y), defined as follows:

[0063]

[0064] Where (x, y) is the image block I j The pixel position in I j (x, y) is the actual gray value of the point; f j (x, y) is the brightness surface obtained by least squares fitting, modeling the smooth reflection trend of the normal surface; λ represents the Laplace value of the brightness at that point, used to enhance the response at the deformed boundary. λ is a coefficient used to adjust the weight of the fitting residual and edge response, with a typical empirical value set between 0.2 and 0.5. This formula introduces a "Laplace regularization term," beyond traditional residual fitting, to improve response sensitivity at concave boundaries. This is a structural enhancement innovation proposed to address the practical problem of difficult-to-distinguish boundaries in highlight areas of a vehicle body.

[0065] Next, the residual map of each region is statistically analyzed to calculate the local average residual and maximum local anomaly response Then construct the image-level deformation strength score r j :

[0066]

[0067] Among them, γ is the deformation edge emphasis factor, which is usually set in the range of 1.5 to 2.0. It is used to amplify the residual response at the concave edge and improve the recognition ability of small-scale strong reflection disturbances.

[0068] On this basis, in order to realize joint modeling with physical behavior trajectory data, a cross-modal confidence fusion scoring function s is further designed. j , mapping the number of trajectories and image residual behavior to the same scoring space:

[0069]

[0070] Here, α and β are the fusion weights of the trajectory behavior score and the image residual score, respectively, and ∈ is a stabilization factor to prevent division by zero. All variables are normalized to achieve uniform dimensionality. This formula embodies a key innovation: a weighted fusion confidence mechanism is established between different physical channels. Rather than simple classification or binary judgment, it combines "local trajectory disorder" and "image brightness anomalies" in a weighted manner to improve the recognition of low-contrast concave areas.

[0071] All input variables in the formula are directly obtained from the output of the previous step (S2) or calculated from the image block, and are clearly defined and reproducible. In particular, c j From step S2 cluster generation, I j (x,y) and f j (x, y) are grayscale values ​​that can be processed by image operation functions; is the standard discrete Laplace operator output, r j and s j Both are scalar values ​​and are used for confidence ranking and region screening.

[0072] The final output includes two variables:

[0073] (1) Confidence score set for each region Used to indicate the confidence level that an area may be a real depression;

[0074] (2) After the confidence is screened, the set of high-confidence candidate regions is retained in It will serve as the input for the next step of "structural graph mapping and semantic annotation" to complete the construction of semantic-level concave expression.

[0075] By jointly analyzing trajectory behavior and image structure responses and constructing a regularized residual scoring mechanism, this step provides a highly stable and scene-adaptive sag detection method, significantly enhancing the system's accuracy under conditions of high light, weak texture, and highly reflective vehicle surfaces, providing key support for reliable sag detection in this patented system.

[0076] S4. Mapping the high-confidence candidate regions and their corresponding confidence score sets to the vehicle structure expansion diagram to complete the structural semantic annotation of the dent detection results;

[0077] The goal of this step is to convert the high confidence image candidate region output in step S3 into and its corresponding confidence score set Mapping to vehicle structure development Complete the structural semantic annotation of the sag detection results.

[0078] This step is a key link in the mapping of "image coordinates → vehicle semantic structure", ensuring that the visual inspection results can correspond one-to-one with the actual vehicle components, supporting subsequent systematic operations such as repair dispatch and parts recommendations.

[0079] Input, Each is a rectangular box, defined as the pixel pair with the upper left corner coordinates (x1, y1) and the lower right corner coordinates (x2, y2); It is a region The normalized confidence score comes from the joint judgment of image brightness residual and trajectory physical behavior.

[0080] System preset structure diagram It is a two-dimensional expansion diagram of the vehicle, covering the main view of front, rear, left and right. The structure diagram is created by three-dimensional model projection during initialization. j Its two-dimensional boundary Ω is marked in the figure. j , and comes with a unique number (such as left front door P_03, right rear fender P_11, etc.).

[0081] First, for each image candidate region According to the camera intrinsic parameter matrix K (obtained through calibration) and the shooting posture extrinsic parameter matrix T i (obtained through the preset installation posture), map the image coordinate points to the structural coordinates. The simplified calculation is as follows:

[0082] (x′,y′)=H i (x,y,1)T (7)

[0083] Among them H i =K·T i Represents the affine transformation matrix from image to structure graph, (x,y) is the center point of the region in the image, (x′,y′) is the center point of the region in the structure graph The transformed position is implemented by pixel mapping and is implemented using the cv2.getPerspectiveTransform() and cv2.perspectiveTransform() functions in OpenCV.

[0084] Map (x′,y′) to After that, traverse the boundaries of each component in the structure diagram j , determine whether it satisfies:

[0085] (x′,y′)∈Ω j (8)

[0086] That is, determine whether the mapping point falls into component P j If it falls within the boundary area, it is considered that the candidate area belongs to the structural component.

[0087] On this basis, we have j Construct a confidence accumulation variable S j . Each high confidence region Mapping to P j , then update S j :

[0088]

[0089] in It is a candidate area The pixel area is represented by ∈, which is a stabilization term to avoid division by zero. This scoring method reflects a scenario-specific design: on a vehicle surface, small, high-confidence dents (such as small impact points on the edge of a door) are more important, so the area is inversely weighted to unity. This scoring strategy prioritizes "obvious small dents" and downplays the impact of "large, low-confidence" areas.

[0090] If eventually a certain S j Exceeding the set threshold τ s (For example, if it is set to the mean of the total confidence score + 1.5 × standard deviation), the part is considered to have recognizable deformation and is added to the final annotation set:

[0091]

[0092] The two output variables of this step are:

[0093] Structural Annotation Collection Indicates which parts are marked as deformed and their corresponding confidence levels;

[0094] Updated structure diagram image exist Highlighted in color The boundaries of all components in the system are used for subsequent system interface display or report generation.

[0095] Example: If the area The mapping point falls into the left front door area Ω 03 ,and Its confidence vote for component P_03 is about 0.025. If there are multiple All fall into Ω 03 , its S 03 Will accumulate multiple times.

[0096] S5. Generate a structured report of the vehicle dent detection results based on the structure annotation set and the vehicle structure diagram image, and push the structured report to the user, maintenance platform, App client or backend management system;

[0097] The core goal of this step S5 is to set the structure annotation output of step S4 With structure diagram image Perform business conversion and generate structured reports on vehicle dent detection results The key information of the report is pushed to the maintenance platform, App client or backend management system, completing the closed-loop transformation from recognition results to actual actions.

[0098] Input, is a set of structural annotations, where each P j Indicates a vehicle component number in the structure diagram, corresponding to an area identified as having deformation, S j Give it a confidence score; It is the image output with visual annotation completed on the structure diagram.

[0099] First, the system starts Parse all marked parts P j , converted into vehicle component name Label through the preset component dictionary j (e.g. P_03 → "left front door"), and combine the component's view angle and the structural diagram coordinates to form each report item Contents include:

[0100] Component Name Label j , anomaly score S j , corresponding to the screenshot segment (from Cutting), component-level recommended operations (such as re-inspection, sheet metal spray repair);

[0101] At the same time, the system will score each S j Convert to service level g j , used to determine whether the exception triggers subsequent service actions:

[0102] g j =S j ·ω j (11)

[0103] where ω j The component importance coefficient is set based on industry maintenance data, such as 1.4 for door panels, 1.2 for bumpers, and 1.0 for roofs. This scoring strategy can improve the priority response capabilities for key components and optimize maintenance efficiency.

[0104] Based on g j The system has completed the following actual actions:

[0105] When g j ≥0.7, the system has packaged the component abnormality information and sent it to the maintenance platform dispatch system to generate a work order;

[0106] When 0.4≤g j <0.7, the system has pushed a "re-examination recommended" reminder to the user's app, along with an image screenshot;

[0107] When g j <0.4, the system has written the results into the vehicle inspection log archiving system for historical records and trend analysis.

[0108] The above actions are performed by the service linkage module deployed in the cloud, which is connected to the Senye automobile maintenance platform through the standard RESTful API interface. The format is a JSON package, and the fields include component name, score, image link and recommended operation type.

[0109] Finally, the system outputs:

[0110] Structured test report All included The items are packaged into PDF and JSON formats and automatically sent to the user's App account and backend system;

[0111] Service linkage completion list A list of executed tasks, including dispatch number, push status, log index, and other metadata, for subsequent reconciliation or service tracking.

[0112] like Figure 2As shown, in a second aspect of the present invention, a vehicle appearance dent detection system based on visual recognition is provided, comprising:

[0113] The data acquisition and processing module is used to collect image sequences of the water droplet flow process on the vehicle body surface during the car wash process, and extract the movement trajectory of each water droplet in the continuous image frames through visual methods;

[0114] The abnormal trajectory analysis and extraction module is used to build a dynamic analysis model to perform abnormal analysis on the water droplet trajectory during the car wash process and obtain a set of candidate concave areas through extraction;

[0115] The concave assessment module is used to perform geometric consistency analysis on the image level and perform deformation scoring on the set of candidate concave regions to obtain a corresponding confidence score set, which is used to indicate the credibility of the region as a real concave and to judge whether a concave exists based on this confidence score.

[0116] The abnormal region mapping module is used to map high-confidence candidate regions and their corresponding confidence score sets to the vehicle structure expansion diagram and complete the structural semantic annotation of the dent detection results;

[0117] The structured report generation module is used to generate a structured report of the vehicle dent detection results based on the structure annotation set and the vehicle structure diagram image, and push the structured report to the user, maintenance platform, App client or backend management system.

[0118] The above embodiments are merely descriptions of preferred embodiments of the present invention and are not intended to limit the scope of the present invention. Without departing from the design spirit of the present invention, various modifications and improvements made to the technical solutions of the present invention by ordinary engineering technicians in this field should fall within the scope of protection determined by the claims of the present invention.

Claims

1. A vehicle exterior dent detection method based on visual recognition, characterized in that: The following steps are involved: S1. Collect an image sequence of the water droplet flow process on the vehicle body surface during the car wash process, and extract the movement trajectory of each water droplet in the continuous image frames through visual methods; S2. Build a dynamic analysis model to perform anomaly analysis on the water droplet trajectory during the car wash process and obtain a set of candidate concave regions; S3. Performing an image-level geometric consistency analysis and deformation scoring on the candidate concave region set to obtain a corresponding confidence score set, which is used to indicate the confidence level that the region may be a real concave region and to determine whether a concave region exists. The confidence scores with a confidence level greater than a preset threshold are selected as a high-confidence candidate region set. S4. Mapping the high-confidence candidate regions and their corresponding confidence score sets to the vehicle structure expansion diagram to complete the structural semantic annotation of the dent detection results; S5. Generate a structured report of the vehicle dent detection result based on the structure annotation set and the vehicle structure diagram image, and push the structured report to the user, maintenance platform, App client or backend management system.

2. The vehicle exterior dent detection method based on visual recognition according to claim 1, characterized in that: In S1, a high-definition industrial camera is used to capture images of the flow of water droplets on the vehicle body surface during the car wash process, and the feature points on the image are extracted using the Shi-Tomasi corner detection algorithm. Subsequently, the Pyramidal Lucas-Kanade optical flow algorithm is used to track the water droplets frame by frame to obtain the position evolution path of the water droplets in the image sequence.

3. The vehicle exterior dent detection method based on visual recognition according to claim 1, characterized in that: In S2, the velocity and direction of the water droplet's trajectory are dynamically modeled, and a weighted velocity mutation is introduced to determine whether the water droplet's trajectory behavior in the concave area is abnormal. The specific calculation formula is as follows: Among them, ∈ i represents the abnormal trajectory score of water droplet i, n i represents the number of frames in which the water droplet i is successfully tracked in the continuous image frames, k represents the kth frame of the image, and θ k Indicates the direction change angle of the trajectory in the kth frame, represents the velocity of water droplet i in the corresponding segment, and δ is a small constant introduced to prevent division by zero.

4. The method for detecting vehicle exterior dents based on visual recognition according to claim 3, characterized in that: After mapping the end point positions of all abnormal trajectories to the image plane, a density-based clustering algorithm is used for cluster analysis. The areas in the clusters that contain more than a certain proportion of abnormal trajectories will be defined as candidate concave areas. The circumscribed rectangle of each cluster is used as the region boundary to form the final set of candidate regions.

5. The vehicle exterior dent detection method based on visual recognition according to claim 1, characterized in that: In S3, the specific operation steps of the geometric consistency analysis include cropping the image blocks corresponding to the suspicious sunken areas obtained by trajectory clustering from the original image sequence, normalizing the brightness channels of the area to remove the overall illumination offset, then using a local second-order polynomial function to perform surface fitting on the brightness image, constructing the expected brightness distribution model, and then calculating the residual map between the actual brightness and the fitted brightness; and then performing statistical analysis on the residual map of each area.

6. The method for detecting vehicle exterior dents based on visual recognition according to claim 1, characterized in that: In S3, the specific calculation formula for the deformation score is as follows: Among them, s j represents the deformation score, α and β are the fusion weights of trajectory behavior score and image residual score, ∈ is the stabilization factor to avoid zero division, c j represents the number of abnormal water droplet trajectories in the corresponding area, r j Represents the deformation strength score at the image level, and max represents the selection of confidence scores greater than the preset threshold as the high-confidence candidate region set. A collection representing the number of abnormal trajectories.

7. The vehicle exterior dent detection method based on visual recognition according to claim 1, characterized in that: In S4, when any high-confidence candidate region is larger than a preset threshold after mapping, it is considered that the component has a recognizable deformation and is added to the final annotation set. Based on the final annotation set, the boundaries of all components in the final annotation set are highlighted on the vehicle structure diagram.

8. A vehicle exterior dent detection system based on visual recognition, characterized in that: Includes the following connected in sequence: The data acquisition and processing module is used to collect image sequences of the water droplet flow process on the vehicle body surface during the car wash process, and extract the movement trajectory of each water droplet in the continuous image frames through visual methods; The abnormal trajectory analysis and extraction module is used to build a dynamic analysis model to perform abnormal analysis on the water droplet trajectory during the car wash process and obtain a set of candidate concave areas through extraction; The concave assessment module is used to perform geometric consistency analysis on the image level and perform deformation scoring on the set of candidate concave regions to obtain a corresponding confidence score set, which is used to indicate the credibility of the region as a real concave and to judge whether a concave exists based on this confidence score. The abnormal region mapping module is used to map high-confidence candidate regions and their corresponding confidence score sets to the vehicle structure expansion diagram and complete the structural semantic annotation of the dent detection results; The structured report generation module is used to generate a structured report of the vehicle dent detection results based on the structure annotation set and the vehicle structure diagram image, and push the structured report to the user, maintenance platform, App client or backend management system.

Images