Vehicle defect positioning method and device, vehicle and computer readable storage medium

Abstract

This application provides a method, apparatus, vehicle, and computer-readable storage medium for locating vehicle defects. The method includes: acquiring viewpoint images of the vehicle's outer surface using at least one camera positioned at a predetermined location around the vehicle, and obtaining camera pose parameters of the at least one camera when acquiring the viewpoint images; identifying defects on the vehicle's outer surface based on the viewpoint images to obtain the defect attributes of the target defect and the defect area of ​​the target defect on the vehicle's outer surface; mapping the defect area onto a three-dimensional model of the vehicle body based on the camera pose parameters and camera calibration parameters to obtain the three-dimensional coordinates of the target defect; comparing the target defect with historical defects stored in the vehicle's database based on the defect's three-dimensional coordinates and defect attributes to obtain a comparison result; and locating the defect on the three-dimensional model of the vehicle body based on the comparison result to obtain a vehicle defect location result. This application solves the technical problem of low accuracy in locating vehicle defects.

Description

Technical Field

[0001] This application relates to the field of vehicle technology, and more specifically, to a method, apparatus, vehicle, and computer-readable storage medium for locating vehicle defects. Background Technology

[0002] In the field of vehicle technology, monitoring the health status of the vehicle body structure is a crucial step in assessing vehicle lifespan and safety. However, current vehicle body structure monitoring solutions primarily rely on single-shot visual inspections. While these methods can detect defects, the data is fragmented, leading to significant errors in damage localization. This makes it difficult to meet the high accuracy and traceability requirements of automotive R&D, resulting in low accuracy in locating vehicle defects.

[0003] There is currently no good solution to the above problems. Summary of the Invention

[0004] This application provides a method, apparatus, vehicle, and computer-readable storage medium for locating vehicle defects, in order to at least solve the technical problem of low accuracy in locating vehicle defects in related technologies.

[0005] According to one aspect of the embodiments of this application, a method for locating vehicle defects is provided. The method includes: acquiring view images of the vehicle's outer surface using at least one camera positioned at a preset location around the vehicle, and obtaining camera pose parameters of the at least one camera when acquiring the view images; identifying defects on the vehicle's outer surface based on the view images to obtain defect attributes of the target defect and the defect area of ​​the target defect on the vehicle's outer surface; mapping the defect area onto a three-dimensional model of the vehicle body based on the camera pose parameters and camera calibration parameters to obtain the three-dimensional coordinates of the target defect; comparing the target defect with historical defects stored in the vehicle based on the three-dimensional coordinates and defect attributes to obtain a comparison result; and locating the defect on the three-dimensional model of the vehicle body based on the comparison result to obtain a vehicle defect location result.

[0006] Furthermore, the camera pose parameters are obtained when the camera is acquiring viewpoint images, including: obtaining the camera's rotation matrix and translation vector in the preset coordinate system when the camera is acquiring viewpoint images; determining the camera's relative position parameters based on the positional relationship between the preset coordinate system and the camera's camera coordinate system; and determining the camera pose parameters based on the rotation matrix, translation vector, and relative position parameters.

[0007] Furthermore, based on the viewpoint image, defect identification is performed on the vehicle's outer surface to obtain the defect attributes of the target defect and the defect area of ​​the target defect on the vehicle's outer surface. This includes: inputting the viewpoint image into the defect identification model to predict the defect type, defect area mask, and defect size of the target defect; determining the defect area of ​​the target defect based on the defect area mask; determining the defect attributes based on at least one of the defect type, defect size, and defect area; and determining the defect area of ​​the target defect based on the defect area mask.

[0008] Furthermore, the defect recognition model includes a crack detection network, a paint defect segmentation network, a plastic part loosening detection network, and a deformation detection network. The viewpoint image is input into the defect recognition model to predict the defect type, defect region mask, and defect size of the target defect. This includes: using the crack detection network to detect the viewpoint image and obtain the crack mask and crack length; using the paint defect segmentation network to detect the viewpoint image and obtain the defect region mask; using the plastic part loosening detection network to detect the viewpoint image and obtain the component bounding box of the loose part and the gap width of the corresponding loose region; using the deformation detection network to detect the viewpoint image and obtain the deformation region and deformation depth; determining the defect type based on at least one of the crack mask, component bounding box, and deformation region; and determining the defect size based on at least one of the crack length, gap width, and deformation depth.

[0009] Furthermore, based on the camera pose parameters and camera calibration parameters, the defect area is mapped onto the vehicle's 3D model to obtain the 3D coordinates of the target defect. This includes: selecting the coordinates of feature points in the defect area; converting the feature point coordinates into a target direction vector in the camera coordinate system according to the intrinsic parameter matrix in the camera calibration parameters; transforming the target direction vector into a preset coordinate system based on the camera pose parameters to form a target ray extending from the camera's optical center along the target direction vector, where the camera's optical center is the camera's optical center; and determining the 3D coordinates of the defect based on the target ray and the vehicle's 3D model.

[0010] Furthermore, based on the target ray and the three-dimensional model of the vehicle body, the three-dimensional coordinates of the defect are determined, including: determining the target intersection point between the target ray and the corresponding model surface of the three-dimensional model of the vehicle body; and determining the intersection point coordinates as the three-dimensional coordinates of the defect.

[0011] Furthermore, based on the three-dimensional coordinates and attributes of the defect, data comparison is performed between the target defect and the vehicle's historical defects to obtain comparison results, including: obtaining the historical defect's three-dimensional coordinates and attributes; performing spatial distance matching between the historical defect's three-dimensional coordinates and the vehicle's three-dimensional coordinates to obtain matching results, wherein the matching results are used to indicate whether the spatial distance between the historical defect's three-dimensional coordinates and the vehicle's three-dimensional coordinates is less than a preset distance threshold; and determining the comparison result based on the matching results, defect attributes, and historical defect attributes.

[0012] Furthermore, based on the matching results, defect attributes, and historical defect attributes, the comparison results are determined, including: in response to the matching result being that the spatial distance is less than a preset distance threshold, determining that the target defect is not a new defect, and determining the comparison results based on the difference between the defect attributes and historical defect attributes, as well as the fact that it is not a new defect; in response to the matching result being that the spatial distance is greater than or equal to a preset distance threshold, determining that the comparison results indicate that the target defect is a new defect.

[0013] Furthermore, based on the difference between the defect attribute and the historical defect attribute, and the non-new defect, the comparison result is determined, including: in response to the difference being greater than a preset attribute difference threshold, the comparison result is determined to be that the target defect is a non-new defect and that the target defect has defect expansion; in response to the difference being less than or equal to the preset attribute difference threshold, the comparison result is determined to be that the target defect is a non-new defect and that the target defect does not have defect expansion.

[0014] Furthermore, based on the comparison results, the vehicle is located on the 3D model of the vehicle body to obtain the vehicle defect location result, including: determining the target location information of the target defect based on the comparison results; and locating the target defect on the 3D model of the vehicle body based on the target location information to obtain the vehicle defect location result.

[0015] Furthermore, based on the comparison results, the target location information of the target defect is determined, including: in response to the comparison results indicating that the target defect is not a new defect and that the target defect has defect expansion, the defect level of the target defect is determined according to the defect expansion information of the defect expansion; the original location information of the target defect on the vehicle body three-dimensional model is updated based on the defect level to obtain the target location information.

[0016] Furthermore, based on the comparison results, the target location information of the target defect is determined, including: in response to the comparison results indicating that the target defect is not a new defect and that the target defect does not have defect extension, the original location information of the target defect on the vehicle body 3D model is determined as the target location information.

[0017] Furthermore, based on the comparison results, the vehicle is located on the 3D model of the vehicle body to obtain the vehicle defect location results, including: in response to the comparison results indicating that the target defect is a newly added defect, the target location information is determined based on the defect attributes and defect area of ​​the target defect.

[0018] Furthermore, the method also includes: determining the vehicle's defect change trend and defect change parameters based on the vehicle defect location results; and constructing a vehicle defect detection report based on the defect change trend and defect change parameters.

[0019] According to another aspect of the embodiments of this application, a vehicle defect localization device is also provided, comprising: an acquisition module, configured to acquire view images of the vehicle's outer surface using a multi-view camera, and acquire camera pose parameters of the camera when acquiring the view images; an identification module, configured to identify defects on the vehicle's outer surface based on the view images, and obtain the defect attributes of the target defect and the defect area of ​​the target defect on the vehicle's outer surface; a mapping module, configured to map the defect area onto a three-dimensional model of the vehicle body based on the camera pose parameters and the camera calibration parameters of the camera, and obtain the three-dimensional coordinates of the target defect; and a comparison module, configured to compare the target defect with historical defects of the vehicle based on the three-dimensional coordinates and defect attributes, obtain a comparison result, and locate the defect on the three-dimensional model of the vehicle body based on the comparison result, thereby obtaining the vehicle defect localization result.

[0020] Further, the acquisition module includes: an acquisition unit for acquiring the rotation matrix and translation vector of the camera in a preset coordinate system when the camera is acquiring a viewpoint image; a first determination unit for determining the relative position parameters of the camera based on the positional relationship between the preset coordinate system and the camera coordinate system; and a second determination unit for determining the camera pose parameters based on the rotation matrix, translation vector, and relative position parameters.

[0021] According to another aspect of the embodiments of this application, a vehicle is also provided, including: a memory storing an executable program; and a processor for running the program, wherein the program executes the methods in various embodiments of this application when it runs.

[0022] According to another aspect of the embodiments of this application, a computer-readable storage medium is also provided, the computer-readable storage medium including a stored executable program, wherein, when the executable program is running, it controls the device where the computer-readable storage medium is located to perform the methods of various embodiments of this application.

[0023] According to another aspect of the embodiments of this application, a computer program product is also provided, including a computer program that, when executed by a processor, implements the methods of various embodiments of this application.

[0024] According to another aspect of the embodiments of this application, a computer program product is also provided, including a non-volatile computer-readable storage medium storing a computer program that, when executed by a processor, implements the methods in various embodiments of this application.

[0025] According to another aspect of the embodiments of this application, a computer program is also provided, which, when executed by a processor, implements the methods of the various embodiments of this application.

[0026] In this embodiment, at least one camera positioned at a predetermined location around the vehicle acquires vantage points of view on the vehicle's outer surface, obtaining camera pose parameters when acquiring these images. Based on the vantage points, defects are identified on the vehicle's outer surface to obtain the defect attributes and the defect area of ​​the target defect on the vehicle's outer surface. Based on the camera pose parameters and camera calibration parameters, the defect area is mapped onto the vehicle's 3D model to obtain the 3D coordinates of the target defect. Based on the 3D coordinates and defect attributes, the target defect is compared with historical defects stored in the vehicle's database to obtain a comparison result. Based on the comparison result, the defect is located on the 3D model of the vehicle body to obtain the vehicle defect location result. Acquiring vantage points and camera pose parameters through at least one camera provides multi-view visual data and spatial attitude references, ensuring the integrity of data acquisition and the traceability of geometric references. Based on the vantage points, defect attributes and defect areas are obtained, automatically extracting damage type, size, and specific range, replacing manual visual inspection and improving recognition accuracy and efficiency. Based on pose and calibration parameters, the defect area is mapped onto the vehicle's 3D model to obtain the defect's 3D coordinates. This achieves a geometric transformation from a 2D image to 3D space, eliminating viewpoint errors and providing accurate spatial location for damage. Based on the defect's 3D coordinates and attributes, historical defects are compared to obtain the comparison results. The defect is then located on the vehicle's 3D model, yielding the vehicle defect location result. Historical data comparison distinguishes between new and extended damage, enabling dynamic tracking and visual annotation of the damage state. This application's embodiment employs multi-view visual acquisition combined with 3D coordinate mapping and historical data comparison. By acquiring camera pose and calibration parameters, the 2D defect area is mapped onto the vehicle's 3D model to generate defect coordinates. These coordinates and attributes are then spatially matched with historical defects to determine the defect state. This achieves precise digital location of damage from a 2D image to a 3D spatial location and intelligent identification of new and old defect states. This results in high location accuracy and quantifiable tracking of the damage evolution process, thus solving the technical problem of low location accuracy for vehicle defects in related technologies. Attached Figure Description

[0027] The accompanying drawings, which are included to provide a further understanding of this application and form part of this application, illustrate exemplary embodiments and are used to explain this application, but do not constitute an undue limitation of this application. In the drawings:

[0028] Figure 1 This is a flowchart of a method for locating vehicle defects according to an embodiment of this application;

[0029] Figure 2 This is a schematic diagram of the overall composition of a vehicle defect location method system according to an embodiment of this application;

[0030] Figure 3 This is a schematic diagram illustrating the operation of an artificial intelligence module for damage recognition according to an embodiment of this application;

[0031] Figure 4 This is a schematic diagram of a three-dimensional coordinate mapping principle according to an embodiment of this application;

[0032] Figure 5 This is a schematic diagram of a baseline comparison module according to an embodiment of this application;

[0033] Figure 6 This is a schematic diagram of a system data flow according to an embodiment of this application;

[0034] Figure 7 This is a schematic diagram of a vehicle defect location device according to an embodiment of this application. Detailed Implementation

[0035] To enable those skilled in the art to better understand the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present application, and not all embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative effort should fall within the scope of protection of the present application.

[0036] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of this application described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0037] According to an embodiment of this application, a method for locating vehicle defects is provided. It should be noted that the steps shown in the flowchart in the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions. Furthermore, although a logical order is shown in the flowchart, in some cases, the steps shown or described may be executed in a different order than that shown here.

[0038] This embodiment provides a method for locating vehicle defects. Figure 1 This is a flowchart of a method for locating vehicle defects according to an embodiment of this application, such as... Figure 1 As shown, the process includes the following steps:

[0039] Step S102: At least one camera positioned at a preset location around the vehicle acquires a view image of the vehicle's outer surface, and obtains the camera pose parameters of the at least one camera when acquiring the view image.

[0040] The aforementioned vehicles refer to the actual automobiles under test during the whole vehicle durability testing phase. These vehicles include complete mechanical, electrical, and body structures, and integrate interfaces for acquiring test metadata. They serve as the processing object of the visual acquisition module, the digital association subject, and the testing carrier, ensuring the traceability and accuracy of the test data.

[0041] The aforementioned preset positions refer to the set of spatial coordinates of cameras that are pre-planned and fixedly installed to cover the entire outer surface of the vehicle. These preset positions are accurately calculated to ensure that the camera's field of view can capture key parts of the vehicle body from multiple angles, providing a stable geometric reference for subsequent 3D reconstruction and serving as a prerequisite for achieving high-defect localization.

[0042] The aforementioned camera is an optical electronic device used to capture optical images and convert them into electrical signals. In a vehicle durability testing system, it is deployed at fixed points or on a mobile platform around the vehicle. The camera's main function is to acquire visual information from the vehicle's exterior surface from multiple angles, generating high-resolution two-dimensional images by recording changes in light signals, providing the raw data source for subsequent defect identification. The camera's high-accuracy imaging capability ensures that even minor damage such as cracks or scratches can be clearly captured, thus guaranteeing the quality of the data foundation for subsequent analysis.

[0043] The aforementioned viewpoint images are two-dimensional planar visual data formed after a camera lens receives reflected light. Viewpoint images represent the geometric and textural appearance of a vehicle's exterior surface at a specific shooting angle. In the damage detection process, by displaying the color, shape, and lighting details of the vehicle's surface, the computer system can make a preliminary judgment on the presence of surface anomalies based on pixel distribution characteristics. The multi-angle coverage of viewpoint images helps eliminate blind spots and improves the probability of recognizing damage to complex curved surfaces.

[0044] The aforementioned camera pose parameters are a mathematical set describing the camera's position coordinates and rotation angles in three-dimensional space. These parameters typically include rotation matrices and translation vectors, used to accurately quantify the relative relationship between the camera's optical center and the vehicle coordinate system or world coordinate system. In technical implementation, camera pose parameters are crucial for back-projecting two-dimensional image pixels into three-dimensional space. Only by accurately obtaining these parameters can a geometric mapping relationship be established between the image plane and the actual vehicle body space, ensuring the accuracy of subsequent three-dimensional coordinate calculations.

[0045] In one optional embodiment, multiple cameras are securely mounted on supports around the test station. These cameras simultaneously capture multi-angle appearance data of the vehicle while it is stationary via highly accurate synchronous trigger signals. The vehicle defect localization system synchronously reads the calibration data stored internally by each camera and the attitude information fed back from external sensors, thereby constructing a spatial acquisition network. This ensures that images captured by different cameras can be subsequently processed based on a unified spatial reference frame, achieving high-precision visual coverage of the vehicle's outer surface without blind spots.

[0046] In another optional embodiment, an autonomous navigation mobile robot equipped with a camera travels around the vehicle. During movement, an inertial measurement unit and visual odometry are used to calculate the dynamic pose changes of the camera relative to a vehicle reference point. The vehicle defect localization system updates the camera pose parameters corresponding to each acquisition frame by fusing environmental maps obtained from LiDAR scanning with camera visual data. This mobile acquisition method is flexible and adaptable to vehicles of different sizes, and can continuously acquire a sequence of viewpoint images with accurate spatiotemporal information while in motion.

[0047] This application embodiment establishes a geometric relationship between two-dimensional images and three-dimensional space by acquiring camera pose parameters, providing a data foundation for subsequently mapping defect areas to the three-dimensional model of the vehicle body, and improving the reliability and traceability of the detection results.

[0048] Step S104: Based on the viewpoint image, perform defect identification on the vehicle's outer surface to obtain the defect attributes of the target defect and the defect area of ​​the target defect on the vehicle's outer surface.

[0049] The aforementioned defect identification is the process by which a computer system uses artificial intelligence algorithms or image processing methods to analyze an input viewpoint image and extract abnormal features. Defect identification aims to separate regions with abnormal shapes, colors, or textures from a complex background. Through techniques such as pattern matching, edge detection, or deep learning classification, it automatically determines whether there are structures in the image that do not conform to normal standards. This process can replace manual visual inspection, improve detection efficiency, and provide basic data support for subsequent quantitative analysis.

[0050] The aforementioned target defects refer to specific damaged objects identified as abnormal by the vehicle defect localization system after the defect identification process. Target defects typically manifest as problems affecting vehicle durability or appearance, such as cracks, paint peeling, dents, or loose parts. Target defects are the core focus of subsequent processing; the vehicle defect localization system needs to perform independent attribute extraction and region labeling for each identified target defect to ensure that every anomaly can be tracked and analyzed.

[0051] The aforementioned defect attributes are a set of quantitative indicators describing the physical characteristics and severity of a target defect. Defect attributes include, but are not limited to, specific parameters such as defect type, defect size, defect area, defect depth, or gap width. By acquiring defect attributes, vehicle defect localization systems can objectively assess the level of damage. For example, determining whether crack length exceeds a safety threshold or whether paint peeling area affects corrosion resistance provides a basis for vehicle repair decisions and durability assessments.

[0052] The aforementioned defect region refers to the specific pixel range or mask outline belonging to the target defect in the view image. The defect region defines the position of the target defect in the image using two-dimensional image coordinates, and can be represented by a mask image generated by a semantic segmentation algorithm. Determining the defect region helps eliminate background noise interference, ensuring that the defect corresponds to a specific part of the vehicle body surface when subsequently mapped to a three-dimensional model. This is a key geometric basis for realizing the conversion from two-dimensional image to three-dimensional space.

[0053] In one optional embodiment, the vehicle defect localization system pre-deploys multiple deep learning networks, trained separately for different types of damage such as cracks, paint defects, loose plastic parts, and deformation. When a viewpoint image is input into the system, the dedicated networks operate in parallel: the crack detection network outputs a crack mask and its length; the paint defect segmentation network outputs a defect region mask; the loose plastic part detection network outputs the component bounding box and gap width; and the deformation detection network outputs the deformation region and its depth. The vehicle defect localization system then aggregates the outputs of each network to obtain the defect attributes and defect region of the target defect.

[0054] In another optional embodiment, the vehicle defect localization system employs a general multi-task learning framework, using a shared feature extraction backbone network to process viewpoint images. The feature extraction network first extracts high-level semantic features from the viewpoint images. Subsequently, multiple parallel prediction models predict the defect type, defect region mask, and defect size parameters, respectively. The vehicle defect localization system calculates the defect area based on the predicted defect region mask, and determines the defect attributes by combining the defect type and size parameters, thereby outputting relevant information about the target defect.

[0055] This application's embodiments achieve automated quantitative assessment of damage by identifying defect attributes and regions, providing an accurate data foundation for subsequent 3D mapping and historical comparison, and improving the objectivity and efficiency of detection.

[0056] Step S106: Based on the camera pose parameters and camera calibration parameters, the defect area is mapped onto the vehicle body 3D model to obtain the 3D coordinates of the target defect.

[0057] The camera calibration parameters mentioned above are a fixed set of values ​​describing the camera's internal optical characteristics and geometric distortions. These parameters include an intrinsic parameter matrix and distortion coefficients. The intrinsic parameter matrix contains the focal length and principal point coordinates, used to describe the projection relationship between the camera's imaging plane and the world coordinate system. The distortion coefficients are used to correct radial and tangential distortions caused by the lens. Camera calibration parameters are essential for eliminating image geometric distortion, achieving high-accuracy measurements, and ensuring that the conversion from image pixels to three-dimensional spatial coordinates conforms to the laws of physical optics.

[0058] The aforementioned 3D vehicle body model is a digital 3D representation of the vehicle's external geometry. The 3D model contains triangular mesh or polygonal patch information of the vehicle's outer surface, recording the coordinates of each vertex in 3D space. The 3D model provides a spatial reference frame for defect localization, enabling vehicle defect localization systems to match physical damage with the digital model. By importing a standard 3D vehicle body model, vehicle defect localization systems can unify the detection benchmarks for different vehicles, achieving standardized recording of damage locations.

[0059] The aforementioned three-dimensional coordinates of the defect represent the specific spatial location of the target defect within the vehicle's three-dimensional coordinate system. These coordinates typically consist of values ​​in three directions, describing the position of the defect's center point or feature point within the vehicle's coordinate system. The three-dimensional coordinates convert pixel positions in a two-dimensional image into physically meaningful spatial coordinates, enabling the quantification, storage, and retrieval of damage locations. They provide fundamental data for subsequent historical comparisons and structural strength assessments.

[0060] In one optional embodiment, the vehicle defect localization system first selects the coordinates of feature points in the defect area and converts these coordinates into a target direction vector in the camera coordinate system based on the intrinsic parameter matrix in the camera calibration parameters. Then, based on the camera pose parameters, the system transforms the target direction vector into a preset coordinate system, forming a target ray that originates from the camera's optical center and extends along the target direction vector. Finally, the system calculates the intersection point between the target ray and the surface of the vehicle's 3D model and determines the intersection point coordinates as the defect's 3D coordinates, thereby achieving an accurate mapping from a 2D image to 3D space.

[0061] In another optional embodiment, the vehicle defect localization system constructs a camera extrinsic parameter matrix using camera pose parameters, uniformly transforming the pixels in the defect area to the world coordinate system. The system traverses the triangular faces of the vehicle's 3D model, employing a ray intersection algorithm to calculate the intersection of the ray passing from the camera's optical center through each defect pixel with the model surface. The system selects the intersection point closest to the camera's optical center as the pixel's mapped position in 3D space. By batch processing the pixels in the defect area, the system generates a 3D point cloud of defects and calculates the center point of the point cloud as the defect's 3D coordinates.

[0062] The aforementioned ray intersection algorithm refers to a mathematical calculation method that uses computational geometry principles to construct a direction vector originating from the camera's optical center and passing through image pixels, transforming it into a ray in three-dimensional space. Then, it calculates the geometric intersection relationship between the ray and the set of triangular facets on the surface of the three-dimensional vehicle model, thereby determining the coordinates of the nearest intersection point between the ray and the model surface.

[0063] This application embodiment achieves the mapping from a two-dimensional defect area to a three-dimensional vehicle body model by introducing camera pose parameters and camera calibration parameters, obtaining physically meaningful three-dimensional coordinates of the defect, and providing a data foundation for accurate damage location and subsequent analysis space.

[0064] Step S108: Based on the three-dimensional coordinates and attributes of the defect, compare the target defect with the historical defects stored in the vehicle to obtain the comparison results. Based on the comparison results, locate the defect on the three-dimensional model of the vehicle body to obtain the vehicle defect location results.

[0065] The aforementioned data comparison refers to the process of matching the spatial distance of the currently detected target defect with historical defects stored in the database and analyzing attribute differences. This process involves unifying the coordinate system, determining distance thresholds, and calculating attribute differences. Its function is to automatically identify the new state and evolution of defects, distinguishing between new and non-new defects. Through data comparison, the system can accurately locate the nodes of damage change, eliminate environmental noise interference, ensure the authenticity and reliability of the detection results, provide accurate data for subsequent defect level updates and report generation, and realize the intelligentization and standardization of the detection process.

[0066] The aforementioned comparison results refer to the judgment conclusions regarding the target defect status obtained through the data comparison process, including information such as newly added defects, non-newly added defects, and whether defect expansion exists. This result integrates spatial matching results and attribute difference analysis, serving as a bridge connecting detection input and output positioning. Its function is to provide a logical basis for subsequent visual positioning on the vehicle's 3D model, determining the defect display style, color label, and level update strategy. Through clear comparison results, the system can dynamically reflect the development trend of vehicle damage, support early warning of high-risk areas, and improve the depth and breadth of durability test data analysis.

[0067] The aforementioned vehicle defect location results refer to the final set of visualized defect annotation information generated on the vehicle's 3D model based on the comparison results. This includes comprehensive data such as the defect's 3D location, attribute parameters, status indicators, and evolution trends. This result is the final output of the inspection process, directly reflecting the vehicle's current health status. Its purpose is to provide complete information support for subsequent report generation, data analysis, and maintenance decisions. Through the vehicle defect location results, the system can transform raw images into structured data, ensuring the integrity and traceability of inspection information, providing accurate durability test data feedback to the vehicle integration department, and contributing to vehicle quality improvement.

[0068] In one optional embodiment, the vehicle defect localization system first acquires the three-dimensional coordinates and attributes of the target defect, and simultaneously retrieves the three-dimensional coordinates and attributes of historical defects stored in the database. The system calculates the spatial Euclidean distance between the target defect's three-dimensional coordinates and those of historical defects, and compares it with a preset distance threshold. If the spatial Euclidean distance is less than the preset threshold, the system determines the target defect is not a new defect, and further calculates the difference between the defect attributes and historical defect attributes. If the difference is greater than a preset attribute difference threshold, the system determines the comparison result indicates defect expansion; otherwise, it determines no defect expansion. If the spatial Euclidean distance is greater than or equal to the preset distance threshold, the system determines the comparison result is a new defect. Subsequently, based on the comparison result, the system determines the target location information of the target defect on the vehicle's three-dimensional model. If defect expansion exists, the defect level is updated and the defect is relocated; if it is a new defect, it is directly located based on the defect attributes. Finally, a vehicle defect localization result containing the three-dimensional coordinates, attributes, and status identifiers of all defects is generated and stored in the durability test database.

[0069] In another optional embodiment, the vehicle defect localization system utilizes a spatial index structure to accelerate the retrieval process of historical defects. First, it determines a set of candidate historical defects based on the three-dimensional coordinates of the target defect. The system then traverses the candidate set, calculating the spatial distance between the three-dimensional coordinates of the historical defects and the target defect, filtering out several historical defects with smaller spatial distances. The system then compares the historical defect attributes of these historical defects with the defect attributes of the target defect, calculating the attribute differences. Based on the principle of minimum spatial distance and an attribute difference threshold, the system determines matching historical defects as comparison benchmarks, thereby generating a comprehensive vehicle defect localization result that includes matching confidence, defect evolution type, and recommended repair strategies.

[0070] This application embodiment achieves intelligent identification and status update of new and old defects through data comparison, and performs visual positioning by combining the three-dimensional model of the vehicle body, which improves the accuracy and timeliness of the detection results and provides reliable data support for vehicle durability assessment.

[0071] In this embodiment, at least one camera positioned at a predetermined location around the vehicle acquires vantage points of view on the vehicle's outer surface, obtaining camera pose parameters when acquiring these images. Based on the vantage points, defects are identified on the vehicle's outer surface to obtain the defect attributes and the defect area of ​​the target defect on the vehicle's outer surface. Based on the camera pose parameters and camera calibration parameters, the defect area is mapped onto the vehicle's 3D model to obtain the 3D coordinates of the target defect. Based on the 3D coordinates and defect attributes, the target defect is compared with historical defects stored in the vehicle's database to obtain a comparison result. Based on the comparison result, the defect is located on the 3D model of the vehicle body to obtain the vehicle defect location result. Acquiring vantage points and camera pose parameters through at least one camera provides multi-view visual data and spatial attitude references, ensuring the integrity of data acquisition and the traceability of geometric references. Based on the vantage points, defect attributes and defect areas are obtained, automatically extracting damage type, size, and specific range, replacing manual visual inspection and improving recognition accuracy and efficiency. Based on pose and calibration parameters, the defect area is mapped onto the vehicle's 3D model to obtain the defect's 3D coordinates. This achieves a geometric transformation from a 2D image to 3D space, eliminating viewpoint errors and providing accurate spatial location for damage. Based on the defect's 3D coordinates and attributes, historical defects are compared to obtain the comparison results. The defect is then located on the vehicle's 3D model, yielding the vehicle defect location result. Historical data comparison distinguishes between new and extended damage, enabling dynamic tracking and visual annotation of the damage state. This application's embodiment employs multi-view visual acquisition combined with 3D coordinate mapping and historical data comparison. By acquiring camera pose and calibration parameters, the 2D defect area is mapped onto the vehicle's 3D model to generate defect coordinates. These coordinates and attributes are then spatially matched with historical defects to determine the defect state. This achieves precise digital location of damage from a 2D image to a 3D spatial location and intelligent identification of new and old defect states. This results in high location accuracy and quantifiable tracking of the damage evolution process, thus solving the technical problem of low location accuracy for vehicle defects in related technologies.

[0072] Optionally, the camera pose parameters are obtained when the camera is acquiring viewpoint images, including: obtaining the rotation matrix and translation vector of the camera in a preset coordinate system when the camera is acquiring viewpoint images; determining the relative position parameters of the camera based on the positional relationship between the preset coordinate system and the camera coordinate system; and determining the camera pose parameters based on the rotation matrix, translation vector, and relative position parameters.

[0073] The aforementioned preset coordinate system is a standard coordinate system used for unified spatial reference, typically defined as the vehicle coordinate system or world coordinate system. The preset coordinate system provides a unified benchmark for spatial positioning, enabling data from different cameras to be fused and calculated within the same spatial framework. In vehicle durability testing, the preset coordinate system is fixed to the vehicle or test site, ensuring that regardless of camera position changes, all defect coordinates are mapped to a unified reference system. This eliminates coordinate confusion caused by different camera positions and forms the basis for multi-view data stitching and 3D positioning.

[0074] The rotation matrix described above is an orthogonal matrix that describes the rotation state of the camera coordinate system relative to a preset coordinate system. The rotation matrix expresses the camera's orientation in space through a combination of rotation angles along the three fundamental axes. The rotation matrix reflects changes in the camera's attitude. In camera pose calculation, the rotation matrix is ​​used to rotate vectors in the camera coordinate system to the preset coordinate system. It is a mathematical tool for constructing spatial transformation relationships, ensuring geometric consistency of direction vectors when transforming between different coordinate systems.

[0075] The translation vector described above describes the displacement of the camera coordinate system origin relative to the origin of the preset coordinate system. The three components of the translation vector represent the distances of the camera origin in the three directions of the preset coordinate system. The translation vector and the rotation matrix together constitute the rigid body transformation parameters, describing the camera's position and orientation in space. In acquiring camera pose parameters, the translation vector is used to compensate for the positional deviation between the camera origin and the reference origin, and is a key element in realizing the transformation from local camera coordinates to global coordinates.

[0076] The camera coordinate system described above is a local three-dimensional Cartesian coordinate system attached to the optical center of the camera. The origin of the camera coordinate system is usually located at the optical center of the camera lens. The camera coordinate system is used to describe the spatial relationship of image pixels within the camera. The raw image data acquired by the camera undergoes preliminary processing and feature extraction in the camera coordinate system. The camera coordinate system is the fundamental reference frame for perspective projection transformations.

[0077] The aforementioned relative position parameters are correction parameters describing the fixed geometric relationship between the camera coordinate system and the preset coordinate system. These parameters are used to correct for camera installation errors or systematic deviations during calibration, typically including additional offsets or fine-tuning angles. The relative position parameters are derived from mechanical installation tolerances or software calibration. When determining the camera pose, the relative position parameters are used to compensate for the initially acquired rotation matrix and translation vector, improving the accuracy of pose calculation. The introduction of relative position parameters enhances the system's adaptability to different installation environments and ensures the accuracy of the pose parameters.

[0078] In one optional embodiment, the vehicle defect localization system first acquires the rotation matrix and translation vector of the camera relative to a preset coordinate system at the instant the image is captured using a visual sensor or inertial measurement unit. Then, the system retrieves pre-stored camera installation data and reads the relative position parameters between the camera coordinate system and the preset coordinate system. These relative position parameters record minute offsets during the camera's physical installation. Finally, the system fuses the rotation matrix, translation vector, and relative position parameters, eliminating installation deviations through matrix operations to calculate the final, highly accurate camera pose parameters, providing an accurate transformation basis for subsequent coordinate mapping.

[0079] This application embodiment corrects the rotation matrix and translation vector by introducing relative position parameters, thereby eliminating camera installation errors, improving the accuracy of camera pose parameter calculation, and thus improving the accuracy of defect three-dimensional coordinate mapping.

[0080] Optionally, defect identification is performed on the vehicle's outer surface based on the viewpoint image to obtain the defect attributes of the target defect and the defect area of ​​the target defect on the vehicle's outer surface, including: inputting the viewpoint image into a defect identification model to predict the defect type, defect area mask, and defect size of the target defect; determining the defect area of ​​the target defect based on the defect area mask; determining the defect attributes based on at least one of the defect type, defect size, and defect area; and determining the defect area of ​​the target defect based on the defect area mask.

[0081] The aforementioned defect recognition model is an artificial intelligence algorithm framework for automating the analysis of visual images and extracting damage features. By learning the visual differences between normal and abnormal states of a vehicle surface, the model possesses the ability to automatically detect, classify, and segment defects in complex backgrounds. It can handle multiple types of damage, such as cracks and paint peeling, and output structured prediction results. The model's role is to transform unstructured image data into structured defect information, replacing traditional manual visual inspection, improving detection efficiency and consistency, and providing a data foundation for subsequent quantitative analysis.

[0082] The aforementioned defect types are classification identifiers characterizing the physical properties and damage patterns of target defects. Defect types typically include specific categories such as cracks, paint peeling, dents, scratches, and loose or deformed plastic parts. Defect types define the mechanism and manifestation of damage, serving as the basis for judging the extent of damage and developing repair plans. Defect types are output through a classification network, and the vehicle defect localization system categorizes the damage based on the prediction results. Clearly defined defect types facilitate the subsequent application of specific dimensional measurement algorithms for different damage characteristics, ensuring the accuracy and relevance of defect attribute descriptions.

[0083] The aforementioned defect region mask is a binary matrix or contour set covering the target defect pixels in the overlay image. The defect region mask delineates the extent of the target defect in the image at the pixel level. For example, background pixels are labeled 0, and defect pixels are labeled 1. The defect region mask eliminates interference from irrelevant backgrounds in the image, defining the spatial boundaries of the defect. As intermediate data, the defect region mask forms the basis for calculating the defect area, extracting the defect size, and performing subsequent 3D mapping. The high resolution of the defect region mask ensures the preservation of defect geometric features, providing reliable pixel-level data support for quantitative analysis.

[0084] The aforementioned defect dimensions are quantitative parameters describing the geometric scale of a target defect. Defect dimensions are expressed as linear measures such as length, width, height, or diameter, depending on the defect type. Examples include the length of a crack, the maximum span of paint peeling, or the width of a loose gap in a plastic part. Defect dimensions reflect the physical extent of damage and are an indicator for assessing the degree of damage. Calculated from defect area masks or feature extraction, defect dimensions provide objective numerical data, allowing for comparative analysis of damage at different periods and locations both laterally and longitudinally.

[0085] The aforementioned defect area is the numerical value mapped from the total number of pixels occupied by the target defect in the view image to actual physical units. The defect area is obtained by statistically analyzing the number of effective pixels in the defect region mask and converting this value using the scale in the camera calibration parameters. The defect area reflects the scale of two-dimensional extended damage such as paint peeling, corrosion, or large-area dents. As a component of defect attributes, the defect area quantifies the degree of damage development; larger areas indicate structural or aesthetic impact and serve as a reference for determining the defect level and whether immediate repair is necessary.

[0086] In one optional embodiment, the vehicle defect localization system first inputs the acquired viewpoint image into a pre-trained defect recognition model. The defect recognition model performs forward propagation operations, outputting in parallel a classification identifier for the target defect as the defect type, generating a pixel-level binary matrix as a defect region mask, and calculating the geometric bounding box of the defect to determine its size. Subsequently, the vehicle defect localization system counts the number of effective pixels in the defect region mask and, combined with camera intrinsic parameters, calculates the actual physical size to obtain the defect area. Finally, the vehicle defect localization system summarizes the defect type, defect size, and defect area information, constructs a defect attribute list, and outputs the defect region of the target defect according to the definition of the defect region mask, completing the conversion from image pixels to structured defect information.

[0087] This application embodiment automatically outputs the defect type, mask, and size through a defect identification model, and calculates the area and region based on the mask, realizing the quantification and structured extraction of defect information, providing high-quality data input for subsequent 3D mapping and historical comparison.

[0088] Optionally, the defect recognition model includes a crack detection network, a paint defect segmentation network, a plastic part loosening detection network, and a deformation detection network. The viewpoint image is input into the defect recognition model to predict the defect type, defect region mask, and defect size of the target defect. This includes: using the crack detection network to detect the viewpoint image and obtain the crack mask and crack length; using the paint defect segmentation network to detect the viewpoint image and obtain the defect region mask; using the plastic part loosening detection network to detect the viewpoint image and obtain the component bounding box of the loose part and the gap width of the corresponding loose region; using the deformation detection network to detect the viewpoint image and obtain the deformation region and deformation depth; determining the defect type based on at least one of the crack mask, component bounding box, and deformation region; and determining the defect size based on at least one of the crack length, gap width, and deformation depth.

[0089] The crack detection network described above is a deep learning sub-model used to identify minute linear fracture structures on vehicle surfaces. Crack detection networks typically employ an encoder-decoder architecture, capable of capturing elongated, discontinuous, low-contrast features. The network performs pixel-level classification, outputting a binarized mask of the crack, and further extracts morphological features to calculate the linear extension distance of the crack. The role of the crack detection network is to separate minute crack damage from the complex vehicle body background, providing high-precision crack geometry information and offering crucial data support for assessing structural integrity.

[0090] The aforementioned paint defect segmentation network is a semantic segmentation deep learning sub-model focused on identifying abnormal regions on the vehicle coating surface. It aims to distinguish normal paint textures from defective areas such as peeling, scratches, bulges, or rust. Through multi-scale feature fusion, the network accurately segments paint damage areas with different textures and colors, outputting a complete defect area mask. The network's role is to extract paint damage boundaries, eliminate lighting interference, ensure the integrity of defect area coverage, and provide accurate data for area calculation and appearance evaluation.

[0091] The aforementioned plastic component loosening detection network is a deep learning sub-model for target detection and regression used to identify abnormal assembly gaps in external plastic components of vehicles. The network determines whether a component is loose by detecting the relative displacement of its edges or changes in gap width. It outputs the bounding box of the loose component to locate its position and regresses to calculate the gap width between the loose component and its adjacent components. The network's function is to monitor the assembly status of non-rigidly connected components, assess the stability of assembly quality by quantifying gap changes, and prevent abnormal noises or wind noise caused by loosening.

[0092] The aforementioned deformation detection network is a deep learning sub-model for 3D reconstruction or depth estimation used to identify deviations in the geometric shape of vehicle body panels from design standards. The deformation detection network reconstructs the 3D geometry of the vehicle surface by analyzing parallax, lighting distortion, or combining depth sensor data in the image. It outputs the contour of the deformed area and the depth deviation value relative to the standard surface. The network's function is to convert 2D image information into 3D deformation information, quantify the degree of geometric damage such as dents or protrusions, and provide geometric parameters for evaluating structural durability and aerodynamic performance.

[0093] The crack mask described above is a binary matrix output by the crack detection network that identifies the location of crack pixels in the viewpoint image. The crack mask outlines the direction and extent of the crack; background pixels are marked as non-cracked, and cracked pixels are marked as cracked. The crack mask is the fundamental data for extracting crack geometric features, used to calculate morphological indicators such as crack length and number of branches. The role of the crack mask is to provide direct visual evidence for crack type determination and size quantification, ensuring the objectivity and traceability of crack detection results.

[0094] The component bounding box described above is a set of rectangular region coordinates output by the plastic part loosening detection network, enclosing the loose component. The component bounding box marks the approximate location of the loose component in the view image, including the pixel coordinates of the upper left and lower right corners. It eliminates interference from irrelevant areas in the image, focusing on the specific component to be detected. The purpose of the component bounding box is to provide the local spatial extent of the defect, facilitating subsequent detailed gap width analysis of that area and improving the targeting and efficiency of the detection.

[0095] The aforementioned gap width is the physical distance between the loose component and adjacent structures, calculated by the regression analysis of the plastic part loosening detection network. Gap width is typically measured in millimeters and reflects the tightness of the component assembly. An abnormal increase in gap width is a primary basis for judging plastic part loosening. The function of gap width is to quantify changes in assembly status; by comparing with baseline gap widths, the degree and trend of loosening can be determined, providing quantitative engineering parameters for maintenance decisions.

[0096] The aforementioned deformation region is a set of pixels in the image that have undergone geometric changes, output by the deformation detection network. The deformation region defines the specific area of ​​the vehicle surface where a depression, bulge, or twist has occurred, in the form of a mask or contour. It eliminates interference from the normal vehicle body surface, focusing on the deformed area. The function of the deformation region is to define the spatial extent of the damage; combined with deformation depth data, the degree of geometric damage can be assessed, providing a regional basis for structural strength analysis.

[0097] The aforementioned deformation depth is the distance deviation of the deformed region surface relative to the ideal standard curved surface, calculated by the deformation detection network. Deformation depth is typically measured in millimeters, with positive values ​​indicating bulges and negative values ​​indicating depressions. Deformation depth quantifies the severity of geometric damage. Its role is to provide a damage measurement in three-dimensional space, transforming visual anomalies in two-dimensional images into measurable physical parameters. It is a key indicator for evaluating vehicle body stiffness and collision repair effectiveness.

[0098] In one optional embodiment, the vehicle defect localization system operates four deep learning sub-networks in parallel, each processing the input viewpoint image. A crack detection network generates a crack mask and extracts the crack length; a paint defect segmentation network generates a region mask for paint defects; a plastic looseness detection network outputs a component bounding box and regresses the gap width; and a deformation detection network outputs the deformation region and depth value. The vehicle defect localization system determines the defect type based on the presence of features output by each network. Subsequently, the system extracts corresponding quantized parameters, such as crack length, gap width, or deformation depth, as the defect size, thereby forming a complete description of the defect type and size.

[0099] This application embodiment achieves the identification and quantification of various heterogeneous damages through multi-network parallel processing and feature fusion, improving the accuracy of defect classification and the comprehensiveness of dimensional measurement, and meeting the needs of automated detection of multiple types of damage in vehicle durability testing.

[0100] Optionally, based on the camera pose parameters and camera calibration parameters, the defect area is mapped onto the vehicle body 3D model to obtain the defect 3D coordinates of the target defect, including: selecting the feature point coordinates in the defect area; converting the feature point coordinates into a target direction vector in the camera coordinate system according to the intrinsic parameter matrix in the camera calibration parameters; converting the target direction vector into a preset coordinate system based on the camera pose parameters to form a target ray that starts from the camera optical center and extends along the target direction vector, where the camera optical center is the optical center of the camera; and determining the defect 3D coordinates based on the target ray and the vehicle body 3D model.

[0101] The aforementioned feature point coordinates refer to the key pixel position data extracted from the identified defect area for spatial localization. These coordinates are typically selected from the center point, geometric centroid, or corner points of the defect area's contour. The feature point coordinates represent the specific pixel position of the defect in the two-dimensional image plane and serve as the initial input data for subsequent three-dimensional reconstruction. The role of feature point coordinates is to act as a geometric anchor connecting the two-dimensional visual image and three-dimensional physical space. By converting them into direction vectors in three-dimensional space, they ensure the accuracy of defect mapping and are the fundamental data unit for realizing the coordinate transformation from a planar image to a three-dimensional model.

[0102] The intrinsic parameter matrix described above describes the internal optical imaging characteristics of the camera. It contains parameters such as focal length and principal point coordinates, and is used to establish a linear transformation relationship between image pixel coordinates and normalized planar coordinates in the camera coordinate system. The intrinsic parameter matrix eliminates the effects of lens distortion, transforming the pixel domain to the metric domain. Its function is to convert pixel positions in a two-dimensional image into direction vectors in the camera coordinate system; it is a mathematical tool for realizing the projection transformation from the image plane to three-dimensional space, ensuring the accuracy of geometric relationships.

[0103] The aforementioned preset coordinate system refers to a standard reference coordinate system used to uniformly describe the spatial location of the vehicle, camera, and damage, typically defined as the vehicle coordinate system. In the vehicle coordinate system, the origin is generally the center of the vehicle's front axle or geometric center, with the positive X-axis pointing towards the front of the vehicle, the positive Y-axis pointing towards the driver's side (or left side), and the positive Z-axis pointing vertically upwards. The purpose of the preset coordinate system is to provide a unified spatial reference for camera data from different perspectives, camera pose parameters, and the vehicle's 3D computer-aided design (CAD) model, eliminating coordinate system differences caused by different camera installation positions or angles, ensuring the accuracy and consistency of defect 3D coordinate calculations, and supporting precise spatial comparison and tracking of damage locations during different inspection cycles.

[0104] The aforementioned target direction vector refers to the spatial geometric vector representing the direction from the camera's optical center to the physical location of the defect, obtained by back-projecting the pixel coordinates of the feature points in the defect region to the camera coordinate system using the camera's intrinsic parameter matrix. The target direction vector describes the propagation direction of light rays in the camera's own coordinate system, serving as an intermediate bridge connecting two-dimensional image pixels and three-dimensional physical points. The role of the target direction vector is to transform positional information on the image plane into spatially meaningful directional information, providing fundamental data for subsequent direction transformation to the vehicle's global coordinate system using camera extrinsic parameters. This ensures that the ray accurately points to the actual location of the defect on the vehicle's surface, making it a key intermediate parameter for achieving three-dimensional coordinate mapping.

[0105] The aforementioned camera optical center is a node in the camera lens optical system, specifically the point where incident light rays appear to converge or diverge after refraction. The camera optical center is considered the origin of the camera coordinate system. It is the center point of perspective projection transformation; all projected rays from the scene to the image plane pass through this point. The function of the camera optical center is to provide a fixed spatial starting point for constructing target rays, ensuring that the rays are consistent with the camera's imaging geometry, and is a fundamental geometric element for achieving 3D reconstruction.

[0106] In one optional embodiment, the vehicle defect localization system first selects feature point coordinates from the defect area. Using the intrinsic parameter matrix in the camera calibration parameters, it converts the feature point coordinates into a target direction vector in the camera coordinate system through inverse projection. Subsequently, the system uses the rotation matrix and translation vector in the camera pose parameters to rotate and translate the target direction vector from the camera coordinate system to a preset coordinate system, thereby forming a target ray extending from the camera's optical center along that direction. Finally, the system calculates the intersection point between the target ray and the surface of the vehicle's 3D model, determining the intersection point coordinates as the defect's 3D coordinates, thus completing a high-precision mapping from a 2D image to 3D space.

[0107] This application embodiment constructs a target ray by selecting feature points, performing inverse transformation of intrinsic parameters, and transforming extrinsic coordinates. It then uses an algorithm to find the intersection between the target ray and the three-dimensional model to determine the three-dimensional coordinates of the defect, thereby realizing the transformation of two-dimensional visual information into three-dimensional spatial location and improving the accuracy and reliability of positioning.

[0108] Optionally, based on the target ray and the three-dimensional model of the vehicle body, the three-dimensional coordinates of the defect are determined, including: determining the target intersection point between the target ray and the corresponding model surface of the three-dimensional model of the vehicle body; and determining the intersection point coordinates as the three-dimensional coordinates of the defect.

[0109] The aforementioned target ray refers to a geometric ray that originates from the camera's optical center and extends along a specific direction in a preset coordinate system after camera pose parameter transformation. The target ray represents the line-of-sight trajectory from the camera's optical center to the physical spatial location corresponding to the defect feature point in the image. The function of the target ray is to transform the defect pixel coordinates on the two-dimensional image plane into directional constraints in three-dimensional space. By calculating the intersection point of this ray with the surface of the vehicle's three-dimensional CAD model, the three-dimensional spatial coordinates of the defect within the vehicle's physical structure can be located. It is a key geometric element for realizing the spatial mapping from visual image to physical model.

[0110] The aforementioned target intersection point is the geometric point where the target ray intersects the surface of the vehicle's 3D model. This target intersection point represents the precise physical location of the defect on the vehicle's 3D model, and is the unique or closest point of contact between the ray and the model's surface. The existence of the target intersection point proves the locatability of the defect in 3D space. The role of the target intersection point is to transform the abstract ray projection into a concrete spatial location point, and it is a key geometric element for realizing the conversion from directional information to positional information.

[0111] In one optional embodiment, the vehicle defect localization system employs a ray intersection algorithm to calculate the intersection of the target ray with the triangular facets in the 3D model of the vehicle body. The system filters out non-back facet intersections that are close to the optical center of the camera and identifies them as target intersections. Subsequently, the system reads the coordinate values ​​of the target intersections in a preset coordinate system and directly assigns these intersection coordinates to the 3D coordinates of the defect, thereby completing the mapping from 2D image pixels to 3D physical coordinates.

[0112] This application embodiment calculates the intersection point between the target ray and the surface of the three-dimensional vehicle body model, and directly extracts the intersection point coordinates as the three-dimensional coordinates of the defect, thereby realizing the mapping from the two-dimensional defect area to the three-dimensional model and ensuring the geometric accuracy and physical authenticity of the positioning results.

[0113] Optionally, based on the three-dimensional coordinates of the defect and the defect attributes, data comparison is performed between the target defect and the vehicle's historical defects to obtain comparison results, including: obtaining the historical defect's three-dimensional coordinates and historical defect attributes; performing spatial distance matching between the historical defect's three-dimensional coordinates and the vehicle's three-dimensional coordinates to obtain matching results, wherein the matching results are used to indicate whether the spatial distance between the historical defect's three-dimensional coordinates and the vehicle's three-dimensional coordinates is less than a preset distance threshold; and determining the comparison results based on the matching results, the defect attributes, and the historical defect attributes.

[0114] The aforementioned historical defects refer to damage objects on a vehicle that have been recorded and archived by the system in previous inspection cycles. Historical defects contain complete damage information, used for historical comparison with currently detected target defects. The existence of historical defects allows the system to track the evolution of the same damage point over time. The role of historical defects is to provide a benchmark for current inspections; by comparing them, new damage or changes in existing damage can be identified, thereby achieving continuous detection of damage status and lifecycle management.

[0115] The aforementioned historical defect 3D coordinates represent the spatial location data of historical defects within the vehicle's 3D coordinate system. These coordinates record the coordinate positions of the damage in previous detections, reflecting the physical distribution of the damage on the vehicle body. The historical defect 3D coordinates share the same coordinate system reference as the currently detected defect 3D coordinates, ensuring spatial comparability. The historical defect 3D coordinates serve as a basis for spatial matching, determining whether the currently detected defect is located in the same physical location as a historically recorded defect, and acting as a spatial indicator to determine whether the damage is new or expanded.

[0116] The aforementioned historical defect attributes are a set of feature information recorded in previous inspection cycles for historical defects. Historical defect attributes include quantitative parameters such as defect type, defect size, defect area, and defect level, describing the previous state of the damage. Historical defect attributes have the same semantic definition as the defect attributes obtained in the current inspection, ensuring data consistency. The purpose of historical defect attributes is to compare their values ​​with current defect attributes to quantify the degree of damage change, such as determining whether cracks have extended or increased in area, thereby assessing the evolution trend and degree of damage change.

[0117] The aforementioned spatial distance refers to the Euclidean distance measured along a straight line in the vehicle's preset coordinate system between the three-dimensional coordinates of the target defect and the three-dimensional coordinates of historical defects. Spatial distance is typically calculated based on the coordinate difference between two points, used to quantify the physical proximity of two defects. Spatial distance is the basis for determining whether a currently identified defect is a "new defect" or an "existing defect extension." If the spatial distance is less than a preset threshold, it is determined to be a defect at the same physical location, and further attribute comparison is used to determine whether an extension has occurred. If the spatial distance is greater than or equal to the preset threshold, it is determined to be an independent new defect.

[0118] The spatial distance matching described above is a process of calculating the geometric distance between the current 3D coordinates of a defect and the 3D coordinates of historical defects in 3D space. Spatial distance matching quantifies the proximity of two points in space by calculating the Euclidean distance between them. This process excludes the time factor and focuses only on the overlap of spatial locations. The purpose of spatial distance matching is to determine whether the currently detected target defect is located in the same physical location as a historical defect. It is a logical step to distinguish between newly added damage and the expansion of existing damage, providing a spatial basis for subsequent attribute comparison.

[0119] The matching results described above describe the results of the spatial distance matching logic. The matching result is typically a Boolean value, indicating whether the spatial distance between the historical defect's 3D coordinates and the defect's 3D coordinates meets a preset distance condition. The matching result indicates whether two defects are considered the same object in space. The purpose of the matching result is to transform continuous distance values ​​into discrete classification information, simplifying the subsequent logical judgment process and determining whether the system treats the current defect as an independent new defect or a continuation of a historical defect.

[0120] The Boolean values ​​or status codes mentioned above refer to discrete numerical signals used to characterize the system's operating results, whether a task was successfully executed, or the truth or falsehood of a specific logical judgment. Among them, Boolean values ​​are usually represented as "True / False" or "1 / 0", used to indicate simple yes / no judgments (such as whether a defect exists or whether a comparison is passed).

[0121] The aforementioned preset distance threshold is a critical distance value used to determine whether two 3D coordinate points represent the same physical location. The preset distance threshold is a pre-defined constant that defines the maximum allowable deviation range for spatial matching. It takes into account factors such as camera calibration error, model mesh discretization error, and measurement noise. The purpose of the preset distance threshold is to serve as a criterion for spatial matching, ensuring that the same damage point can still be accurately identified even with minor measurement errors, thus balancing the sensitivity and robustness of the matching process.

[0122] The comparison results described above represent the final diagnostic conclusion derived from comprehensive spatial location matching and attribute change analysis. The comparison results describe the relationship between the target defect and historical defects. The comparison results are the system's core judgment on the evolution of damage status, directly guiding subsequent annotation and report generation. The role of the comparison results is to transform raw detection data into diagnostic information, providing a direct basis for vehicle durability assessment and maintenance decisions.

[0123] In one optional embodiment, the vehicle defect localization system first retrieves historical defect records belonging to the same vehicle's Vehicle Identification Number (VIN) from the database. For each historical defect, the system calculates the Euclidean distance between the historical defect's three-dimensional coordinates and the currently detected defect's three-dimensional coordinates. If the calculated spatial distance is less than a preset distance threshold, the system determines a successful match, classifying the defect as spatially adjacent. If the calculated spatial distance is greater than the preset distance threshold, the match fails. Subsequently, for successfully matched defects, the system compares the current defect attributes with the historical defect attributes, calculating the attribute difference to determine if there is any expansion. For defects that fail to match, they are directly classified as newly added defects. Finally, the system summarizes the judgment results to form a complete comparison result list, including three states: newly added, expanded, and unchanged.

[0124] This application embodiment uses a method that combines spatial distance matching and attribute comparison to associate current defects with historical defects, accurately distinguishing the evolution of new damage from existing damage, and providing a reliable technical means for quantitative tracking of the entire life cycle of damage.

[0125] Optionally, the comparison result is determined based on the matching result, defect attributes, and historical defect attributes, including: in response to the matching result being that the spatial distance is less than a preset distance threshold, determining that the target defect is not a new defect, and determining the comparison result based on the difference between the defect attributes and historical defect attributes, and the fact that it is not a new defect; in response to the matching result being that the spatial distance is greater than or equal to the preset distance threshold, determining that the comparison result is that the target defect is a new defect.

[0126] The aforementioned non-newly discovered defects refer to a logical state where the spatial location of the currently detected target defect highly overlaps with defects already recorded in the historical database, thus being classified as the same damaged object. Non-newly discovered defects indicate that the damage did not occur within the latest testing cycle, but rather is an existing problem that has persisted throughout the vehicle durability testing process. The determination of non-newly discovered defects relies on the results of spatial distance matching, i.e., the current coordinates are extremely close to the historical coordinates. The purpose of non-newly discovered defects is to classify detection data from "newly discovered" to "old continuation," laying the foundation for subsequent analysis of damage evolution trends, avoiding duplicate counting of the same damage, and ensuring the accuracy of statistical data.

[0127] The aforementioned defect attribute difference refers to the mathematical difference between the currently detected defect attribute value and the historical defect attribute value. The defect attribute difference quantifies the physical change of damage in the latest period by calculating the difference between the current defect size, area, or depth and the corresponding historical parameters. The defect attribute difference can be positive, negative, or zero, representing damage expansion, contraction, or stabilization, respectively. The role of the defect attribute difference is to reflect the dynamic evolution of damage, serving as a key quantitative indicator for determining whether damage has worsened or needs repair, and providing data support for assessing the degree of degradation in vehicle structural durability.

[0128] The aforementioned newly identified defects refer to target defects whose spatial location does not overlap with records in the historical database, and are therefore determined to be damage objects appearing for the first time in the latest inspection cycle. Newly identified defects indicate that the damage is a newly occurring structural or cosmetic failure of the vehicle. The determination of newly identified defects depends on spatial distance matching failure, i.e., the distance between the current coordinates and the nearest historical coordinates exceeds a preset threshold. The purpose of newly identified defects is to mark new failure points in vehicle durability, serve as a data source for assessing vehicle design weaknesses and improvement directions, and also trigger a new round of maintenance or structural analysis.

[0129] In one optional embodiment, the vehicle defect localization system first receives spatial distance matching results. If the matching result shows that the spatial distance is less than a preset distance threshold, the target defect is determined to be a non-new defect, and historical defect attributes are retrieved. Subsequently, the vehicle defect localization system calculates the numerical difference between the current defect attribute and the historical defect attribute. If the difference exceeds a preset attribute change threshold, the comparison result is determined to be a non-new defect with defect expansion. If the difference is within the threshold range, it is determined to be a non-new defect with no defect expansion. If the matching result shows that the spatial distance is greater than or equal to the preset distance threshold, the comparison result is directly determined to be a new defect, without the need for attribute difference calculation, thus completing the complete classification of the current defect status.

[0130] This application embodiment achieves accurate differentiation between new and extended target defects by using classification logic based on spatial distance thresholds and refined judgment of attribute differences, ensuring the logical rigor of damage tracking data and providing reliable data support for dynamic assessment of vehicle durability.

[0131] Optionally, the comparison result is determined based on the difference between the defect attribute and the historical defect attribute, as well as the non-new defect, including: in response to the difference being greater than a preset attribute difference threshold, determining that the comparison result is that the target defect is a non-new defect and that the target defect has defect expansion; in response to the difference being less than or equal to the preset attribute difference threshold, determining that the comparison result is that the target defect is a non-new defect and that the target defect does not have defect expansion.

[0132] The aforementioned preset attribute difference threshold is a numerical limit used to determine whether the defect state has changed. The preset attribute difference threshold defines the maximum allowable natural fluctuation range or measurement error range of defect attributes between adjacent inspection cycles. This threshold is typically set based on historical data statistical analysis or engineering experience, aiming to distinguish between genuine physical damage evolution and normal measurement noise. The function of the preset attribute difference threshold is to serve as a standard for logical judgment. If the preset attribute change exceeds the preset attribute difference threshold, it is considered that the damage has substantially expanded or worsened, thereby avoiding misjudging minor data fluctuations as defect expansion and improving the reliability of the comparison results.

[0133] The aforementioned defect propagation refers to the objective phenomenon where the physical size, area, depth, or severity of a target defect increases compared to its historical state within the latest testing cycle. Defect propagation reflects the worsening trend of damage during vehicle durability testing and is an important indicator for evaluating structural fatigue life and safety. Defect propagation typically manifests as increased crack length, larger areas of paint peeling, or wider gaps. The role of defect propagation is to qualitatively describe the dynamic evolution of damage, alerting maintenance personnel or engineers to the structural integrity risks at that location, and serving as a key basis for developing maintenance strategies and conducting failure analyses.

[0134] In one optional embodiment, the vehicle defect localization system first calculates the numerical difference between the currently detected defect attribute and historical defect attributes. Then, the system compares this numerical difference with a preset attribute difference threshold. If the calculated difference is greater than the preset threshold, the system determines that the target defect has expanded, generating a comparison result indicating a non-new defect with defect expansion. If the calculated numerical difference is less than or equal to the preset threshold, the system determines that the target defect has not undergone substantial change, generating a comparison result indicating a non-new defect with no defect expansion. Through this logical judgment, the system achieves automated classification of damage evolution states.

[0135] This application embodiment introduces a preset attribute difference threshold to achieve quantitative determination of the expansion state of non-new defects, effectively distinguishes between real damage evolution and measurement noise, improves the accuracy and reliability of defect tracking analysis, and provides data support for the dynamic evaluation of vehicle durability.

[0136] Optionally, based on the comparison results, the vehicle defect is located on the three-dimensional model of the vehicle body to obtain the vehicle defect location result, including: determining the target location information of the target defect based on the comparison results; and locating the target defect on the three-dimensional model of the vehicle body based on the target location information to obtain the vehicle defect location result.

[0137] The aforementioned target location information is a digital data structure used to accurately annotate the location and state of target defects on the vehicle's 3D model. This information includes the defect's 3D coordinates, defect type, defect attributes, defect level, and an evolutionary state identifier determined based on comparison results. The target location information transforms abstract comparison conclusions into visualized model annotation data. Its purpose is to provide the rendering engine with all the parameters required for drawing, ensuring that the defect's display position on the 3D model is accurate, its attributes are rich, and its state is correct, allowing users to intuitively view damage details.

[0138] The aforementioned 3D vehicle body model is a digital 3D representation of the vehicle's external geometry. As the carrier and background for localization operations, the 3D vehicle body model provides the geometric reference framework required for defect annotation. It records the topological structure and spatial coordinate system of the vehicle surface, enabling accurate projection of 2D inspection data into 3D space. The role of the 3D vehicle body model is to provide a spatial benchmark for defect localization, ensuring that defect annotations conform to the actual geometry of the vehicle and supporting multi-angle observation and analysis.

[0139] In one optional embodiment, the vehicle defect localization system first receives the comparison results. If the comparison results show that the target defect is a new defect, the vehicle defect localization system generates target localization information containing highlighted and new defect markers based on the current defect attributes. If the comparison results show that the target defect is not a new defect but has been extended, the vehicle defect localization system generates target localization information containing extended defect markers and update level markers based on the updated defect attributes. Subsequently, the vehicle defect localization system maps the three-dimensional coordinates in the target localization information onto the corresponding triangular facets of the vehicle body's three-dimensional model, and renders the corresponding graphic and text labels at the aforementioned locations. Finally, the system summarizes the defect annotation information to form the vehicle defect localization result, completing the process from data calculation to visualization.

[0140] The aforementioned triangular facet refers to the smallest geometric unit after discretizing the surface of the three-dimensional CAD model of the vehicle body, which is a polygonal plane composed of three vertices and the three sides connected to them.

[0141] This application embodiment dynamically generates target location information based on the comparison results and accurately maps it to the three-dimensional model of the vehicle body, realizing the visual expression of the defect status, improving the intuitiveness and usability of the defect location results, and helping engineers to efficiently conduct damage assessment and decision-making.

[0142] Optionally, based on the comparison results, the target location information of the target defect is determined, including: in response to the comparison results indicating that the target defect is a non-new defect and that the target defect has defect extension, the defect level of the target defect is determined according to the defect extension information of the defect extension; and the original location information of the target defect on the vehicle body three-dimensional model is updated based on the defect level to obtain the target location information.

[0143] The aforementioned defect extension information is a quantitative dataset describing the physical state changes of non-newly added defects within the latest testing cycle. Defect extension information includes differences in defect attributes and descriptions of the resulting evolution trends. It reflects the degree to which damage worsens with testing mileage or time, and is a key basis for assessing structural safety degradation. The purpose of defect extension information is to provide a data foundation for re-determining defect levels, ensuring that updates to defect levels accurately reflect the current degree of damage, rather than simply adopting outdated conditions.

[0144] The aforementioned defect levels are grading indicators used to characterize the degree of impact of a target defect on the structural or aesthetic integrity of a vehicle. Defect levels are typically categorized as minor, moderate, severe, or urgent, with different levels corresponding to different repair priorities and handling strategies. The defect level is a result of a comprehensive assessment based on defect attributes and expansion trends. The purpose of the defect level is to transform complex physical parameters into intuitive indicators, guiding the allocation of repair resources and adjustments to testing schedules.

[0145] The aforementioned original location information is static labeled data representing the historical records of the target defect in the database or model. This original location information includes the defect's initial 3D coordinates, initial defect type, initial defect attributes, and initial defect level. It represents a snapshot of the defect's state at the time of its first discovery or at the end of the previous detection cycle. The original location information serves as the reference object for update operations. If the defect's state changes, the vehicle defect location system modifies the system based on the original location information, ensuring data version continuity and traceability, and preventing data loss or corruption.

[0146] In one optional embodiment, the vehicle defect localization system first identifies defects that are not newly added but have been extended, and extracts the corresponding defect extension information. Then, the system invokes a defect level determination rule engine, inputting the updated defect attributes to calculate a new defect level. Next, the system retrieves the original location information of the defect on the vehicle's 3D model, updates the stored defect level field, updates the defect attribute values ​​to the latest values, and adds an extension identifier. Finally, the generated structured data containing the updated information becomes the new target location information, used for subsequent model rendering and report generation.

[0147] This application embodiment dynamically updates the defect level based on defect extension information and corrects the original location information on the model accordingly, realizing real-time synchronization and visualization of defect status on the three-dimensional model, ensuring the timeliness and accuracy of detection data, and providing accurate status feedback for the dynamic assessment of vehicle durability.

[0148] Optionally, based on the comparison results, the target location information of the target defect is determined, including: in response to the comparison results indicating that the target defect is a non-new defect and that the target defect does not have defect extension, the original location information of the target defect on the vehicle body three-dimensional model is determined as the target location information.

[0149] In one optional embodiment, the vehicle defect localization system first receives the comparison results and determines that the current target defect is not a newly added defect and there is no defect expansion. The system then retrieves the historical identifier corresponding to the defect from the database and reads the stored original location information, including 3D coordinates, defect type, defect attribute values, and current defect level. After confirming that no new level needs to be calculated or attributes updated, the system directly assigns the read original location information to the current target location information. Subsequently, the system sends this unchanged target location information to the visualization module, rendering the defect on the vehicle's 3D model with the original style and color, thus completing the localization process and achieving efficient data processing and accurate reproduction of historical states.

[0150] This application embodiment identifies non-new defects without expansion and directly reuses the original location information, avoiding redundant data updates and computational overhead, ensuring the stability and consistency of defect location data, and improving the efficiency and reliability of the system when processing large-scale data.

[0151] Optionally, based on the comparison results, the vehicle is located on a 3D model of the vehicle body to obtain the vehicle defect location results, including: in response to the comparison results indicating that the target defect is a newly added defect, the target location information is determined based on the defect attributes and defect area of ​​the target defect.

[0152] In one optional embodiment, the vehicle defect localization system first receives the comparison results and determines that the current target defect is a newly added defect. The system then extracts the defect attributes and defect area of ​​the target defect. Based on the defect attributes, the system calls a rule engine to determine the initial defect level. Next, the system combines the defect's 3D coordinates to construct structured target localization information containing coordinates, attributes, level, and a new identifier. Finally, the system writes the target localization information into a database and renders new annotations at the corresponding positions on the vehicle's 3D model, thus completing the entire process of locating the newly added defect.

[0153] This application embodiment generates target location information directly based on the attributes and regions of the target defect, enabling rapid capture and visualization of newly added defects. This ensures accurate recording and immediate presentation of newly discovered damage, providing support for vehicle durability monitoring and early warning.

[0154] Optionally, the method further includes: determining the vehicle's defect change trend and defect change parameters based on the vehicle defect location results; and constructing a vehicle defect detection report based on the defect change trend and defect change parameters.

[0155] The aforementioned defect change trends refer to the dynamic evolution of vehicle damage conditions as test mileage, time, or inspection cycles progress. Defect change trends typically manifest as increases or decreases in the number of damages, escalations in damage severity, or a concentration of damage frequency in specific areas. These trends reflect the degradation process and failure modes of the vehicle structure during durability testing. The purpose of defect change trends is to provide engineers with a macroscopic perspective on durability assessment, helping to identify weaknesses in vehicle design, predict remaining lifespan, and provide data support for improving the vehicle body structure.

[0156] The aforementioned defect variation parameters are specific numerical indicators used to quantify the degree of defect evolution. These parameters include statistics such as the number of new defects, the number of expanding defects, the average defect area growth rate, the maximum defect depth increment, and defect density distribution. These parameters transform qualitative trend descriptions into quantitative data, enabling horizontal and vertical comparisons of damage conditions across different vehicles and test cycles. The purpose of these parameters is to provide accurate calculations for trend analysis, ensuring the objectivity and scientific rigor of the evaluation results.

[0157] The aforementioned defect detection report is a comprehensive document generated based on defect change trends and parameters. It typically includes sections such as basic vehicle information, a detection overview, damage distribution maps, trend charts, a detailed defect list, and repair recommendations. The defect detection report can be directly used for project presentations, technical reviews, and archiving. Its purpose is to transform complex, low-level inspection data into easily understandable decision support information, showcasing the vehicle's health status during testing and guiding subsequent maintenance work and design improvements.

[0158] In one optional embodiment, the vehicle defect location system first extracts historical and current defect data from the vehicle defect location results. The system calculates the number of new and expanding defects for each inspection cycle, and plots a line graph showing the defect count as a function of mileage to determine the defect growth rate. Simultaneously, the system statistically analyzes the defect density at different locations, generating a heatmap to display areas of concentrated damage. Subsequently, the system embeds the statistical results and charts into a predefined report template, generating a defect detection report containing text descriptions, charts, and data analysis conclusions. This report can be automatically associated with the vehicle's VIN code and test mileage and stored in a durability testing database for user access and download.

[0159] This application embodiment extracts key parameters from the positioning results and analyzes the evolution trend to construct a structured defect detection report, realizing a leap from single-point detection data to systematic durability assessment, improving the value and usability of detection data, and providing strong data support for vehicle research and development.

[0160] This application discloses an automatic damage labeling system and method for vehicle body testing, the method comprising:

[0161] The full-vehicle vision acquisition module acquires multi-view images of the vehicle's exterior surface and camera pose parameters during multiple testing cycles of the durability test. The acquired images are then input into the damage recognition artificial intelligence (AI) module, which uses a deep learning model to identify the damage type, damage area, and damage level in the images, and outputs the damage recognition results.

[0162] The 3D coordinate mapping module, combined with camera calibration parameters and the vehicle CAD model, maps the damage area in the 2D image to the vehicle's 3D coordinate system, generating the 3D spatial coordinates of the damage. The baseline comparison module registers and compares the image from the current detection cycle with the baseline image established before the test, identifying new damage and the changing trends of existing damage.

[0163] The automatic annotation and report generation module assigns a unique identifier (Damage Unique ID) to each damage instance, generating a digital inspection report that includes a damage comparison diagram, 3D model annotations, damage parameters, and evolution curves. Through the data platform interface module, the inspection report is linked to the vehicle VIN code, test mileage, and road conditions and stored in the durability test database.

[0164] This application embodiment achieves quantitative recording of the damage evolution process through a baseline comparison module, achieves digital positioning of the damage location through a three-dimensional coordinate mapping module, and forms a fully automated closed loop from acquisition, identification, positioning, tracking to data management through the collaboration of multiple modules.

[0165] This application embodiment constructs a complete closed-loop system integrating test data from full-vehicle image acquisition, three-dimensional coordinate mapping, AI damage recognition, time-based comparison tracking, and automatic report generation, rather than a fragmented single-function design.

[0166] This application embodiment integrates a multi-view visual acquisition module, an AI recognition module, and an automatic report generation module to automate the entire process from vehicle entry into the inspection station to report output, eliminating the need for manual intervention. This embodiment reduces the time for a single full-vehicle inspection from several hours to several minutes, improving inspection efficiency and avoiding missed detections and misjudgments caused by human fatigue.

[0167] This application embodiment utilizes a three-dimensional coordinate mapping module to accurately map damage in a two-dimensional image to the vehicle's three-dimensional coordinate system, generating millimeter-level precision spatial coordinates and IDs for each damage. This application embodiment achieves the digital and computable recording of damage locations, supporting accurate reproduction and full-vehicle damage distribution analysis in any subsequent period. For example, the text description could be "below the left front door."

[0168] This application embodiment utilizes a baseline comparison module to automatically compare the damage states of the same vehicle across different test cycles, accurately identifying new damage and the dimensional expansion of existing damage, such as crack extension and increased spalling area, and quantifying the damage evolution process as a trend curve. This application embodiment solves the problem of being unable to dynamically track damage development, providing crucial process data for durability failure analysis.

[0169] This application embodiment uses a data platform interface module to upload structured test data to the test database and automatically associates it with metadata such as test mileage, road type, and operating conditions, forming a full lifecycle durability damage database centered on the Vehicle Identification Number (VIN). This application embodiment provides data-driven decision support for vehicle body structure improvement and test plan development.

[0170] This application embodiment accurately maps the damaged areas identified in the two-dimensional image to the three-dimensional coordinate system of the body-in-white CAD model through camera calibration and ray intersection algorithms, outputting spatial coordinates with millimeter-level accuracy, realizing the leap from textual description to calculable digital coordinates of the damage location. This application embodiment establishes a baseline image before the test, and combines image registration, differential analysis, and AI secondary confirmation to achieve automatic comparison of the damage status of the same vehicle across multiple inspection cycles. It can quantitatively record the evolution process such as crack extension and peeling area increase, and generate trend curves. This application embodiment employs multiple dedicated deep learning networks for parallel processing, identifying different types of damage such as cracks, paint defects, plastic loosening, and deformation. It also integrates a fully automated process link to connect inspection data with test mileage and operating condition information, forming a traceable full lifecycle durability damage database.

[0171] The core of the full-vehicle vision acquisition module in this application lies in the collaborative acquisition and high-precision calibration of multi-source vision data. Its implementation requires comprehensive consideration from three aspects: mechanical installation, optical parameter calibration, and electrical integration.

[0172] In terms of mechanical structure, this application provides three optional acquisition schemes for different test site conditions, which can be used individually or in combination: a fixed multi-camera surround system, a fixed-point photography station, and a mobile surround photography robot.

[0173] A fixed multi-camera surround system solution. Multi-source vision acquisition units are distributed around the inspection station, covering the entire surface of the vehicle through a preset spatial geometry layout. The shooting angle is dynamically adapted by using an environmentally adaptable protective structure and an adjustable gimbal mechanism. The system is interconnected with the processing unit through a high-speed communication network to achieve synchronous acquisition of multi-node images.

[0174] For example, six industrial-grade high-definition cameras are deployed around the testing station, specifically as follows: 45° front left, 45° front right, left front, right front, 45° rear left, and 45° rear right. The cameras are installed 1.5m above the ground, with the lens center 1.8m from the vehicle's outer edge. The cameras use aluminum alloy housings with an IP65 protection rating, suitable for outdoor testing environments. The camera pan / tilt head has motorized tilt and swivel functions, with a tilt adjustment range of ±15° and a horizontal swivel range of ±10°, used for fine-tuning the shooting angle to ensure compatibility with different vehicle models. All cameras are connected to the server via an Ethernet switch for synchronized triggering.

[0175] Fixed-point photography station solution. The vehicle positioning and image acquisition module adopts a high-precision mechanical constraint and automated robot collaborative operation mechanism. Micrometer-level repeatability constraints on vehicle parking positions are achieved through ground wheel positioning grooves, ensuring spatial benchmark consistency. A six-axis collaborative robot, fixed by pre-embedded bolts, is equipped with high-definition visual imaging and line structured light depth sensing sensors. It performs automated multi-angle scanning operations according to a preset multi-dimensional spatial path (covering multiple points at different heights on the left and right sides and front and rear areas). While ensuring high efficiency in single-pass full-vehicle acquisition, the rigid base fixing structure ensures the mechanical stability and data acquisition accuracy of the equipment under long-term high-frequency operation.

[0176] For example, after the vehicle enters the designated workstation, four wheel positioning slots are installed on the ground to ensure that the accuracy of each parking position is within ±10mm. Two six-axis collaborative robots are positioned on both sides of the workstation, with high-definition cameras and line structured light depth sensors mounted at their end effectors. The robots are pre-set with 12 shooting points (two each at heights for the front left, center left, rear left, front right, center right, and rear right), and a single full-vehicle shot takes approximately 90 seconds. The robot base is secured with pre-embedded anchor bolts to ensure long-term operational stability.

[0177] A mobile surround-viewing robot solution. The mobile data acquisition unit utilizes a robot platform with autonomous navigation capabilities, integrating LiDAR and vision technology to achieve adaptive path planning and dynamic obstacle avoidance around the vehicle. The chassis employs a four-wheel independent drive structure, possessing high mobility and high-precision positioning capabilities, and is equipped with a vertically lifting gimbal. By adjusting the camera height, blind spots are eliminated, thereby achieving comprehensive coverage and data acquisition of complex areas such as the inner sides of the vehicle's wheel arches and the roof.

[0178] For example, an autonomous navigation mobile robot platform with dimensions of 800mm × 600mm × 400mm is used, equipped with a lifting gimbal that can rise to 1.8m. The robot is equipped with LiDAR and a Visual Simultaneous Localization and Mapping (Visual SLAM) system, enabling autonomous path planning and obstacle avoidance around the vehicle. The robot chassis uses four-wheel independent drive, with a maximum movement speed of 0.5m / s and a positioning accuracy of ±20mm. The lifting gimbal can rise vertically from 0.5m to 1.8m, ensuring coverage of the inner wheel arches and the roof area.

[0179] In the optical system, the image acquisition module employs a global shutter sensor, featuring high resolution and high frame rate characteristics to ensure image clarity and capture capability in dynamic scenes. The optical system can be flexibly configured according to the deployment scenario; fixed systems use fixed-focus lenses to provide a fixed field of view, while mobile systems employ motorized zoom lenses with autofocus support to adapt to shooting needs at different distances and angles. Furthermore, a polarizing filter is integrated into the front of the lens to effectively suppress glare interference from highly reflective surfaces such as car paint, improving image contrast and the accuracy of defect identification.

[0180] For example, the camera uses a global shutter complementary metal-oxide-semiconductor (CMOS) sensor with a resolution of at least 20 megapixels and a frame rate of 30fps. The lens focal length is selected based on the installation distance: a 16mm fixed-focus lens with a field of view of approximately 40° is used for the fixed surround system; an 8-35mm motorized zoom lens with autofocus is used for the mobile robot. A polarizing filter is added to the front of the lens to eliminate high-gloss reflections from the car paint surface.

[0181] In terms of electrical integration, the data acquisition system is equipped with an independent industrial control computer. The industrial control computer synchronizes the camera trigger signal via Precision Time Protocol (PTP), achieving a synchronization accuracy better than 1ms. The system reserves a Controller Area Network (CAN) interface for reading vehicle VIN codes and test mileage information.

[0182] In the camera calibration process, camera calibration is the foundation for subsequent 3D coordinate mapping. This application's embodiment adopts a two-step calibration method, that is, a combination of intrinsic parameter calibration and extrinsic parameter calibration.

[0183] In intrinsic parameter calibration, after the initial camera installation or replacement, a checkerboard calibration board is used for calibration. The camera captures multiple images of the calibration board from different angles, and the focal length, principal point coordinates, and distortion coefficients are calculated using camera calibration methods. The calibration results are stored in the camera configuration file, with a repeatability better than 0.1 pixels. For example, the checkerboard calibration board can have 12×9 corner points with a grid spacing of 30mm.

[0184] In extrinsic parameter calibration, for fixed systems, multiple coded markers with known 3D coordinates are placed on the workstation floor. By capturing images of these markers, the rotation matrix R and translation vector T of the camera relative to the world coordinate system are calculated. For mobile robot systems, a calibration target sphere placed on the vehicle is scanned by a LiDAR scanner to establish the transformation relationship between the camera and the LiDAR coordinate system. For example, the diameter of the calibration target sphere can be 100mm, and its spatial coordinates can be known.

[0185] In the vehicle body 3D model and coordinate mapping module, the system supports importing 3D models in mainstream CAD formats. After importing the model, the system automatically resolves the vehicle body node coordinate system, typically using the whole vehicle coordinate system, where the positive X direction is the front direction, the positive Y direction is the driver's side, and the positive Z direction is vertically upward. The origin is located at the center of the front axle or the vehicle's geometric center.

[0186] For the body-in-white model and the outer body panel model, the system uses triangular facet processing to ensure smooth model surfaces and moderate computational efficiency. Each triangular facet stores vertex coordinates and normal vector information for subsequent ray intersection calculations. For example, the facet side length can be controlled between 5-10mm.

[0187] Given the pixel coordinates of the damage point in the image Its corresponding camera coordinate system coordinates Calculated using the following formula:

[0188] ;

[0189] Where K is the camera intrinsic parameter matrix, and d is the depth value (which can be obtained from a depth camera or estimated through image matching). Then, the extrinsic parameters are used to... Transform to world coordinate system The calculation formula is as follows:

[0190] ;

[0191] in, Rotation matrix, This is the translation vector. During the intersection of the ray and the CAD model, starting from the camera's optical center, along the direction vector... Emit a ray and find its intersection with all triangles of the 3D model. Let be the optical center of the camera. A ray-triangle intersection algorithm is employed, which is computationally efficient and numerically stable. The first intersection point between the ray and the model surface is the mapping point of the damage on the 3D model.

[0192] For the damaged area (not a single point), the contour point set of the damage mask is extracted and mapped to the three-dimensional model to form the contour of the damaged area in three-dimensional space.

[0193] The coordinate output format for each damage record includes the coordinates (X, Y, Z) of the damage center point relative to the nearest critical vehicle body node. Damage localization data is standardized using a multi-dimensional coordinate system to ensure the accuracy and traceability of spatial information. That is, the coordinate output includes spatial coordinates (X, Y, Z) to provide a global spatial reference. Simultaneously, it records relative vector coordinates to critical structural nodes of the vehicle body to visually represent the geometric relationship of the damage within the local structure. Furthermore, it maps the texture coordinates (U, V coordinates, UV) of the damaged area onto the surface of the 3D model, supporting texture-based visualization annotation and rendering, thereby achieving a complete expression of multi-dimensional damage spatial information from localization and relative reference to surface mapping.

[0194] For example, "150mm from the root of the left A-pillar in the X direction and 80mm from the Z direction".

[0195] In the damage identification phase of the AI ​​module, the network architecture and training data are first determined. The AI ​​module adopts a multi-task learning framework, sharing the underlying feature extraction network, while the top-level branches output the identification results for different damage types.

[0196] The underlying feature extraction network can use a deep convolutional neural network (Deep CNN) as the backbone network, with the input image size and feature map size being preset. The neural network backbone network is pre-trained on the ImageNet dataset and then transferred to the damage recognition task. For example, the preset size of the input image can be 512×512 pixels, and the feature map size can be 16×16×2048.

[0197] In the crack detection model (CrackNet) stage, a semantic segmentation network is used to segment the viewpoint image into pixel-level defect regions, generating a defect mask. The semantic segmentation network employs an encoder-decoder structure and extracts image features through a multi-scale feature fusion mechanism. During model training, a combined loss function is used to improve the class imbalance problem. The combined loss function includes the intersection-union ratio loss or set overlap loss to measure the overlap between the predicted and ground truth regions, and the cross-entropy loss to measure pixel-by-pixel classification accuracy. Based on the defect mask, the geometric topological features of the defects are extracted, and the size parameters of the defects are calculated.

[0198] A U-Net architecture is employed, where the input feature map is upsampled and output as a crack segmentation mask of the same size as the input. The training data contains a large number of labeled crack images with pixel-level annotation accuracy. A combination of set overlap loss and binary cross-entropy is used as the loss function to address class imbalance caused by the small pixel proportion of cracks. After the model outputs the crack mask, a skeleton extraction algorithm is used to calculate the crack length. For example, the length error can be controlled within ±2mm.

[0199] The PaintDefect-Seg model employs a semantic segmentation deep learning architecture, utilizing the Atrous Spatial Pyramid Pooling (ASPP) module to capture multi-scale contextual information. The training data contains a large number of labeled paint defect images, including peeling, scratches, bulges, and corrosion. After outputting a defect region mask, the pixel area is calculated and converted to the actual area (using camera calibration parameters). Damage levels are categorized as minor, moderate, and severe based on area.

[0200] The LoosePart-RCNN model for detecting loose plastic parts employs a fast region convolutional neural network architecture, with a Region Proposal Network (RPN) generating candidate regions. The training data includes numerous images of annotated loose parts, with annotations including part category, bounding box, and gap width. After the model outputs bounding boxes, edge detection algorithms accurately locate the parts' edges and calculate the gap width. For example, the looseness threshold is a gap > 2mm or a change from the baseline state > 1mm.

[0201] The irregular surface deformation detection model (ShapeDiffNet) employs a deep learning architecture to process point cloud data, with input being 3D point clouds acquired by a depth camera. The model first extracts local geometric features, then uses a classification network to determine the presence of deformation, and finally outputs the deformation depth through a regression network. The training data contains a large amount of point cloud data.

[0202] In this embodiment, the model deployment and inference improvement strategy of the damage recognition AI module aims to achieve efficient real-time data processing capabilities. Specifically, by converting the trained deep learning model into a format supported by the high-performance inference engine and using half-precision floating-point operations for quantization acceleration, computational resource consumption is reduced and throughput is increased. On a computing terminal equipped with a high-performance graphics processing unit, millisecond-level fast inference for single-view images is achieved, and concurrent inference of images acquired simultaneously by multiple cameras is supported through a parallel processing mechanism, thereby meeting the real-time analysis needs of high-frequency, high-volume visual data in vehicle durability testing. In addition, to ensure the generalization ability and detection accuracy of the model during long-term operation, the system constructs a dynamic update mechanism, adopting incremental learning or periodic fine-tuning strategies, and continuously iterating and improving the model using newly acquired and manually verified high-quality labeled data, ensuring that the damage recognition algorithm can adapt to changes in different vehicle models, different lighting conditions, and different damage morphologies, maintaining the robustness and accuracy of the system throughout its entire lifecycle.

[0203] In the baseline comparison module stage, the prerequisite for baseline comparison is accurate alignment of images of the same area in different periods. This system adopts a multi-level registration strategy, that is, a combination of coarse registration and fine registration.

[0204] In coarse registration, the current image and the baseline image are initially aligned by projection transformation based on camera extrinsic parameters and shooting pose. In fine registration, based on coarse registration, scale-invariant feature transform (SIFT) feature points are extracted and matched. The homography matrix is ​​estimated using the random sample consensus (RANSAC) algorithm to achieve sub-pixel-level image alignment accuracy. For regions with sparse texture, a deep learning-based image alignment network is used for dense registration.

[0205] After image registration, the structural similarity index needs to be calculated pixel by pixel. The calculation formula is as follows:

[0206] ;

[0207] in, Representing an image The average brightness (mean) is used to reflect the image. The overall light intensity level. Representing an image The average brightness (mean) is used to reflect the image. The overall light intensity level. image and images The covariance between them. Used to represent an image The square of the average brightness. Used to represent an image The square of the average brightness. Reflects the correlation of brightness changes in local areas between two images, i.e., the similarity of structural information. image The variance of x represents the fluctuation of pixel values ​​in image x, reflecting the contrast or texture richness of the image. image The variance of the image The fluctuation of pixel values ​​reflects the contrast or texture richness of an image. This is a constant for brightness comparison. A constant for contrast comparison. Pixels with SSIM values ​​below a threshold are marked as variation regions. For the depth map, a depth difference map is calculated, and regions with differences greater than a preset distance are marked as potential deformations. The variation regions of the three primary colors (Red, Green, Blue, RGB) are fused with the depth variation regions to generate a variation mask. The threshold can be 0.85, and the preset distance can be 2mm.

[0208] In the environmental noise filtering stage, the change mask contains a large number of pseudo-changes caused by dust, dirt, water stains, and changes in lighting. The noise filtering scheme adopted in this application embodiment first performs morphological filtering, then performs AI secondary verification, and finally performs temporal filtering.

[0209] In the morphological filtering stage, isolated variation regions with an area smaller than a preset area are removed. In the AI ​​secondary verification, the variation regions are cropped and sent to the damage recognition AI module, retaining only the variations that the AI ​​model identifies as true damage. For example, the preset area could be 10 mm².

[0210] During the time-domain filtering stage, areas of change that appear in multiple consecutive detection cycles are identified as actual damage, while areas of change that appear only once are marked as suspected damage and await confirmation in the next cycle.

[0211] In the automatic labeling and report generation module, an encoding structure containing at least one preset field is constructed, with each preset field corresponding to a different attribute dimension of the damage. Based on the currently detected damage features, the vehicle model identifier is mapped to the corresponding first field, the body part identifier to the corresponding second field, the cumulative damage count to the corresponding third field, and the detection time information to the corresponding fourth field. Finally, the mapping results of each field are combined according to preset splicing rules to generate a damage identifier. This damage identifier is immutable throughout the vehicle's entire lifecycle and is used to associate and store historical detection data and evolution records of the damage to support the traceability and comparative analysis of damage status.

[0212] In the ID generation rules, damage identification adopts structured coding rules. By combining vehicle type code, spatial location code, sequence identifier, and timestamp information, a readable digital identifier is generated to support the full lifecycle traceability and management of damage data. For example, the damage ID can use an 18-bit encoding format, with a 4-digit vehicle type code, a 4-digit location code, a 6-digit sequence number, and a 4-digit discovery date.

[0213] The report generation module uses a template-based design, with the output format including a cover page (vehicle model, VIN code, inspection mileage, inspection date), a damage overview table showing the number of new damages in the current period, the cumulative number of damages, and the distribution of each type, and a 3D model annotation map showing the damage locations marked with color blocks on the vehicle body model. The damage details page is a separate page for each damage, including comparison images, 3D coordinates, damage parameters, and trend curves. Repair suggestions are automatically generated based on damage type and severity.

[0214] During trend curve generation, for each damage ID, the system records the parameter changes (area, length, depth) across each detection cycle. The trend curve is plotted with detection mileage on the x-axis and damage parameters on the y-axis, smoothed using cubic spline interpolation. The curve is embedded in the report, and a warning is displayed when parameters exceed preset thresholds.

[0215] In the durability test data platform interface module, the application programming interface (API) design adopts the Representational State Transfer Application Programming Interface (RESTful API) design specification.

[0216] During data association, when uploading a report, the system automatically retrieves the following data from the test management platform: Test mileage data is read from the vehicle's odometer via the CAN bus. Road type data is obtained by matching the test track map with the Global Positioning System (GPS) trajectory.

[0217] For example, operating condition information includes full load / empty load, cold engine / hot engine. Driver information is used to exclude human factors. After the above data is correlated, it is stored in the database, supporting multi-dimensional querying, analysis, installation, testing, and verification.

[0218] During the system deployment phase, the process encompasses the entire workflow, including mechanical infrastructure construction, hardware integration and installation, parameter calibration, and software algorithm configuration. For fixed acquisition solutions, a stable mechanical base is installed using a high-precision concrete foundation and pre-embedded anchor bolts. The spatial attitude of the camera gimbal is calibrated to complete the internal and external parameter calibration, and finally, industrial control hardware and AI inference software are deployed. For mobile acquisition solutions, a navigation map is built based on LiDAR environmental perception to plan the detection path, automatic charging facilities are configured to enable autonomous device return, and navigation parameters and shooting points are set, thereby completing the system integration and debugging of the mobile robot.

[0219] For example, in the installation of a fixed system, the foundation construction involves pouring a concrete foundation at the testing station and pre-embedding anchor bolts with an accuracy of ±2mm. The camera gimbal is then fixed to the foundation, and the horizontal and vertical alignment is adjusted to an installation accuracy of ±0.5°.

[0220] Perform internal and external parameter calibration. Install the industrial control computer, configure the network, and deploy the AI ​​model.

[0221] During mobile robot deployment, LiDAR scans the workstation environment, creates a map, and plans the detection path. Automatic charging stations are installed next to the workstation, allowing the robot to automatically return to its charging position after completing its inspection. Robot navigation parameters and gimbal camera positions are configured.

[0222] To ensure the system's reliability and effectiveness, this application establishes a verification standard, which means that the system sets multi-dimensional performance verification indicators, covering the recall and precision thresholds for damage identification, the measurement error range of geometric dimensions (such as crack length, paint area, and loose gaps), and the quantitative accuracy of deformation depth. This ensures that the defect detection algorithm has high accuracy and precision under various damage types, meeting the requirements for detection reliability and quantitative analysis capabilities.

[0223] For example, in crack detection, the recall rate is ≥95%, the precision is ≥92%, and the length error is ≤±2mm. For paint defects, the Intersection over Union (IoU) is ≥0.85, and the area error is ≤±5%. In looseness detection, the recall rate is ≥93%, the precision is ≥90%, and the gap error is ≤±0.5mm. In deformation detection, the depth error is ≤±0.3mm. Coordinate mapping accuracy testing involves attaching a target (with known three-dimensional coordinates) to the vehicle surface. After the system identifies and maps the target, the deviation between the mapped coordinates and the true value is calculated, with an RMS error ≤±2mm.

[0224] Repeatability testing involves multiple consecutive inspections of the same vehicle within the same inspection cycle, with the repeatability (coefficient of variation) of the damage identification results ≤ a preset threshold. For example, if 5 consecutive inspections are conducted within the same inspection cycle, the preset threshold could be 2%. Durational comparison testing involves artificially creating new damage on the vehicle surface; the system should be able to identify and mark this as new damage without false alarms. System response time is from the time the vehicle enters the workstation to the time the report is generated. For example, the total time for a fixed system should ≤ 5 minutes, and the total time for a mobile robot system should ≤ 8 minutes.

[0225] Figure 2 This is a schematic diagram of the overall composition of a vehicle defect location method system according to an embodiment of this application, as shown below. Figure 2As shown, the system first acquires multi-view images of the vehicle's exterior surface and camera pose parameters through a full-vehicle vision acquisition module, providing the raw data foundation for subsequent processing. Then, the acquired images are input in parallel to an artificial intelligence module for defect identification, outputting defect type, defect area, and defect attributes. Furthermore, these images are input to a 3D coordinate mapping module, which, combined with camera calibration parameters, maps the 2D defect area to spatial coordinates on the vehicle's 3D model. This mapping is then input to a baseline comparison module, where it is registered with historical baseline data to identify new or expanded damage. The defect attributes, 3D coordinates, and comparison results output from these three modules are combined into an automatic annotation and report generation module, generating structured data containing ID, 3D location, and evolution trends. Finally, the data platform interface module associates the processed inspection report with metadata such as test mileage and operating conditions, uploading it to the durability testing database to achieve persistent data storage and full lifecycle management.

[0226] The full-vehicle vision acquisition module solves the problems of limited field of view, strong subjectivity, and ambiguous positioning in traditional manual inspection by acquiring images from multiple perspectives and with high precision, providing standardized and high-fidelity visual input for automated inspection. The artificial intelligence module utilizes deep learning algorithms to achieve automated and high-precision identification of various damage types, such as cracks, paint defects, and loose plastic parts, improving inspection efficiency and consistency while reducing the rate of human error. The 3D coordinate mapping module achieves precise conversion from 2D pixel coordinates to 3D physical space coordinates, assigning each damage with millimeter-level precision spatial location, transforming damage localization from vague textual descriptions into calculable and reproducible digital coordinates. The baseline comparison module, through historical image registration and differential analysis, enables automatic comparison of the damage state of the same vehicle in different test cycles, quantifying and recording the evolution process of damage (such as crack extension and area expansion), solving the problem of difficult damage tracking. The automatic annotation and report generation module fuses multi-source data to generate standardized digital inspection reports, linking the entire lifecycle information of damage through IDs, achieving structured storage and intuitive visualization of damage information. The data platform interface module realizes multi-source data association with VIN code as the core, providing traceable and mineable big data support for vehicle body structure improvement, durability and failure analysis.

[0227] Figure 3 This is a schematic diagram illustrating the operation of an artificial intelligence module for damage recognition according to an embodiment of this application. Figure 3As shown, the process begins with inputting the acquired image, followed by image preprocessing to provide a standardized data foundation for subsequent multi-task recognition. The preprocessed image is then input in parallel to the crack detection module, paint defect detection module, plastic loosening detection module, and deformation detection module. Each module independently extracts and identifies features of specific damage types, outputting specialized results such as crack masks, defect regions, loosening boundaries, and deformation depth. Furthermore, the results from these four specialized detection modules are converged into a result fusion module. Through multi-source information association and logical judgment, redundancy is eliminated, and the final damage type and attributes are determined. Finally, the result fusion module outputs a damage list containing complete defect types, region coordinates, and attribute parameters, realizing the transformation from raw images to structured defect data.

[0228] Figure 4 This is a schematic diagram illustrating a three-dimensional coordinate mapping principle according to an embodiment of this application. Figure 4 As shown, the damage location in the two-dimensional image is first obtained. Combined with camera parameter calibration, coordinate transformation calculation is performed to map the two-dimensional pixel coordinates to three-dimensional space. Spatial registration and intersection operations are then performed with the three-dimensional vehicle body model. Finally, the three-dimensional coordinates of the damage are determined. This embodiment eliminates lens distortion and projection errors through accurate camera parameter calibration, achieves accurate mapping from the two-dimensional visual plane to the three-dimensional physical space using coordinate transformation calculation, and provides geometric constraints and spatial references using the three-dimensional vehicle body model, ensuring the accuracy of damage localization. This solves the problem that two-dimensional detection cannot reflect the true spatial location of the damage, providing a reliable spatial data foundation for subsequent damage tracking, comparative analysis, and digital management.

[0229] Figure 5 This is a schematic diagram of a baseline comparison module according to an embodiment of this application. Figure 5 As shown, the current period image is first acquired. Then, the current period image is registered with the baseline image to eliminate viewpoint and positional deviations. Next, image comparison analysis is performed to identify pixel-level differences and determine if any differences exist. If differences exist, it is further determined whether the damage is real; if it is real damage, it is marked as new or aggravated damage. Finally, the damage record is updated; if no differences exist or the damage is not real, it is determined as unchanged. This embodiment ensures spatial consistency of images from different periods through baseline registration and accurately captures subtle changes using image comparison analysis. This embodiment effectively filters out false changes caused by environmental factors such as dust and lighting through a dual verification mechanism of difference judgment and real damage, improving the accuracy of damage identification. Finally, by dynamically updating the damage record, continuous and quantitative tracking of the vehicle body damage evolution process is achieved, providing reliable data support for durability assessment.

[0230] Figure 6 This is a schematic diagram of a system data flow according to an embodiment of this application. Figure 6 As shown, the vehicle defect localization system first acquires multi-view images of the vehicle's outer surface. Then, the acquired images are input in parallel to three modules: damage identification, location mapping, and change comparison. The damage identification module analyzes the image content to determine the defect type and attributes; the location mapping module converts the defect area to three-dimensional spatial coordinates; and the change comparison module compares the current data with historical baselines to identify new or expanded damage. The outputs of these three modules are combined into a report generation module to form a complete inspection report. Finally, the vehicle defect localization report is uploaded to a database for archiving. This embodiment achieves simultaneous detection, localization, and comparison through a parallel processing mechanism, improving system response speed and processing efficiency. This embodiment ensures the accuracy and completeness of the inspection report in defect characterization, spatial localization, and evolution tracking through multi-source data fusion of damage identification, location mapping, and change comparison. This embodiment achieves structured, standardized, and full lifecycle management of inspection data through automated report generation and database storage, providing an efficient and traceable data foundation for data analysis and decision support in vehicle durability testing.

[0231] According to an embodiment of this application, an apparatus embodiment for a method of locating vehicle defects is provided. Figure 7 This is a schematic diagram of a vehicle defect location device according to an embodiment of this application, as shown below. Figure 7 As shown, it should be noted that the device can be used to perform the above-mentioned method for locating vehicle defects. The device includes: an acquisition module 702, an identification module 704, a mapping module 706, and a comparison module 708.

[0232] The acquisition module 702 is used to acquire view images of the vehicle's outer surface using a multi-view camera and obtain the camera pose parameters when acquiring the view images; the recognition module 704 is used to identify defects on the vehicle's outer surface based on the view images and obtain the defect attributes and defect area of ​​the target defect on the vehicle's outer surface; the mapping module 706 is used to map the defect area onto the vehicle's three-dimensional body model based on the camera pose parameters and camera calibration parameters to obtain the three-dimensional coordinates of the target defect; the comparison module 708 is used to compare the target defect with the vehicle's historical defects based on the defect's three-dimensional coordinates and defect attributes, obtain the comparison results, and locate the defect on the vehicle's three-dimensional body model based on the comparison results to obtain the vehicle defect location results.

[0233] The acquisition module includes: an acquisition unit for acquiring the rotation matrix and translation vector of the camera in a preset coordinate system when the camera is acquiring a viewpoint image; a first determination unit for determining the relative position parameters of the camera based on the positional relationship between the preset coordinate system and the camera coordinate system; and a second determination unit for determining the camera pose parameters based on the rotation matrix, translation vector, and relative position parameters.

[0234] The identification module is further configured to input the viewpoint image into the defect identification model to predict the defect type, defect region mask, and defect size of the target defect; determine the defect area of ​​the target defect based on the defect region mask; determine the defect attributes based on at least one of the defect type, defect size, and defect area; and determine the defect region of the target defect based on the defect region mask.

[0235] The defect recognition model includes a crack detection network, a paint defect segmentation network, a plastic part loosening detection network, and a deformation detection network. The recognition module further utilizes the crack detection network to detect cracks in the viewpoint image, obtaining a crack mask and crack length; the paint defect segmentation network to detect defects in the viewpoint image, obtaining a defect region mask; the plastic part loosening detection network to detect loose parts in the viewpoint image, obtaining the component bounding box of the loose part and the gap width of the corresponding loose region; and the deformation detection network to detect deformation regions and deformation depths in the viewpoint image. Based on at least one of the crack mask, component bounding box, and deformation region, the defect type is determined; and based on at least one of the crack length, gap width, and deformation depth, the defect size is determined.

[0236] The mapping module is also used to select the coordinates of feature points in the defect area; convert the feature point coordinates into a target direction vector in the camera coordinate system according to the intrinsic parameter matrix in the camera calibration parameters; transform the target direction vector into a preset coordinate system based on the camera pose parameters to form a target ray that starts from the camera optical center and extends along the target direction vector, where the camera optical center is the optical center of the camera; and determine the three-dimensional coordinates of the defect based on the target ray and the three-dimensional model of the vehicle body.

[0237] The mapping module is also used to determine the target intersection point between the target ray and the corresponding model surface of the vehicle body 3D model; and to determine the intersection point coordinates as the defect 3D coordinates.

[0238] The comparison module is also used to obtain the historical defect's three-dimensional coordinates and attributes; to perform spatial distance matching between the historical defect's three-dimensional coordinates and the defect's three-dimensional coordinates to obtain a matching result, wherein the matching result is used to indicate whether the spatial distance between the historical defect's three-dimensional coordinates and the defect's three-dimensional coordinates is less than a preset distance threshold; and to determine the comparison result based on the matching result, defect attributes, and historical defect attributes.

[0239] The comparison module is also used to determine that the target defect is not a new defect in response to the matching result that the spatial distance is less than a preset distance threshold, and to determine the comparison result based on the difference between the defect attributes and historical defect attributes, as well as the fact that it is not a new defect; and to determine that the target defect is a new defect in response to the matching result that the spatial distance is greater than or equal to the preset distance threshold.

[0240] The comparison module is also used to determine that the target defect is a non-new defect and that the target defect has defect expansion when the difference is greater than a preset attribute difference threshold; and to determine that the target defect is a non-new defect and that the target defect has defect expansion when the difference is less than or equal to a preset attribute difference threshold.

[0241] The comparison module is also used to determine the target location information of the target defect based on the comparison results; and to locate the defect on the vehicle body 3D model based on the target location information to obtain the vehicle defect location result.

[0242] The comparison module is also used to respond to the comparison result that the target defect is not a new defect and the target defect has defect expansion, and to determine the defect level of the target defect based on the defect expansion information of the defect expansion; and to update the original positioning information of the target defect on the three-dimensional model of the vehicle body based on the defect level to obtain the target positioning information.

[0243] The comparison module is also used to determine the original positioning information of the target defect on the vehicle body three-dimensional model as the target positioning information in response to the comparison result that the target defect is a non-new defect and the target defect does not have defect expansion.

[0244] The comparison module is also used to determine the target location information based on the defect attributes and defect area of ​​the target defect in response to the comparison result indicating that the target defect is a newly added defect.

[0245] The device is also used to determine the vehicle's defect change trend and defect change parameters based on the vehicle defect location results; and to construct a vehicle defect detection report based on the defect change trend and defect change parameters.

[0246] Embodiments of this application also provide a vehicle, including: a memory storing an executable program; and a processor for running the program, wherein the program executes the methods described in various embodiments of this application when it runs.

[0247] Embodiments of this application also provide a computer-readable storage medium including a stored executable program, wherein, when the executable program is running, it controls the device where the computer-readable storage medium is located to perform the methods of various embodiments of this application.

[0248] Embodiments of this application also provide a computer program product, including a computer program that, when executed by a processor, implements the methods of various embodiments of this application.

[0249] Embodiments of this application also provide a computer program product, including a non-volatile computer-readable storage medium for storing a computer program that, when executed by a processor, implements the methods in various embodiments of this application.

[0250] Embodiments of this application also provide a computer program that, when executed by a processor, implements the methods described in the various embodiments of this application.

[0251] In the above embodiments of this application, the descriptions of each embodiment have different focuses. For parts not described in detail in a certain embodiment, please refer to the relevant descriptions of other embodiments.

[0252] In the several embodiments provided in this application, it should be understood that the disclosed technical content can be implemented in other ways. The device embodiments described above are merely illustrative; for example, the division of units can be a logical functional division, and in actual implementation, there may be other division methods. For instance, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the displayed or discussed mutual coupling, direct coupling, or communication connection may be through some interfaces; the indirect coupling or communication connection between units or modules may be electrical or other forms.

[0253] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0254] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0255] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as a USB flash drive, read-only memory (ROM), random access memory (RAM), portable hard drive, magnetic disk, or optical disk.

[0256] The above description is only a preferred embodiment of this application. It should be noted that for those skilled in the art, several improvements and modifications can be made without departing from the principle of this application, and these improvements and modifications should also be considered within the scope of protection of this application.

Claims

1. A method for locating vehicle defects, characterized in that, include: At least one camera positioned at a predetermined location around the vehicle acquires a view image of the vehicle's outer surface, and obtains the camera pose parameters of at least one camera when acquiring the view image. Based on the viewpoint image, defects are identified on the outer surface of the vehicle to obtain the defect attributes of the target defect and the defect area of ​​the target defect on the outer surface of the vehicle. Based on the camera pose parameters and the camera calibration parameters, the defect region is mapped onto the vehicle body 3D model to obtain the defect 3D coordinates of the target defect; Based on the three-dimensional coordinates of the defect and the defect attributes, the target defect and the historical defects stored in the vehicle are compared to obtain the comparison results. Based on the comparison results, the defect is located on the three-dimensional model of the vehicle body to obtain the vehicle defect location results.

2. The method according to claim 1, characterized in that, Obtaining the camera pose parameters when the camera is acquiring the viewpoint image includes: Obtain the rotation matrix and translation vector of the camera in the preset coordinate system when the camera is acquiring the viewpoint image; Based on the positional relationship between the preset coordinate system and the camera coordinate system of the camera, the relative position parameters of the camera are determined; The camera pose parameters are determined based on the rotation matrix, the translation vector, and the relative position parameters.

3. The method according to claim 1, characterized in that, Based on the aforementioned viewpoint image, defect identification is performed on the vehicle's outer surface to obtain the defect attributes of the target defect and the defect region of the target defect on the vehicle's outer surface, including: The viewpoint image is input into the defect recognition model to predict the defect type, defect region mask, and defect size of the target defect; Based on the defect region mask, the defect area of ​​the target defect is determined; The defect attribute is determined based on at least one of the defect type, the defect size, and the defect area; Based on the defect region mask, the defect region of the target defect is determined.

4. The method according to claim 3, characterized in that, The defect recognition model includes a crack detection network, a paint defect segmentation network, a plastic loosening detection network, and a deformation detection network. The viewpoint image is input into the defect recognition model to predict the defect type, defect region mask, and defect size of the target defect, including: The crack detection network is used to detect the viewpoint image to obtain the crack mask and crack length; The defect region mask is obtained by detecting the view image using the paint defect segmentation network. The loose plastic part detection network is used to detect the view image to obtain the component boundary box of the loose part and the gap width of the corresponding loose area of ​​the loose part; The deformation detection network is used to detect the viewpoint image to obtain the deformation region and deformation depth; The defect type is determined based on at least one of the crack mask, the component bounding box, and the deformation region; The defect size is determined based on at least one of the crack length, the gap width, and the deformation depth.

5. The method according to claim 1, characterized in that, Based on the camera pose parameters and the camera calibration parameters, the defect region is mapped onto the vehicle's 3D body model to obtain the 3D coordinates of the target defect, including: Select the coordinates of the feature points in the defect region; Based on the intrinsic parameter matrix in the camera calibration parameters, the coordinates of the feature points are converted into target direction vectors in the camera coordinate system; Based on the camera pose parameters, the target direction vector is transformed into a preset coordinate system to form a target ray that starts from the camera optical center and extends along the target direction vector, wherein the camera optical center is the optical center of the camera; Based on the target ray and the three-dimensional model of the vehicle body, the three-dimensional coordinates of the defect are determined.

6. The method according to claim 5, characterized in that, Based on the target ray and the three-dimensional model of the vehicle body, the three-dimensional coordinates of the defect are determined, including: Determine the target intersection point between the target ray and the target surface of the corresponding model of the three-dimensional vehicle body model; The coordinates of the intersection point of the target intersection are determined as the three-dimensional coordinates of the defect.

7. The method according to claim 1, characterized in that, Based on the three-dimensional coordinates of the defect and the defect attributes, a data comparison is performed between the target defect and the historical defects of the vehicle to obtain the comparison results, including: Obtain the three-dimensional coordinates and attributes of the historical defects; Spatial distance matching is performed between the historical defect 3D coordinates and the defect 3D coordinates to obtain a matching result, wherein the matching result is used to indicate whether the spatial distance between the historical defect 3D coordinates and the defect 3D coordinates is less than a preset distance threshold; The comparison result is determined based on the matching result, the defect attribute, and the historical defect attribute.

8. The method according to claim 7, characterized in that, Determining the comparison result based on the matching result, the defect attribute, and the historical defect attribute includes: In response to the matching result that the spatial distance is less than the preset distance threshold, it is determined that the target defect is not a newly added defect, and the comparison result is determined based on the difference between the defect attribute and the historical defect attribute, as well as the fact that it is not a newly added defect; In response to the matching result being that the spatial distance is greater than or equal to the preset distance threshold, it is determined that the comparison result is that the target defect is a newly added defect.

9. The method according to claim 8, characterized in that, The comparison result is determined based on the difference between the defect attribute and the historical defect attribute, and the non-new defect, including: In response to the difference being greater than a preset attribute difference threshold, it is determined that the comparison result is that the target defect is the non-new defect and that the target defect has defect expansion; In response to the difference being less than or equal to the preset attribute difference threshold, it is determined that the comparison result is that the target defect is the non-new defect and that the target defect does not have the defect extension.

10. The method according to claim 9, characterized in that, Based on the comparison results, the vehicle defect is located on the 3D model of the vehicle body to obtain the vehicle defect location results, including: Based on the comparison results, the target location information of the target defect is determined; Based on the target positioning information, the vehicle is positioned on the three-dimensional model of the vehicle body to obtain the vehicle defect positioning result.

11. The method according to claim 10, characterized in that, Based on the comparison results, the target location information of the target defect is determined, including: In response to the comparison result indicating that the target defect is the non-new defect and that the target defect has the defect extension, the defect level of the target defect is determined based on the defect extension information of the defect extension; Based on the defect level, the original location information of the target defect on the three-dimensional model of the vehicle body is updated to obtain the target location information.

12. The method according to claim 10, characterized in that, Based on the comparison results, the target location information of the target defect is determined, including: In response to the comparison result indicating that the target defect is the non-new defect and that the target defect does not have the defect extension, the original positioning information of the target defect on the vehicle body three-dimensional model is determined as the target positioning information.

13. The method according to claim 10, characterized in that, Based on the comparison results, the vehicle defect is located on the 3D model of the vehicle body to obtain the vehicle defect location results, including: In response to the comparison result indicating that the target defect is the newly added defect, the target location information is determined based on the defect attributes of the target defect and the defect region.

14. The method according to any one of claims 1 to 13, characterized in that, The method further includes: Based on the vehicle defect location results, the defect change trend and defect change parameters of the vehicle are determined; Based on the defect change trend and the defect change parameters, a defect detection report for the vehicle is constructed.

15. A device for locating vehicle defects, characterized in that, include: The acquisition module is used to acquire view images of the vehicle's outer surface through a multi-view camera, and to acquire camera pose parameters of the camera when acquiring the view images. The identification module is used to identify defects on the outer surface of the vehicle based on the view image, and to obtain the defect attributes of the target defect and the defect area of ​​the target defect on the outer surface of the vehicle. The mapping module is used to map the defect region onto the vehicle body 3D model based on the camera pose parameters and the camera calibration parameters of the camera, so as to obtain the defect 3D coordinates of the target defect. The comparison module is used to compare the target defect with the vehicle's historical defects based on the defect's three-dimensional coordinates and the defect attributes, obtain the comparison result, and locate the defect on the vehicle's three-dimensional model based on the comparison result to obtain the vehicle defect location result.

16. The apparatus according to claim 15, characterized in that, The acquisition module includes: The acquisition unit is used to acquire the rotation matrix and translation vector of the camera in a preset coordinate system when the camera is acquiring the view image; The first determining unit is used to determine the relative position parameters of the camera based on the positional relationship between the preset coordinate system and the camera coordinate system of the camera. The second determining unit is used to determine the camera pose parameters based on the rotation matrix, the translation vector, and the relative position parameters.

17. A vehicle, characterized in that, include: Memory, which stores executable programs; A processor for running the program, wherein the program, when running, performs the method according to any one of claims 1 to 14.

18. A computer-readable storage medium, characterized in that, The computer-readable storage medium includes a stored executable program, wherein, when the executable program is executed, it controls the device on which the storage medium is located to perform the method according to any one of claims 1 to 14.

Images