Vehicle bottom anomaly detection method based on neural network

Abstract

The invention discloses a vehicle bottom anomaly detection method based on a neural network, and belongs to the technical field of image processing and intelligent security check. The method comprises the following steps: acquiring an on-site vehicle bottom image and a historical vehicle bottom image, and carrying out preprocessing and size normalization on the images; performing component-level segmentation on the field image by adopting an automatic mask generation model based on a neural network, and extracting a mask region; matching and spreading the masks in the two frames of images by using a video target tracker; and judging that the target which is not tracked successfully is an abnormal object, and generating a visual report. The method does not need to depend on a predefined template or known dangerous object types, and has an open type anomaly detection capability. Compared with an existing scheme, the method has the advantages of being high in robustness, high in detection precision, convenient to deploy and the like, and is suitable for automatic safety inspection of the vehicle chassis in various scenes.

Description

Technical Field

[0001] This invention relates to the fields of artificial intelligence, computer vision, and image processing, and specifically to a method for detecting anomalies under vehicles based on neural networks, applicable to foreign object detection tasks under vehicles in various complex scenarios. Background Technology

[0002] With increasing societal emphasis on public safety, vehicle chassis inspection, as an important security check method, has been widely applied in areas such as checkpoints, highway entrances, and entrances to important locations.

[0003] Currently, common methods for anomaly detection based on vehicle undercarriage images include template matching or image comparison methods, as well as AI methods based on target recognition.

[0004] However, methods based on template matching or image comparison heavily rely on rigid image alignment. Slight changes in vehicle parking position, shooting angle, or lighting conditions (such as shadows and reflections) can lead to a large number of erroneous pixels, resulting in a high false alarm rate and poor robustness. For example, Chinese patent application CN112330646A discloses a method for detecting anomalies under high-speed trains based on two-dimensional images. Its core involves first manually labeling fault-free train models to create a "template" containing various normal parts; during detection, a trained visual learning algorithm (such as CascadeRcnn) is used to identify and locate various components in the real-time acquired images of the train's undercarriage; finally, the identified components are compared with the pre-established template to identify differences and determine if an anomaly exists. This approach is essentially a combination of "template comparison" and "known target recognition," and its detection capability is limited by the completeness of the template and the model's ability to recognize known parts.

[0005] AI methods based on target recognition have weak generalization capabilities, failing to detect unknown anomalies and only able to identify specific categories of hazardous materials already included in the training dataset. For novel hazardous materials, disguises, smuggled goods, or illegally modified parts that have not been learned from, the model cannot identify them, resulting in serious security vulnerabilities, the so-called "open set problem." Chinese patent application CN111091111A discloses a method for identifying hazardous targets under vehicles. First, it performs feature point matching (e.g., using the SURF algorithm) and image correction between real-time acquired "target images" and "standard images" stored in a database to align the two images. Then, it uses a block matching strategy (e.g., the NCC algorithm) to initially find approximate areas where differences may exist. Finally, supported by a pre-trained hazardous target detection network (e.g., YOLOv3), it performs precise detection on these differing areas to identify specific types of hazardous targets. This approach is an "align first, then detect" strategy, and its final recognition performance heavily relies on a pre-established database containing various known hazardous material samples. Summary of the Invention

[0006] To address the shortcomings of existing technologies, this invention aims to provide a neural network-based method for detecting anomalies under vehicles. By constructing a comparison mechanism between standard vehicle under-body images and on-site vehicle under-body images, and combining depth image segmentation and target tracking technologies, it achieves efficient identification and labeling of unknown abnormal objects.

[0007] To achieve the above objectives, the present invention adopts the following technical solution:

[0008] A method for detecting anomalies under vehicles based on neural networks includes the following steps:

[0009] S1. Deploy a vehicle undercarriage image acquisition system in the vehicle security inspection lane to capture real-time images of the vehicle's undercarriage as on-site images, and retrieve historical images of the vehicle's undercarriage from the vehicle information database as a comparison benchmark.

[0010] S2. Perform standardized preprocessing on the on-site vehicle undercarriage images and historical vehicle undercarriage images collected in step S1, and scale the on-site vehicle undercarriage images and historical vehicle undercarriage images to a fixed ratio according to the structural characteristics of the vehicle chassis.

[0011] S3. Image segmentation processing: An automatic mask generation model based on neural networks is used to analyze and segment the standardized on-site vehicle undercarriage image at the component level. The chassis components and potential anomalies in the on-site vehicle undercarriage image are segmented one by one, and the mask representation and spatial location information of each segmented object in the on-site vehicle undercarriage image are finally output.

[0012] S4. After segmentation, assign a unique identifier ID to each valid mask output by the automatic mask generation model, and record the parameters of spatial location information, area, and grayscale features of each mask to form a "component-mask" mapping table for the on-site vehicle undercarriage image.

[0013] S5. Target Tracking: Using historical vehicle undercarriage images as target frames, and the mask of each segmented object in the on-site vehicle undercarriage image as the initial mask, the object position information is transmitted based on the feature matching principle. The depth feature vector of each segmented object in the on-site vehicle undercarriage image is extracted. Then, similar regions are located in the target frame and a prediction mask is generated. The degree of matching between the prediction mask and the initial mask is evaluated by spatial overlap calculation, and the geometric consistency between the two is quantified.

[0014] S6. Filtering effective tracking targets: Based on the predicted mask quality score index, when the predicted mask quality score is greater than the preset quality score threshold, it is considered a target that can be tracked stably.

[0015] S7. Output the unmatched targets in the on-site vehicle undercarriage image as differential targets, which may be anomalous objects;

[0016] S8. Finally, generate a multi-dimensional visualization report.

[0017] Furthermore, in step S3, in the automatic mask generation model based on neural networks, the sampling density of segmentation points is set to 64 points / edge, and a small object feature enhancement module is added to the decoder part to ensure that small objects with a diameter of less than 5cm can be effectively identified.

[0018] Furthermore, in step S3, the automatic mask generation model based on neural networks adopts a multi-layer convolutional structure to extract shallow and deep features of the image respectively, and combines features from different levels through a feature fusion module.

[0019] Furthermore, in the neural network-based automatic mask generation model in step S3, a mask filtering mechanism is adopted. By setting a confidence threshold, regions with confidence scores lower than the confidence threshold are automatically filtered and segmented. At the same time, based on the mask area, masks with areas smaller than a preset low threshold or larger than a preset high threshold are removed.

[0020] Furthermore, the specific process of step S5 is as follows:

[0021] S5.1 Tracker Initialization: Initialize the video target tracker using each valid mask segmented from the on-site vehicle undercarriage image as the initial target; the video target tracker records the appearance features of the initial target. Based on the assumption that the vehicle does not move significantly, the default position of the initial target in the historical vehicle undercarriage image is similar to that in the on-site vehicle undercarriage image.

[0022] S5.2, Feature Encoding of Segmented Components: Each mask obtained by segmenting the on-site vehicle undercarriage image is used to generate cue features through a cue encoder;

[0023] S5.3 Modulate the current frame image features: Through the feature attention module of the Transformer structure, the overall visual features of the current frame, i.e., the on-site vehicle undercarriage image, and the memory features of the historical vehicle undercarriage images are interacted across frames to enhance the discriminative power of the current frame features for the target; wherein, the visual features refer to the fusion features of shallow and deep features of the image, and the memory features of the historical vehicle undercarriage images refer to the long-term features stored after feature extraction of the historical vehicle undercarriage images;

[0024] S5.4 Historical vehicle undercarriage image component segmentation mask propagation prediction: Combining the visual features, cue features, and memory features of historical vehicle undercarriage images, a predicted mask and predicted mask quality score of the historical vehicle undercarriage image are generated through a mask decoder with a Transformer structure.

[0025] S5.5 Optimize the prediction mask;

[0026] S5.6. Based on the predicted mask output by the mask decoder, extract the pointer features of the corresponding segmented objects in the historical vehicle undercarriage image, and use them as the identity ID of the corresponding segmented objects in the historical vehicle undercarriage image; the segmented objects in the historical vehicle undercarriage image refer to the segmentation results when performing component segmentation on the historical vehicle undercarriage image previously.

[0027] Furthermore, in step S4, the unique identifier ID is generated by extracting the pointer features of the segmented object corresponding to the mask; in step S7, the unique identifier ID of each valid mask in the on-site vehicle undercarriage image is compared with the identity ID of the segmented object extracted in step S5.6, and the difference target is calculated. The segmented target corresponding to the ID that exists in both the on-site vehicle undercarriage image and the historical vehicle undercarriage image is considered to exist normally. The segmented object corresponding to the ID that appears in the on-site vehicle undercarriage image but does not exist in the historical vehicle undercarriage image is the difference target, which is the unmatched target in the on-site vehicle undercarriage image and may be an abnormal object.

[0028] Furthermore, in step S8, the visualization report includes an anomaly close-up view, a global segmentation view, and a historical reference view:

[0029] The anomaly close-up view is used to highlight abnormal targets and their location markers in the on-site vehicle undercarriage image; the global segmentation view is used to show the distribution of all detected objects in the on-site vehicle undercarriage image; and the historical reference view is used to present objects that were successfully matched in historical vehicle undercarriage images.

[0030] The present invention also provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the above-described method.

[0031] The present invention also provides a computer device, including a processor and a memory, wherein the memory is used to store a computer program; and the processor is used to execute the computer program to implement the above-described method.

[0032] The beneficial effects of this invention are as follows:

[0033] 1. Wide detection range and open-ended anomaly detection capability: This invention does not rely on a predefined database of hazardous materials or anomalies for training. Its core is to detect any object that is "inconsistent with historical data." Therefore, whether it is a new type of explosive, smuggled goods, or unregistered structural modifications, as long as the object is not present in historical images, it can be identified, greatly improving the generalization ability and the coverage of security checks, and solving the fundamental defect of existing AI target recognition methods that cannot detect unknown anomalies.

[0034] 2. Strong robustness and adaptability to complex security inspection environments: This invention adopts a "segmentation first, tracking later" strategy, elevating image comparison from pixel-level to component-level (object-level) comparison. The image segmentation model effectively ignores background changes such as lighting, shadows, and water stains, accurately extracting object contours. The target tracking algorithm focuses on the deep features of the object and is insensitive to slight shifts in vehicle position. This allows this method to effectively overcome the false alarm problem caused by environmental changes in traditional template matching methods, resulting in more stable performance in practical applications.

[0035] 3. High detection accuracy and precise localization: This invention employs pixel-level image segmentation technology to accurately depict the contours (masks) of anomalous objects, rather than the coarse bounding boxes of traditional object detection. Combined with optimizations specifically for small object detection, even small anomalous objects can be effectively detected. By setting a reasonable mask prediction score threshold (e.g., 0.6), matching relationships can be accurately determined, effectively reducing the false negative rate.

[0036] 4. Intuitive results presentation, facilitating manual verification: The output three-view visualization report highlights abnormal objects in the on-site image, while providing a complete on-site segmentation diagram and historical reference diagrams, offering comprehensive and clear information. Security personnel can quickly identify abnormalities and take action without laborious manual comparison, improving handling efficiency. Attached Figure Description

[0037] Figure 1 This is a schematic diagram of the overall process of the method in Embodiment 1 of the present invention;

[0038] Figure 2This is a flowchart of the target tracking process in Embodiment 1 of the present invention. Detailed Implementation

[0039] The present invention will be further described below with reference to the accompanying drawings. It should be noted that this embodiment is based on the present technical solution and provides detailed implementation methods and specific operation processes, but the protection scope of the present invention is not limited to this embodiment.

[0040] Example 1

[0041] This embodiment provides a method for detecting anomalies under vehicles based on neural networks, such as... Figure 1 As shown, it includes the following steps:

[0042] S1. Deploy a vehicle undercarriage image acquisition system in the vehicle security inspection lane to capture real-time images of the vehicle's undercarriage as on-site images. At the same time, retrieve historical undercarriage images of the vehicle from the vehicle information database based on information such as the license plate number as a comparison benchmark.

[0043] S2. Standardize and preprocess the on-site vehicle undercarriage images and historical vehicle undercarriage images collected in step S1. Based on the structural characteristics of the vehicle chassis, scale the on-site vehicle undercarriage images and historical vehicle undercarriage images to a fixed ratio to eliminate the scale inconsistency caused by differences in shooting angle or distance, and establish a unified spatial reference system for subsequent analysis.

[0044] S3. Image segmentation processing: An automatic mask generation model based on neural networks is used to parse and segment the standardized on-site vehicle undercarriage image. The chassis components (such as exhaust pipe, fuel tank, suspension system) and potential abnormal objects (such as attached packages, modified metal brackets) in the on-site vehicle undercarriage image are segmented one by one. Finally, the mask representation and spatial location information (bounding box) of each segmented object in the on-site vehicle undercarriage image are output.

[0045] In this embodiment, the neural network-based automatic mask generation model possesses multi-level feature fusion capabilities. It enhances the recognition sensitivity of small objects by increasing sampling density and sets up a confidence filtering mechanism to automatically exclude low-reliability segmentation results, enabling accurate identification of various chassis components and attachments, including tiny foreign objects. Specifically:

[0046] Enhance small object detection capability: Set the segmentation point sampling density to 64 points / side, and add a small object feature enhancement module to the decoder to ensure that tiny objects with a diameter of less than 5cm (such as small explosives and knives) can be effectively identified.

[0047] Multi-level feature extraction: The neural network-based automatic mask generation model adopts a multi-level convolutional structure to extract shallow features (such as edges and textures) and deep features (such as the overall shape of components) of the image respectively. The feature fusion module combines features from different levels to improve the segmentation accuracy of complex chassis structures (such as areas where multiple pipelines intersect).

[0048] Mask filtering mechanism: Set a confidence threshold (e.g., 0.7) to automatically filter and segment regions with confidence scores below the confidence threshold (e.g., false targets formed by noise interference). At the same time, filter based on mask area to remove masks with an area of ​​less than 10 pixels (too small, possibly noise) or greater than 10,000 pixels (too large, possibly background area), reducing the redundancy of subsequent tracking calculations.

[0049] S4. After segmentation, assign a unique identifier ID (e.g., ID:001, ID:002) to each valid mask output by the automatic mask generation model. The unique identifier ID is generated by extracting the pointer features of the segmented object corresponding to the mask (the pointer features are constructed based on the depth feature vector, spatial position information and grayscale features of the segmented object). At the same time, record the spatial position information (boundary box coordinates), area, grayscale features and other parameters of each mask to form a "part-mask" mapping table of the vehicle underside image at the scene.

[0050] S5. Target Tracking: Using historical vehicle undercarriage images as the target frame, and the mask of each segmented object in the current vehicle undercarriage image as the initial mask, object position information is transmitted based on feature matching principles. Depth feature vectors of each segmented object in the current vehicle undercarriage image are extracted. Then, similar regions are located in the target frame, and predicted masks are generated. The degree of matching between the predicted mask and the initial mask is evaluated through spatial overlap calculation, quantifying the geometric consistency between the two.

[0051] Furthermore, such as Figure 2 As shown, the specific process of step S5 is as follows:

[0052] S5.1 Tracker Initialization: Initialize the video target tracker using each valid mask segmented from the on-site vehicle undercarriage image as the initial target. The video target tracker records the apparent features of the initial target. Based on the assumption that the vehicle does not move significantly, the default position of the initial target in the historical vehicle undercarriage image is similar to that in the on-site vehicle undercarriage image.

[0053] S5.2, Segmentation Component Feature Encoding: Each mask obtained from segmenting the on-site vehicle undercarriage image is used to generate cue features through a prompt encoder. The mask is downsampled into low-resolution features, which are used for feature fusion in subsequent historical vehicle undercarriage image mask prediction to improve prediction accuracy.

[0054] S5.3 Modulating Current Frame Image Features: Through the feature attention module of the Transformer structure, the overall visual features of the current frame, i.e., the on-site vehicle undercarriage image, are interacted with the memory features of the historical vehicle undercarriage images across frames to enhance the discriminative power of the current frame features (i.e., the optimized features of the current frame's visual features after attention interaction, as mentioned above) against the target (using historical information to suppress background interference); wherein, the visual features refer to the fusion features of the shallow edge / texture and deep morphological features of the image, and the memory features of the historical vehicle undercarriage images refer to the long-term features stored after feature extraction of the historical vehicle undercarriage images.

[0055] S5.4 Historical vehicle undercarriage image component segmentation mask propagation prediction: Combining the visual features, cue features, and memory features of historical vehicle undercarriage images, a predicted mask and a predicted mask quality score (used to determine the quality of the predicted mask) for the historical vehicle undercarriage image are generated through a Transformer-structured mask decoder.

[0056] S5.5 Optimize the prediction mask, including filling small holes, removing noise, and binarizing according to a threshold to improve the quality of the prediction mask.

[0057] S5.6. Based on the predicted mask output by the mask decoder, extract the pointer features of the corresponding segmented objects in the historical vehicle undercarriage image, which serve as the identification ID of the corresponding segmented object in the historical vehicle undercarriage image (this identification ID is of the same feature type as the unique identifier ID of each valid mask in the on-site vehicle undercarriage image). The segmented objects in the historical vehicle undercarriage image refer to the segmentation results when performing component segmentation on the historical vehicle undercarriage image previously.

[0058] S6. Filtering Valid Tracking Targets: Based on the predicted mask quality score, targets with a predicted mask quality score greater than a preset quality score threshold (e.g., 0.6) are considered stably tracked. Only targets with stable tracking are retained, excluding tracking failures caused by occlusion, motion blur, etc.

[0059] S7. Anomaly detection: Compare the unique identifier ID of each valid mask in the on-site vehicle undercarriage image with the identity ID of the segmented object extracted from the historical vehicle undercarriage image in step S5.6 to calculate the difference target.

[0060] The segmented targets corresponding to IDs that appear in both the on-site vehicle undercarriage images and historical vehicle undercarriage images are considered to be present normally. The segmented objects corresponding to IDs that appear in the on-site vehicle undercarriage images but do not appear in the historical vehicle undercarriage images are considered discrepancy targets, which are unmatched targets in the on-site vehicle undercarriage images and may be anomalous objects (such as newly added foreign objects).

[0061] S8. Finally, a multi-dimensional visualization report is generated, which includes an anomaly close-up view, a global segmentation view, and a historical reference view.

[0062] The anomaly close-up view is used to highlight abnormal targets and their location markers in the on-site vehicle undercarriage image; the global segmentation view is used to show the distribution of all detected objects in the on-site vehicle undercarriage image; and the historical reference view is used to present objects that were successfully matched in historical vehicle undercarriage images.

[0063] Example 2

[0064] This embodiment provides an application of the method described in Embodiment 1 to vehicle security checks at the entrance of an important conference venue. This scenario requires a rapid and thorough inspection of all entering vehicles for chassis anomalies. Specifically, it includes the following steps:

[0065] Step 1: Initial vehicle registration and historical undercarriage image collection

[0066] A car passed through the security checkpoint of the conference center for the first time. As the vehicle drove at a constant speed past the underground vehicle undercarriage scanning imaging system, the high-speed linear scan camera automatically captured multiple images of the undercarriage and stitched them together to form a complete, high-resolution (e.g., 4096×2048 pixels) image. The vehicle's license plate recognition confirmed it was a first-time entry, and the current undercarriage image was then stored as a "historical undercarriage image" in the vehicle information database, linked to the vehicle's license plate number, model, color, and other information.

[0067] Step 2: The vehicle re-enters the area for on-site undercarriage image acquisition.

[0068] The following day, the vehicle re-entered the security checkpoint. The undercarriage scanning imaging system captured images of the vehicle's undercarriage in the same manner. At this time, a magnetic explosive device, approximately 15cm × 8cm × 5cm in size, was found to have been installed near the rear axle of the vehicle chassis by criminals and secured with black tape.

[0069] Step 3: Image Preprocessing

[0070] The security inspection computer system retrieves historical undercarriage images of the vehicle from the vehicle information database and preprocesses them together with the newly acquired on-site undercarriage images. To eliminate scale differences caused by factors such as vehicle speed and positional shifts, the security inspection computer system performs a size normalization operation. Both images are uniformly scaled to a fixed size of 1662×682 pixels, and the image format is ensured to be three-channel RGB to meet the input requirements of the subsequent neural network-based automatic mask generation model.

[0071] Step 4: On-site vehicle undercarriage image segmentation

[0072] The preprocessed on-site undercarriage image is fed into a neural network-based automatic mask generation model (e.g., a Segment Anything Model optimized to adapt to the complex undercarriage environment). The automatic mask generation model performs component-level segmentation on the on-site undercarriage image, and its parameters have been specifically optimized for this application scenario as follows:

[0073] Enhanced small object detection capability: The sampling density of segmentation points inside the automatic mask generation model is set to a high level to ensure that masks can be effectively generated for small suspicious objects (such as explosives in this case), rather than ignoring them as background or noise.

[0074] Multi-level feature extraction: The automatic mask generation model effectively distinguishes the metallic luster of the exhaust pipe, the complex structure of the suspension arm, and the rough texture of the tape with attached explosives by utilizing its multi-level convolutional structure.

[0075] Mask filtering mechanism: Each segmentation result output by the automatic mask generation model carries a confidence score. A confidence threshold of 0.7 is set to filter out low-confidence segmentation results. Simultaneously, filtering is performed based on the mask area to exclude interference items such as oil stains and gravel with areas too small (less than 100 pixels).

[0076] After this step, the automatic mask generation model outputs multiple segmented objects, including normal components such as the exhaust pipe, fuel tank, suspension system, and drive shaft, as well as the newly added explosive device. Each segmented object is represented by a precise pixel-level segmentation mask and a rectangular bounding box.

[0077] Step 5: Target Tracking and Matching

[0078] This step is the core of this embodiment. The on-site image of the vehicle's underside is used as the "first frame" of the video sequence, the historical image of the vehicle's underside is used as the "second frame," and the video target tracker is activated.

[0079] Tracker initialization: Assign a unique ID to the mask of each segmented object in the "first frame" (the image under the vehicle on site) (e.g., exhaust pipe mask, fuel tank mask, explosive mask, etc.) and initialize a tracker instance.

[0080] Tracking propagation and matching: The tracker attempts to find the corresponding locations of these segmented objects in the "second frame" (historical undercarriage image).

[0081] For inherent components such as exhaust pipes and fuel tanks, since they exist in both frames of images, the tracker can successfully locate them in the historical undercarriage images. By judging the predicted mask confidence score in the historical undercarriage images, the obtained values ​​are all much higher than the preset threshold of 0.6 (for example, the confidence prediction scores are generally above 0.85).

[0082] However, for that explosive mask, because it simply didn't exist in the historical images of the vehicle's undercarriage, the tracker couldn't find any matching target in the "second frame." Its tracking status was marked as "failed."

[0083] Anomaly detection: Iterate through all tracking tasks initiated from the "first frame". Any segmented object that fails to be tracked or has a confidence prediction score below 0.6 is identified as an "anomaly". In this example, only the segmented object corresponding to the explosive is marked as an anomaly.

[0084] Step Six: Result Output and Alarms

[0085] The system immediately generates a three-view visualization report and pushes it to the monitoring terminal of the on-site security personnel, while simultaneously issuing an audible and visual alarm.

[0086] First view (close-up of the anomaly): Only the image under the vehicle at the scene is shown, and the detected explosive is marked with a bright red bounding box and a semi-transparent mask, and labeled "Anomaly ID: 001".

[0087] Second view (overall view of the scene segmentation): Displays an image of the vehicle's undercarriage at the scene, and marks all successfully segmented objects with different colored bounding boxes to help security personnel understand the overall layout of the vehicle chassis.

[0088] Third view (historical reference image): Displays historical images of the vehicle's undercarriage and marks all successfully matched normal parts.

[0089] Based on this intuitive report, security personnel can quickly pinpoint the exact location of the anomaly and activate the emergency response plan without tedious manual comparison.

[0090] Example 2

[0091] This embodiment provides an application of the method described in Embodiment 1 in the detection of smuggled goods in container trucks in logistics parks. It is used to detect whether the chassis of container trucks contains undeclared modifications or smuggled goods.

[0092] In a large bonded logistics park, a container truck needs to enter and exit the park multiple times a day. The truck is registered, and the vehicle information database contains historical images of its undercarriage in its standard condition. To smuggle goods, the driver welded an irregularly shaped metal compartment inside the vehicle's chassis beam, using the existing structure as cover, to conceal high-value electronic products. This embodiment's method includes the following steps:

[0093] Step 1: Image Acquisition and Preprocessing

[0094] When the truck enters the park, the undercarriage scanning system collects images of its undercarriage as usual and retrieves historical undercarriage images from the vehicle information database. Because the truck chassis is longer, the original images collected by the system have a higher resolution (e.g., 8192×2048 pixels), but in the preprocessing stage, they are also scaled to a uniform size according to a preset aspect ratio (e.g., based on a typical truck chassis) for subsequent analysis.

[0095] Step 2: Image Segmentation

[0096] The preprocessed images of the vehicle's undercarriage are fed into an automatic mask generation model. Due to the extremely complex structure of the truck chassis, with its intricate network of pipes and numerous components, the multi-level feature extraction configuration of the automatic mask generation model plays a crucial role. The automatic mask generation model can identify weld seams in newly added metal compartments and metallic reflective properties different from the original beams through shallow features (edges, textures), and can segment large components such as the driveshaft, differential, and leaf spring suspension as a whole through deep features (overall shape).

[0097] Ultimately, the automatic mask generation model successfully identified and segmented the newly added metallic occlusion into an independent object, generating its corresponding mask and bounding box, even though most of it was obscured by other components.

[0098] Step 3: Target Tracking and Anomaly Detection

[0099] Once the tracking process is initiated, all normal components, such as the drive shaft, differential, fuel tank, and air reservoir, segmented from the on-site undercarriage images, are successfully matched in historical undercarriage images, with mask prediction scores all exceeding the set threshold of 0.6.

[0100] For the newly added metal compartment, the video target tracker could not find any corresponding object in the historical images of the undercarriage, and the tracking failed.

[0101] Therefore, the metal compartment was identified as an "abnormal object".

[0102] Step 4: Result Output and Manual Review

[0103] Alarms are sent to the workstations of customs inspectors.

[0104] The generated visualization report clearly marks the location, shape, and size of the unusual metallic compartment.

[0105] Unlike Example 1, the "abnormal object" at the bottom of the vehicle in this example is not a foreign attachment, but a modification of the vehicle body structure. However, the method in this example does not require prior knowledge of the form of modification and can still effectively detect it.

[0106] Based on the report, the inspectors directed the vehicle to the inspection area and used portable testing equipment to focus on the areas highlighted in the report, ultimately discovering the smuggled goods hidden there.

[0107] For those skilled in the art, various corresponding changes and modifications can be made based on the above technical solutions and concepts, and all such changes and modifications should be included within the protection scope of the claims of this invention.

Claims

1. A method for detecting undercarriage anomalies based on neural networks, characterized in that, Includes the following steps: S1. Deploy a vehicle undercarriage image acquisition system in the vehicle security inspection lane to capture real-time images of the vehicle's undercarriage as on-site images, and retrieve historical images of the vehicle's undercarriage from the vehicle information database as a comparison benchmark. S2. Perform standardized preprocessing on the on-site vehicle undercarriage images and historical vehicle undercarriage images collected in step S1, and scale the on-site vehicle undercarriage images and historical vehicle undercarriage images to a fixed ratio according to the structural characteristics of the vehicle chassis. S3. Image segmentation processing: An automatic mask generation model based on neural networks is used to analyze and segment the standardized on-site vehicle undercarriage image at the component level. The chassis components and potential anomalies in the on-site vehicle undercarriage image are segmented one by one, and the mask representation and spatial location information of each segmented object in the on-site vehicle undercarriage image are finally output. S4. After segmentation, assign a unique identifier ID to each valid mask output by the automatic mask generation model, and record the parameters of spatial location, area, and grayscale features of each mask to form a "component-mask" mapping table for the on-site vehicle undercarriage image. S5. Target Tracking: Using historical vehicle undercarriage images as target frames, and the mask of each segmented object in the on-site vehicle undercarriage image as the initial mask, the object position information is transmitted based on the feature matching principle. The depth feature vector of each segmented object in the on-site vehicle undercarriage image is extracted. Then, similar regions are located in the target frame and a prediction mask is generated. The degree of matching between the prediction mask and the initial mask is evaluated by spatial overlap calculation, and the geometric consistency between the two is quantified. S6. Filtering effective tracking targets: Based on the predicted mask quality score index, when the predicted mask quality score is greater than the preset quality score threshold, it is considered a target that can be tracked stably. S7. Output the unmatched targets in the on-site vehicle undercarriage image as differential targets, which may be anomalous objects; S8. Finally, generate a multi-dimensional visualization report.

2. The method according to claim 1, characterized in that, In step S3, the automatic mask generation model based on neural networks sets the sampling density of segmentation points to 64 points / edge, and adds a small object feature enhancement module to the decoder part to ensure that tiny objects with a diameter of less than 5cm can be effectively identified.

3. The method according to claim 1, characterized in that, In step S3, the automatic mask generation model based on neural networks adopts a multi-layer convolutional structure to extract shallow and deep features of the image respectively, and combines features from different levels through a feature fusion module.

4. The method according to claim 1, characterized in that, In the neural network-based automatic mask generation model in step S3, a mask filtering mechanism is adopted. By setting a confidence threshold, regions with confidence scores lower than the confidence threshold are automatically filtered and segmented. At the same time, based on the mask area, masks with areas smaller than the preset low threshold or larger than the preset high threshold are removed.

5. The method according to claim 1, characterized in that, The specific process of step S5 is as follows: S5.1 Tracker Initialization: Initialize the video target tracker using each valid mask segmented from the on-site vehicle undercarriage image as the initial target; the video target tracker records the appearance features of the initial target. Based on the assumption that the vehicle does not move significantly, the default position of the initial target in the historical vehicle undercarriage image is similar to that in the on-site vehicle undercarriage image. S5.2, Feature Encoding of Segmented Components: Each mask obtained by segmenting the on-site vehicle undercarriage image is used to generate cue features through a cue encoder; S5.3 Modulate the current frame image features: Through the feature attention module of the Transformer structure, the overall visual features of the current frame, i.e., the on-site vehicle undercarriage image, and the memory features of the historical vehicle undercarriage images are interacted across frames to enhance the discriminative power of the current frame features for the target; wherein, the visual features refer to the fusion features of shallow and deep features of the image, and the memory features of the historical vehicle undercarriage images refer to the long-term features stored after feature extraction of the historical vehicle undercarriage images; S5.4 Historical vehicle undercarriage image component segmentation mask propagation prediction: Combining the visual features, cue features, and memory features of historical vehicle undercarriage images, a predicted mask and predicted mask quality score of the historical vehicle undercarriage image are generated through a mask decoder with a Transformer structure. S5.5 Optimize the prediction mask; S5.

6. Based on the predicted mask output by the mask decoder, extract the pointer features of the corresponding segmented objects in the historical vehicle undercarriage image, and use them as the identity ID of the corresponding segmented objects in the historical vehicle undercarriage image; the segmented objects in the historical vehicle undercarriage image refer to the segmentation results when performing component segmentation on the historical vehicle undercarriage image previously.

6. The method according to claim 5, characterized in that, In step S4, the unique identifier ID is generated by extracting the pointer features of the segmented object corresponding to the mask; in step S7, the unique identifier ID of each valid mask in the on-site vehicle undercarriage image is compared with the identity ID of the segmented object extracted in step S5.6, and the difference target is calculated. The segmented target corresponding to the ID that exists in both the on-site vehicle undercarriage image and the historical vehicle undercarriage image is considered to exist normally. The segmented object corresponding to the ID that appears in the on-site vehicle undercarriage image but does not exist in the historical vehicle undercarriage image is the difference target, which is the unmatched target in the on-site vehicle undercarriage image and may be an abnormal object.

7. The method according to claim 1, characterized in that, In step S8, the visualization report includes an anomaly close-up view, a global segmentation view, and a historical reference view: The anomaly close-up view is used to highlight abnormal targets and their location markers in the on-site vehicle undercarriage images; The global segmentation view is used to display the distribution of all detected objects in the on-site vehicle undercarriage image; the historical reference view is used to present objects that were successfully matched in historical vehicle undercarriage images.

8. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, implements the method described in any one of claims 1-7.

9. A computer device, characterized in that, It includes a processor and a memory, the memory being used to store a computer program; the processor being used to execute the computer program to implement the method of any one of claims 1-7.

Images