Container archive management method and system based on digital twinning

Abstract

The application provides a container archive management method and system based on digital twinning, comprising: obtaining a container image, which is pasted to a preset 3D model as texture after being recognized and corrected to construct a digital twin; comparing images of the same container at different time points to quantitatively analyze state changes; taking the digital twin as the core, integrating basic information, historical images and analysis results to construct and dynamically update a digital archive. The application also provides a corresponding system. The application realizes the digitization and visualization of archive management, improves the accuracy and objectivity of state evaluation, and improves the management efficiency.

Description

Technical Field

[0001] This invention relates to the field of container management technology, and more specifically, to a container file management method and system based on digital twins. Background Technology

[0002] As the core standardized transport unit in the modern logistics system, the asset management and status monitoring of containers are crucial for container owners, leasing companies, carriers, and other stakeholders.

[0003] Traditional container record management relies primarily on manual on-site inspections, filling out paper forms, and taking photographs for archiving. However, this traditional model has several inherent flaws. First, the information collection and management processes are disconnected. Image data taken by on-site personnel is often stored piecemeal on different personal devices or in folders, lacking a systematic connection with container ledger information, leading to difficulties in subsequent information retrieval and incomplete record keeping. Second, condition comparison lacks objective standards. When assessing changes in container condition at different points in time (e.g., before and after leasing), managers often rely on personal experience to subjectively compare photos taken at different times and angles, making it difficult to accurately quantify the degree of damage or aging. This not only affects the accuracy of the assessment but also easily leads to business disputes. Furthermore, the image collection process itself lacks standardization. The arbitrariness of factors such as shooting angle, lighting conditions, and image completeness results in low effective utilization of a large amount of image information, failing to truly and comprehensively reflect the actual condition of the container.

[0004] Patent document CN117197581A discloses a method for automatically archiving container image data based on computer vision recognition. This document only solves the superficial efficiency problem of automatic photo archiving and compliance verification of packing photos. It cannot solve the problems of poor visualization, lack of objective quantitative standards for container status assessment, fragmented historical status tracing, fragmented data storage, and narrow business coverage. Its efficiency still has room for improvement.

[0005] In addition, existing image matching algorithms only support the comparison of full-area images of containers from complete, orthorectified, and standardized camera positions. On-site inspection and container repair acceptance scenarios often only capture images of local areas of the container. These images have a small coverage area and the shooting angle / scaling ratio differs greatly from historical full-area images. Traditional SIFT global registration cannot extract a sufficient number of corresponding feature points, and the differential and connected component quantization processes all fail, making it impossible to complete cross-period tracing and change analysis of the same local physical area of ​​the same container. At the same time, existing solutions directly use easily changeable damage areas such as rust, stains, and damage as cross-period matching features. After the container is cleaned and repaired, the matching benchmark is lost, and the damage quantization accuracy and matching recall rate are difficult to guarantee.

[0006] In summary, the existing management methods are inefficient. The process of creating, updating, and querying records consumes a lot of manpower and cannot achieve dynamic and real-time monitoring of container status, resulting in significant delays in management decisions. Summary of the Invention

[0007] To address the shortcomings of existing technologies, the purpose of this invention is to provide a container file management method and system based on digital twins. It aims to solve technical problems in existing container file management technologies, such as cumbersome image acquisition and management processes, lack of accurate comparison mechanisms for multiple time periods, and the disconnect between digital files and physical status management.

[0008] A container file management method based on digital twins according to the present invention includes: Step S1: Acquire a container image, perform recognition and correction processing on the image, and paste the processed image as a texture onto a preset white label container 3D model to construct a digital twin corresponding to the appearance of the container; Step S2: When images of the same container at different time points are acquired, the regions corresponding to the same position in the images at different time points are compared. The comparison is divided into two comparison routes according to the image coverage: the complete standardized container surface image adopts the global SIFT registration route, and the local non-standardized container surface image adopts the stable structure anchor point local matching route; the two types of images are uniformly projected onto the unified UV coordinate system in the digital twin to complete spatial alignment and quantify the appearance state changes of the same physical area of ​​the container. Step S3: Using the digital twin as the core, integrate the basic information, historical images, and status change analysis results of the container to construct and dynamically update the digital archive of the container.

[0009] Preferably, the identification and correction process includes image preprocessing, box information identification, and image angle correction; The image preprocessing includes denoising, enhancement, and grayscale preprocessing; The box information recognition uses OCR to extract the box number and feature matching to determine the standard surface of the box to which the image belongs; The image angle correction is achieved by extracting feature points on the container edge, calculating the angle and offset between the image and the frontal viewpoint to generate a homography perspective transformation matrix, and then using the matrix to complete the mapping of all pixel coordinates, thereby correcting the distorted image into a standard orthophoto container surface image.

[0010] Preferably, before performing step S2 comparison, a four-level metadata hierarchical positioning strategy is set to lock the target box and box surface; Level A: Image metadata carries both the container number and container face identifier, allowing direct retrieval of all historical images of that container and that face. Level B: Metadata only contains the container number. After manually selecting the corresponding container, the process is transferred to Level A. Level C: The metadata only contains the container label. The container number is automatically recognized by OCR. If the recognition is successful, it is transferred to Level A. If the recognition fails, the container number is manually entered. Level D: Metadata is completely missing. Manually enter the container number and container label, or start a coarse image search of the entire database. After completing the positioning of the container and its surface, the corresponding sub-route comparison process is then executed.

[0011] Preferably, the local matching route for the stable structural anchor points includes: Step 2.1: Pre-train the structural anchor point detection model. The structural anchor point detection model can identify the stable inherent structure of the box over time, including primary anchor points: corner pieces and corner posts; secondary anchor points: door lock rods, door hinges, CSC nameplates, and side panel corrugations; and tertiary anchor points: ventilators, refrigeration units, and labels. Step S2.2: Detect all visible stable structure anchors in the local image, extract local feature descriptors of the anchor neighborhood, generate feature hash codes using Local Sensitive Hash (LSH), and match historical images containing the same anchors in the historical image hash index. Step S2.3: Map the current local image and the matching historical image to the UV coordinate system of the digital twin, find the intersection region of the UV space of the two images, perform SIFT fine pixel registration on the intersection region, and unify the comparison benchmark.

[0012] Preferably, the global SIFT registration route includes: performing pixel-level registration of the corrected images of the same box surface at different time periods through SIFT feature matching to eliminate the reference deviation caused by the slight displacement during shooting.

[0013] Preferably, the changes in the appearance of the same physical area of ​​the quantization box in step S2 include: Multi-channel feature layer extraction extracts three layers of feature matrices: RGB color channel, grayscale texture channel, and edge contour channel, to distinguish differences including color difference, texture difference, and contour difference; The pixel difference mask generation performs a difference operation on the feature matrices of the two images before and after the image to generate a difference mask matrix D(i,j), where the values ​​in the matrix represent the magnitude of the difference at the corresponding pixel positions. The region connected component segmentation binarizes the mask matrix to segment independent and differential connected components. Each connected component corresponds to a damaged / stained / clean area. The multi-dimensional quantitative index calculation calculates quantitative parameters for each connected component, including area quantization, damage degree quantization, location quantization, and rate of change quantization. The quantitative data import and visualization rendering process binds all quantitative indicators to the corresponding positions in the twin model, generating an labeled image comparison report.

[0014] Preferably, the digital archives adopt a three-tier linked storage architecture consisting of a structured database, object storage, and a 3D twin model; The structured database is used to store basic metadata of the archives, including box number, specifications, manufacturing information, timestamp of each collection, quantitative comparison indicators, maintenance records, responsible party for operation, and cleaning / repair work order number; The object storage is used to store the original acquired images, corrected textures, 3D twin files of various versions, visualization comparison report PDFs, and damage heat maps in a hierarchical manner. The twin model is used to automatically generate a new version of the 3D twin texture file after each data acquisition and update of the box, and archives it independently according to the timestamp, without overwriting the historical version.

[0015] A container file management system based on digital twins according to the present invention includes: The image processing module is used to perform container number recognition, container surface recognition, and angle correction on the acquired container images; The digital twin building module is used to paste processed images as textures onto a preset 3D model to build a digital twin of a shipping container; The image comparison module is used to perform difference comparison and quantitative analysis on images of the same container at different points in time. The database management module is used to store basic information about the containers, digital twins, historical images, and comparative analysis reports, forming a dynamically updated digital archive.

[0016] Preferably, the image comparison module includes: The metadata hierarchical retrieval unit is used to quickly locate the target container and its surface based on the four-level metadata positioning logic; The stable structure anchor matching unit has a built-in pre-trained structure anchor detection model and LSH hash retrieval engine, and executes the steps of the local matching route of the stable structure anchor. The UV space unified registration unit projects the local / complete image onto the twin model's UV coordinate system to find the intersection, achieving pixel-level fine alignment; The multi-channel quantization analysis unit extracts the feature matrices of RGB, grayscale, and edge layers, generates the difference mask matrix D(i,j), segments the connected components, and calculates four types of quantification indicators: area, damage level, UV location, and rate of change.

[0017] Preferably, the digital archives adopt a three-tier linked storage architecture consisting of a structured database, object storage, and a 3D twin model; The structured database is used to store basic metadata of the archives, including box number, specifications, manufacturing information, timestamp of each collection, quantitative comparison indicators, maintenance records, responsible party for operation, and cleaning / repair work order number; The object storage is used to store the original acquired images, corrected textures, 3D twin files of various versions, visualization comparison report PDFs, and damage heat maps in a hierarchical manner. The twin model is used to automatically generate a new version of the 3D twin texture file after each data acquisition and update of the box, and archives it independently according to the timestamp, without overwriting the historical version.

[0018] Compared with the prior art, the present invention has the following beneficial effects: 1. This invention transforms traditional paper-based, discrete archives into dynamic, visualized archives centered on 3D digital twins, realizing the digitization and visualization of archive management, making information presentation more intuitive and management clear at a glance.

[0019] 2. This invention replaces subjective human judgment with automated image comparison and quantitative analysis, making the assessment of container aging and damage, repair and maintenance, cleaning and painting more accurate and objective, thereby effectively reducing potential business disputes.

[0020] 3. This invention automates the entire process of image processing, archiving, and comparison, significantly reducing manual intervention, improving the efficiency of file management, and lowering related manpower and communication costs.

[0021] 4. This invention establishes a complete historical image and status change chain, realizing traceable management of the container's status throughout its entire life cycle from factory to scrap, providing reliable data for asset valuation and risk control. Attached Figure Description

[0022] Other features, objects, and advantages of the present invention will become more apparent from the following detailed description of non-limiting embodiments with reference to the accompanying drawings: Figure 1 This is a flowchart illustrating a container file management method based on digital twins, provided as an embodiment of the present invention. Detailed Implementation

[0023] The present invention will now be described in detail with reference to specific embodiments. These embodiments will help those skilled in the art to further understand the present invention, but do not limit the invention in any way. It should be noted that those skilled in the art can make several changes and improvements without departing from the concept of the present invention. These all fall within the protection scope of the present invention.

[0024] This invention uses a 3D model of a white-label shipping container as a virtual carrier. Images of various sides of the container collected through different channels during daily operations are processed, identified, and corrected before being pasted into the 3D model to construct a digital twin of the container. An image comparison module accurately compares images of the same location at different times, quantifying and analyzing the container's aging, wear, cleanliness, and repair status. A dynamically updated digital archive is built, enabling visualized and traceable management of the container's status. Example 1 According to the present invention, a container file management method based on digital twins is provided, such as... Figure 1 As shown, it includes the following steps: Step S1: Establish the basic support system. This basic support system includes a virtual carrier and technical modules such as an image recognition module, an image angle transformation module, a 3D texture pasting module, an image comparison module, and a database management module. Specifically, a 3D model of a white-label container with specifications consistent with commonly used containers (20GP / OT / FR / RF, 40GP / HQ / OT / FR / RF, etc.) is created. The texture pasting areas of the 12 faces of this 3D model (front (door photo), back, left side, right side, top, bottom, and corresponding interior face) are clearly defined, providing a virtual foundation for the visualization of container status and multi-time period comparison. The image recognition module is used to recognize the container number (with an accuracy rate of no less than 98%) and the corresponding face of the container. The image angle transformation module is used to correct non-frontal view images to an orthophoto frontal view. The 3D texture pasting module is used to paste the corrected images onto the 3D model to construct a digital twin. The image comparison module is used to perform difference analysis and quantification of images from the same location at different times. The database management module is used to store, retrieve, and back up all image data and related information.

[0025] These pre-built white-label 3D models are stored in the database management module for later use. As an optional implementation, the database management module can adopt an architecture that combines a relational database (such as PostgreSQL) with an object storage (such as MinIO). The former is used to store structured data such as box number, size, status, and reports, while the latter is used to store massive amounts of raw images and model files.

[0026] Step S2: Intelligent image processing and digital twin construction. Step S2 includes: Step S2.1 Image Recognition and Verification: Fixed cameras or mobile terminals (such as smartphones or dedicated data acquisition devices) deployed at the yard gate, hoisting equipment, and repair area can be used to capture multiple surface images of the container, thereby obtaining container image data. The container image data is then imported into the image recognition module. Preprocessing is performed on the images, including denoising, enhancement, and grayscale conversion. For example, Gaussian filters are used to eliminate sensor noise or compress artifacts, and histogram equalization is employed to enhance image contrast and highlight container details (such as rust and scratches). Finally, the images are converted to grayscale to simplify subsequent calculations.

[0027] Subsequently, container number and container face recognition are performed. Container number information is extracted using OCR recognition technology, and the specific container face corresponding to the image is identified through image feature matching. Specifically, the extraction of container number information using OCR recognition technology includes: 1) Object detection: detecting the position coordinates of the container number, cropping a partial image containing only the container number characters, binarizing the cropped image, and then using OCR to identify the corresponding container number; 2) Multimodal OCR inference: performing multimodal OCR visual inference recognition of the container number on the image, and outputting an array of container numbers. The process of identifying the specific container face corresponding to the image through image feature matching includes: pre-storing standard contours, corner pieces, door locks, and ventilation opening feature templates for 12 standard faces of various types of containers such as 20GP / 40HQ / RF in the database; extracting stable feature points such as corner points, container borders, door lock structures, and ventilation openings from the image to be identified using the SIFT algorithm; performing feature matching between the extracted feature points and the 12 standard face templates one by one, and calculating the similarity score of each template; taking the standard face with the highest similarity score as the container face corresponding to the current image; if the highest similarity is lower than the threshold, it is marked as an abnormal shooting angle and pushed to the image angle transformation module for correction and secondary identification.

[0028] Finally, a combination of automatic verification (comparing with the container ledger) and manual verification is used to ensure that the identification results are accurate.

[0029] Step S2.2 Image Angle Correction: For images acquired from non-frontal perspectives (understandably, when acquiring container image data, perspective distortion may exist due to shooting angle issues; for example, the camera may not be parallel to the container surface during shooting, resulting in an image appearing larger in the foreground and smaller in the background), the image angle transformation module extracts the container edge feature points, calculates the angle and offset between the image and the frontal perspective, and assumes that the set of pixel coordinates of the four corner points of the actual container is obtained through the extraction of edge feature points: P={p1(x1,y1),p2(x2,y2),p3(x3,y3),p4(x4,y4)} Standard coordinate set of corner points of the box in a standard orthophoto view: Q={q1(u1,v1),q2(u2,v2),q3(u3,v3),q4(u4,v4)} Formula for calculating the angle of view deflection: θ=arctan(ΔX avg / ΔZ avg ) In the formula, θ represents the horizontal deflection angle of the front image relative to the standard orthophoto front, in degrees, and ΔX avg ΔZ represents the average pixel offset in the horizontal direction of the image box. avg This represents the average pixel difference along the depth direction of the box. Formula for calculating vertical pitch angle: φ=arctan(ΔY avg / ΔZ avg ) In the formula, φ represents the vertical pitch angle of the current image relative to the standard orthophoto front, in degrees, and ΔY avg This represents the average pixel offset along the vertical axis of the image box. Formula for calculating the overall pixel offset of an image: δ= In the formula, δ represents the total pixel offset of the image relative to the standard frontal view.

[0030] After calculating the three parameters θ, φ, and δ, the perspective transformation matrix is ​​input to complete geometric correction. A perspective transformation and geometric correction algorithm is used to correct the image to an orthographic frontal view. Specifically, a basic rotation matrix is ​​generated based on the included angles θ and φ, and a translation matrix is ​​generated by combining this with the offset δ. These are then fused to obtain the homography perspective transformation matrix M. Pixel coordinate mapping transformation maps all pixels in the image to be corrected through matrix M, mapping the tilted and perspective-distorted pixel coordinates to the standard orthographic plane coordinate system. Image boundary filling and cropping: After mapping, blank distortion areas appear at the image edges. The average pixel value of the box is used for boundary filling, and then the effective image containing only the box is cropped. Gray-scale gradient detail restoration: Local contrast restoration is performed on the mapped image to restore subtle textures such as rust, scratches, and markings, avoiding loss of detail after transformation. Correction result verification output: The box feature points of the corrected image are re-extracted and matched a second time with the standard template. If the similarity meets the standard, the corrected image is output; if the similarity does not meet the standard, the included angle parameters are recalculated and corrected again. After correction, details such as stains, wear, and markings on the container in the image are preserved to ensure consistency with the actual appearance of the container.

[0031] Step S2.3 Digital Twin Construction: The calibrated image is precisely attached to the corresponding face of the white label container 3D model using the 3D texture pasting module; as the number of images collected daily increases, the texture information of each face of the 3D model is gradually supplemented until the textures of all six faces of the container are completed, constructing a digital twin that is completely consistent with the appearance of the real container, and updating it synchronously to the database management module.

[0032] Step S3: Comparison and digital management of images from the same location across multiple time periods. Step S3 includes: Step S3.1 Image Retrieval and Comparison Trigger: Based on the container company's management needs, images of the same container, at the same location (same side), and from different time periods are retrieved through the database management module, triggering the image comparison module for comparison and quantitative analysis. The comparison process can be triggered in various ways. For example, in business scenarios such as container leasing and return, the system can trigger it automatically; or, the administrator can manually select a container in the system and specify two or more time points for comparison according to management needs (such as periodic inspections and damage tracing).

[0033] Step S3.2 Difference Analysis and Report Generation: The image comparison module performs feature extraction and alignment processing on the retrieved multi-time period images, accurately identifies and quantifies the difference features in the images, including natural aging traces, artificial wear traces, changes before and after cleaning of the dirty tank, changes before and after repair of the damaged tank, etc., and generates a visual comparison report to clarify the difference location, degree of difference and trend of change.

[0034] Specifically, pixel-level precise alignment of images involves using SIFT feature matching to register corrected images of the same box surface from different time periods, eliminating reference deviations caused by minute shooting displacements. After alignment, multi-channel feature layer extraction is performed, extracting feature matrices from the RGB color channel, grayscale texture channel, and edge contour channel to distinguish three types of differences: color difference (rust, stains), texture difference (scratches, wear), and contour difference (deformation, dents). Next, a pixel difference mask is generated by performing a difference operation on the feature matrices of the two images to generate a difference mask matrix D(i,j), where the values ​​in the matrix represent the magnitude of the difference at the corresponding pixel positions. Then, connected component segmentation is performed, binarizing the mask matrix to segment independent difference connected components, with each connected component corresponding to a damaged / stain / clean area. Finally, multi-dimensional quantization index calculation is performed, calculating four types of quantization parameters for each connected component. Area quantization: The total pixel area of ​​the difference region is converted into the actual physical area of ​​the box (cm²). Damage severity quantification: Regional average difference amplitude, divided into three levels: mild / moderate / severe; Location quantization: Record the coordinates of connected components (doors, side panels, corner pieces, bottom, etc.); Rate of change quantification: the rate at which the area of ​​difference changes per unit time.

[0035] The quantitative data import and visualization rendering process binds all quantitative indicators to the corresponding positions in the twin model, generating an labeled image comparison report.

[0036] Step S3.3 Digital Management and Control: Based on comparison reports, achieve digital management of container status. For natural aging, analyze the aging rate and location, and formulate targeted maintenance plans; for human-induced wear and tear, trace the wear and tear scenarios and responsibilities, and optimize the usage and transfer process; for cleaning soiled containers, quantify the cleaning effect and optimize cleaning standards; for repairing damaged containers, inspect the repair quality to ensure that it meets usage requirements.

[0037] Step S4: Digital archive construction and dynamic updating. Step S4 includes: Step S4.1 Digital Archive Construction: Using the container digital twin as the core, a complete digital archive is constructed. The archive content includes basic container information (container number, specifications, manufacturer, production date), daily collected original images, corrected images, 3D twin models, collection information, image comparison reports, maintenance and repair records, etc., forming a standardized archive ledger. The digital archive adopts a three-layer linked storage architecture of structured database, object storage, and 3D twin models. The structured database stores basic metadata, including container number, specifications, production information, timestamps of each collection, quantitative comparison indicators, maintenance records, responsible parties for operations, and cleaning / repair work order numbers. The object storage is used for hierarchical storage of original collected images, corrected textures, 3D twin files of various versions, visual comparison reports (PDF), and damage heatmaps. The twin model is used to automatically generate a new version of the 3D twin texture file after each collection update of the container, archived independently by timestamp, without overwriting historical versions.

[0038] Step S4.2 Dynamic Update: When the appearance of the container changes (aging, wear, cleaning, repair, etc.), the corresponding images are supplemented through the daily collection process. The operations of steps S3 and S4 are repeated to update the 3D twin and digital archives, ensuring that the virtual model, archive information and physical container are always synchronized. At the same time, functions such as archive query, twin model viewing and historical image tracing are provided to support management decisions. Specifically, the file retrieval entry triggers the user to input the target container number, and the system reads the container's full lifecycle timeline index table. The index table automatically arranges all collection nodes in ascending order of time. The timeline node visualization rendering front end displays an interactive timeline, with each time node bound to the corresponding version of the 3D twin model, the original image of the current period, and the current damage quantification report. It supports clicking on any node to jump to view the complete status of the container in the current period. For cross-time period status comparison and tracing, selecting any two nodes on the timeline automatically calls the image comparison module, loads the container surface textures corresponding to the two nodes, and automatically generates a cross-time period difference quantification report, intuitively displaying the changes in container aging and wear within that time period. For full-link data traceability, clicking on any damage record within the timeline allows for reverse tracing: collection personnel, collection equipment, site location, original image taken at the time, previous maintenance work orders, and responsibility determination records, achieving complete closed-loop tracing.

[0039] Example 2 This embodiment, based on the method described in Embodiment 1, further elaborates on the specific application of the scheme in different management scenarios to support the analysis of multiple state changes.

[0040] As an application, the present invention can be used to monitor the natural aging process of shipping containers. For a container that has been in maritime transport for a long time, the paint on its surface will gradually fade and chalk due to sun and rain exposure, and rust may appear in some areas and spread over time. In order to develop a scientific refurbishment and maintenance plan, it is necessary to quantitatively assess this degree of aging.

[0041] In practice, an annual inspection cycle can be set. Every year when a container enters the designated yard, the system will automatically or manually collect standardized images of the six outer surfaces of the container and update the digital twin archive timeline synchronously.

[0042] Suppose an administrator needs to assess the aging condition of a shipping container over the past three years. They can select the container in the system and specify the comparison point, such as comparing an image from three years ago with the most recent image. The image comparison module is triggered, comparing images of the same surface (e.g., the top surface) from two different periods. It extracts only differences in features such as natural rust, paint fading, and weathering, filtering out man-made scratches and collision damage. An aging rate threshold V0 is set. Based on historical data along the timeline, the system calculates the aging rate of each container area, automatically generating regional aging heatmaps. For containers with aging rates exceeding the threshold, preventative anti-corrosion maintenance plans are proactively pushed out and stored in a digital archive for long-term preservation.

[0043] In another application, the present invention can also be used to determine liability for damage caused by human factors to container bodies. For example, after a railway transport, a long scratch is found on the left side of a container. Since the container passes through multiple responsible parties during transport (such as the originating yard, the railway carrier, and the destination yard), determining the point at which the scratch occurred is crucial for assigning responsibility. Complete images of the container are collected at both the outbound and return points of the container rental, generating two time-point twin models. By comparing the outbound and return images, areas with non-natural aging differences, such as collision dents, human-caused scratches, and cargo friction wear, are filtered out, and the damaged area and damage level are quantified. The timeline is linked to the rental order and carrier information, automatically generating a wear liability determination report, marking the location and quantified dimensions of any new damage, serving as evidence in rental business disputes, permanently archived and tamper-proof.

[0044] In another application, the present invention can also be used for the quantitative acceptance of the cleaning effect of the dirty bin. Images of the bin are collected before and after the cleaning operation to construct twin maps before and after cleaning; the area of ​​removal of stains, oil stains, and residual goods traces is quantified, and the cleaning completion rate before and after is calculated; based on the cleaning completion rate, it is determined whether the standard is met, a standardized cleaning acceptance report is generated, and it is stored in the timeline archive of the bin for the purpose of evaluating the work quality of the cleaning service provider.

[0045] In another application, the present invention can also be used for quality inspection of damaged container repairs. Images are collected twice: once upon arrival at the repair site for damage assessment, and again upon completion and departure from the factory. Pre- and post-repair twin models are then created. The repaired areas are compared to quantify the smoothness of dent repairs, the area covered by rust repainting, and the matching degree of damaged parts replacement. Simultaneously, any new repair damage is identified. A repair quality inspection report is output, and the archive timeline is updated synchronously, recording the repaired areas and quantitative repair indicators, providing data for subsequent residual value assessment and re-leasing acceptance.

[0046] Example 3 The present invention also provides a container file management system based on digital twins. The container file management system based on digital twins can be implemented by executing the process steps of the container file management method based on digital twins. That is, those skilled in the art can understand the container file management method based on digital twins as a preferred embodiment of the container file management system based on digital twins.

[0047] According to the present invention, a container file management system based on digital twins includes: an image processing module for performing container number recognition, container surface recognition, and angle correction on acquired container images; recognizing the container number in the image using optical character recognition technology; recognizing the container surface to which the image belongs through image feature matching; and, when the image is a non-orthographic viewpoint, extracting container edge feature points, calculating the angle and offset between the image and the frontal viewpoint, and correcting it to an orthographic frontal viewpoint using perspective transformation and geometric correction algorithms.

[0048] The digital twin building module is used to paste processed images as textures onto a preset 3D model to build a digital twin of a shipping container; The image comparison module is used to perform difference comparison and quantitative analysis on images of the same container at different time points. The quantitative analysis includes: performing pixel-level registration on corrected images of the same container surface at different time periods using SIFT feature matching to eliminate the reference deviation caused by slight displacement during shooting; multi-channel feature layer extraction to extract three layers of feature matrices: RGB color channel, grayscale texture channel, and edge contour channel, distinguishing differences including color difference, texture difference, and contour difference; pixel difference mask generation by performing a difference operation on the feature matrices of the two images to generate a difference mask matrix D(i,j), where the values ​​in the matrix represent the magnitude of the difference in the corresponding pixel position; region connected component segmentation by binarizing the mask matrix to segment independent difference connected components, with each connected component corresponding to a damaged / stain / clean area; multi-dimensional quantitative index calculation for calculating quantitative parameters for each connected component, including area quantization, damage degree quantization, position quantization, and rate of change quantization; and quantitative data storage and visualization rendering by binding all quantitative indicators to the corresponding positions in the twin model and generating an annotated image comparison report. The image comparison module further includes: a metadata hierarchical retrieval unit, used to quickly locate the target box and box surface according to the four-level metadata positioning logic; a stable structure anchor point matching unit, which has a built-in pre-trained structure anchor point detection model and LSH hash retrieval engine, and executes the steps of the local matching route of the stable structure anchor point; a UV space unified registration unit, which projects the local / complete image onto the UV coordinate system of the twin model to find the intersection and complete pixel-level fine alignment; and a multi-channel quantization analysis unit, which extracts the RGB, grayscale, and edge three-layer feature matrices, generates the difference mask matrix D(i,j), segments the connected components, and calculates four types of quantization indicators: area, damage level, UV position, and rate of change.

[0049] The database management module stores basic container information, digital twins, historical images, and comparative analysis reports, forming a dynamically updated digital archive. This digital archive employs a three-tiered, interconnected storage architecture: a structured database, object storage, and 3D twin models. The structured database stores basic metadata, including container number, specifications, manufacturing information, timestamps for each data collection, quantitative comparison indicators, maintenance records, responsible parties for operations, and cleaning / repair work order numbers. The object storage layer stores original collected images, corrected textures, 3D twin files of various versions, PDF visualization comparison reports, and damage heatmaps. The twin model automatically generates a new version of the 3D twin texture file after each data collection update, archives it independently by timestamp, and does not overwrite previous versions.

[0050] Example 4 This embodiment provides a method for cross-period tracing and change detection of local box surface images, as a supplement and deepening of the image comparison and digital management described in step S3 of Embodiment 1. This method aims to solve the technical problem of how to compare local box surface images with incomplete coverage and non-standardized shooting angles during actual acquisition operations.

[0051] It is understandable that the image comparison method described in Example 1 is based on the premise that the images to be compared are full-face corrected images of the same container and the same side. Under this ideal condition, pixel-level registration can be well achieved through global feature matching, and subsequent differential and quantization analyses can also run stably. However, in many practical business scenarios, such as spot checks during maintenance, tracing specific damage, local confirmation after cleaning of soiled containers, and final acceptance after repair of damaged containers, on-site personnel often only take pictures of a local area of ​​the container. The resulting "local container surface image" may only cover a small part of the corresponding complete container surface, and its shooting angle, scale, distance, etc., may differ significantly from the full-face images in historical archives or local images from other periods.

[0052] For such localized images, directly applying the existing global registration method will fail because the two images (e.g., a local image and a historical full-area image) lack a sufficiently large overlap area, resulting in an insufficient number of corresponding feature point pairs. This leads to registration failure. Registration failure directly prevents subsequent steps such as pixel differencing, connected component segmentation, and quantization analysis from being performed, thus hindering the ability to trace and analyze the changes in the "same physical area within the same container" across different time periods. Furthermore, damage itself (such as rust, stains, and breakage) changes over time; using these variable features as the basis for cross-period matching makes it difficult to guarantee both recall and precision.

[0053] To address this, this embodiment proposes a more robust local image matching method. This method employs a metadata-priority and dual-path hierarchical architecture: ① It prioritizes using the acquired metadata (box number, face identifier) ​​to locate the historical image set of the box and face for precise registration, skipping the full database search; ② When metadata is missing, it degrades step-by-step according to "face selection / OCR / full database search"; ③ It retains the original SIFT global registration; ④ It adds ROI neighborhood stable features, LSH hashing, and unified UV grid difference as supplementary methods.

[0054] Specifically, it includes the following steps: Step S301: New Image Import. The process begins when a new, possibly localized, box surface image is acquired and uploaded to the system.

[0055] Step S302: Check metadata integrity. The system first checks the metadata uploaded along with the image. Based on the integrity of the metadata, four different levels of sub-processes are initiated: Level A: The metadata is the most complete, including accurate "box number" and "box face" identifiers. This is the ideal situation, where the system can directly narrow the search scope to all historical image sets of that specific box number and specific box face in the database, and then directly proceed to the subsequent fine registration process.

[0056] Level B: The metadata only contains the "box number" but no "box face" identifier. In this case, the system will present the user with a selection interface containing 12 standard faces of this box type, guiding the user to manually specify which face the local image belongs to. Once the user selects, the problem transforms into the Level A situation.

[0057] Level C: The metadata may contain a "box face" identifier, but no "box number". In this case, the system will first call the optical character recognition module to attempt to automatically identify the box number from the image. If the recognition is successful, it will proceed to Level A; if it fails, the user will be prompted to manually enter the box number.

[0058] Level D: Metadata is completely missing, with neither container number nor container surface information. The system will require the user to manually enter the container number and surface information. As an alternative, the system can also initiate a "full-database coarse localization" process based on image content, that is, using the visual features of the local image to perform a large-scale search throughout the entire historical image database to find the most similar candidate image, but this is usually computationally expensive and has limited accuracy.

[0059] Through the above-mentioned hierarchical positioning strategy, regardless of the initial metadata status, the problem can ultimately be reduced to the case of known box number and box face, laying the foundation for subsequent accurate comparison.

[0060] After completing the container number and container surface location, the system proceeds to step S303: Image Capability Assessment. This step automatically determines which comparison technique is more suitable for the current local image. The system evaluates multiple image attributes, such as whether the image covers the entire container surface and whether the shooting parameters are within the preset calibration range. If the evaluation result indicates that the image is a complete container surface image taken by a fixed industrial camera at the site under standardized camera positions, the system will select to proceed to step S304: Global Registration Route. This route is the method described in Example 1, which has fast calculation speed and high global accuracy. Conversely, if the evaluation determines that the image was taken by a handheld terminal under non-standardized conditions, with significant lighting, viewing angle, or field of view deviations, or if the image itself is a local image, the system will select to proceed to step S305: Local Feature Matching Route.

[0061] Step S305: Local Feature Matching Route. This is the method proposed in this embodiment for non-standardized local images. Its technical insight lies in the fact that intertemporal matching must be based on "intertemporally stable" features, rather than the damaged areas (rust, breakage, stains, repair marks) that change over time as the object being detected. Truly stable features are the inherent, standardized structural textures of the container (including corrugated segments, rivet neighborhoods, and corner component local structures) within the neighborhood of the damaged area (e.g., within 0.3 to 1 meter). To achieve this, the system pre-trains a structural anchor detection model using a deep learning model. This detector can detect and locate various types of structural anchors in any input container image, such as: Primary structural anchor points: including ISO standard corner brackets (8 per box) and corner posts (length consistent with box height, 4 per box).

[0062] Secondary structural anchor points: including door lock rods, door hinges, CSC safety nameplates, ISO box number labels, and side panel corrugations.

[0063] Level 3 structural anchor points include at least one of the following: hazardous materials labels, inspection and approval labels, ventilators, refrigeration units, and manufacturer markings.

[0064] Primary structural anchor points are mandatory, while secondary and tertiary structural anchor points are dynamically activated based on the box shape and shooting scene. A structural anchor point detection model was pre-trained using a deep learning model; the training data and process are as follows: First, collect more than 3,000 images of containers covering different container types (20GP / 40GP / 40HQ / OT / FR / RF), different viewing angles (orthogonal / 15° / 30° / 45° / 60°), different degrees of damage (intact / minor damage / moderate damage / severe damage), and different lighting conditions (front lighting / backlighting / cloudy / nighttime supplemental lighting); Then, labeling tools such as LabelImg / LabelMe were used for labeling; there were 12 labeling categories in total: ① Corner fittings (cls=0); ② Corner posts (cls=1); ③ Door lock rods (cls=2); ④ Door hinges (cls=3); ⑤ CSC safety nameplates (cls=4); ⑥ ISO box number labels (cls=5); ⑦ Side panel corrugations (cls=6); ⑧ Manufacturer's logo (cls=7); ⑨ Refrigeration unit (cls=8); ⑩ Inspection certificate labels (cls=9). Dangerous goods label (cls=10); Ventilator (cls=11); Next, each anchor point of each image is labeled as a quintuple (class_id, x_center, y_center, width, height), where (x_center, y_center) are the pixel coordinates of the center point of the bounding box, and (width, height) are the pixel values ​​of the width and height of the bounding box; Finally, the dataset was divided into training, validation, and test sets in an 8:1:1 ratio.

[0065] When employing a local feature matching approach, the system first runs the structural anchor detection model on the newly input local image to find all stable structural anchors within its visible range. Then, the system extracts local feature descriptors centered on these stable structural anchors. Next, using locality-sensitive hashing (LSH), these high-dimensional feature descriptors are hashed into short binary codes. The characteristics of LSH ensure that features similar in the original feature space are likely to have the same hash code. The system can use these hash codes to find historical images containing the same or similar structural anchor features in a pre-built historical image feature hash index. This process replaces global registration, enabling rapid mapping from a local image to the corresponding physical region in another historical image (which could be a full-area image or another local image).

[0066] Step S306: Fine Registration and Difference Analysis. Whether through step S304 or S305, a rough correspondence between the current local image and one or more historical candidate images is obtained. The goal of this step is to perform a unified and precise difference analysis based on this. The core idea is to use the UV coordinate system of the 3D twin model as a unified reference. For each pair of matched images (current local image and a historical image), the system performs the following operations: 1. Project both images onto the UV coordinate system of the twin model using their respective homography transformation matrices (which describe the mapping relationship from the 2D image to the UV coordinates of the 3D model, and can be calculated through anchor points or global registration). Calculate the intersection of the areas covered by the two images in UV space.

[0067] 2. Project this shared coverage area back from UV space onto the two original images to obtain two image patches that require fine comparison. At this point, perform a fine pixel-level registration on these two image patches (which now cover the exact same physical area) using the SIFT algorithm to eliminate any final minor discrepancies.

[0068] 3. After registration, the difference analysis process can be performed exactly the same as step S3.2 in Example 1: that is, perform difference operation on the feature matrix to generate difference mask matrix D(i,j). In order to reduce the false differences caused by illumination changes, histogram matching can be performed before RGB channel difference. Then, binarize and segment the mask matrix. Finally, calculate four quantization indicators for each connected component.

[0069] A key difference is that the location quantification index is now recorded as the (u,v) coordinates of the center point of the connected domain in the unified UV coordinate system of the twin model, as well as its distance and azimuth from the origin of the reference coordinate system of its respective box surface.

[0070] 4. Finally, the system will write the four calculated quantitative indicators, along with their corresponding UV locations, acquisition times, original image identifiers, and other information, into the database and mark them as a difference event record that occurred in the UV area during that specific time period.

[0071] In this way, even scattered, non-standard local images can be effectively integrated into the dynamic digital archive of the container, achieving seamless and quantifiable traceability of changes in the state of any physical area of ​​the container.

[0072] Those skilled in the art will understand that, besides implementing the system and its various devices, modules, and units provided by this invention in the form of purely computer-readable program code, the same functions can be achieved entirely through logical programming of the method steps, making the system and its various devices, modules, and units of this invention function in the form of logic gates, switches, application-specific integrated circuits, programmable logic controllers, and embedded microcontrollers. Therefore, the system and its various devices, modules, and units provided by this invention can be considered as a hardware component, and the devices, modules, and units included therein for implementing various functions can also be considered as structures within the hardware component; alternatively, the devices, modules, and units for implementing various functions can be considered as both software modules implementing the method and structures within the hardware component.

[0073] Specific embodiments of the present invention have been described above. It should be understood that the present invention is not limited to the specific embodiments described above, and those skilled in the art can make various changes or modifications within the scope of the claims, which do not affect the essence of the present invention. Unless otherwise specified, the embodiments and features described in this application can be arbitrarily combined with each other.

Claims

1. A container file management method based on digital twins, characterized in that, include: Step S1: Acquire a container image, perform recognition and correction processing on the image, and paste the processed image as a texture onto a preset white label container 3D model to construct a digital twin corresponding to the appearance of the container; Step S2: When images of the same container at different time points are acquired, the regions corresponding to the same position in the images at different time points are compared. The comparison is divided into two comparison routes according to the image coverage: the complete standardized container surface image adopts the global SIFT registration route, and the local non-standardized container surface image adopts the stable structure anchor point local matching route; the two types of images are uniformly projected onto the unified UV coordinate system in the digital twin to complete spatial alignment and quantify the appearance state changes of the same physical area of ​​the container. Step S3: Using the digital twin as the core, integrate the basic information, historical images, and status change analysis results of the container to construct and dynamically update the digital archive of the container.

2. The container file management method based on digital twins according to claim 1, characterized in that, The recognition and correction process includes image preprocessing, box information recognition, and image angle correction. The image preprocessing includes denoising, enhancement, and grayscale preprocessing; The box information recognition uses OCR to extract the box number and feature matching to determine the standard surface of the box to which the image belongs; The image angle correction is achieved by extracting feature points on the container edge, calculating the angle and offset between the image and the frontal viewpoint to generate a homography perspective transformation matrix, and then using the matrix to complete the mapping of all pixel coordinates, thereby correcting the distorted image into a standard orthophoto container surface image.

3. The container file management method based on digital twins according to claim 1, characterized in that, Before performing step S2 comparison, set a four-level metadata hierarchical positioning strategy to lock the target container and container surface; Level A: Image metadata carries both the container number and container face identifier, allowing direct retrieval of all historical images of that container and that face. Level B: Metadata only contains the container number. After manually selecting the corresponding container, the process is transferred to Level A. Level C: The metadata only contains the container label. The container number is automatically recognized by OCR. If the recognition is successful, it will be transferred to Level A. If the recognition fails, the container number will be manually entered. Level D: Metadata is completely missing. Manually enter the container number and container label, or start a coarse image search of the entire database. After completing the positioning of the container and its surface, the corresponding sub-route comparison process is then executed.

4. The container file management method based on digital twins according to claim 1, characterized in that, The local matching route for the stable structure anchor points includes: Step 2.1: Pre-train the structural anchor point detection model. The structural anchor point detection model can identify the stable inherent structure of the box over time, including primary anchor points: corner pieces and corner posts; secondary anchor points: door lock rods, door hinges, CSC nameplates, and side panel corrugations; and tertiary anchor points: ventilators, refrigeration units, and labels. Step S2.2: Detect all visible stable structure anchors in the local image, extract local feature descriptors of the anchor neighborhood, generate feature hash codes using Local Sensitive Hash (LSH), and match historical images containing the same anchors in the historical image hash index. Step S2.3: Map the current local image and the matching historical image to the UV coordinate system of the digital twin, find the intersection region of the UV space of the two images, perform SIFT fine pixel registration on the intersection region, and unify the comparison benchmark.

5. The container file management method based on digital twins according to claim 1, characterized in that, The global SIFT registration route includes: performing pixel-level registration of corrected images of the same box surface at different time periods through SIFT feature matching to eliminate reference deviations caused by minute displacements during shooting.

6. The container file management method based on digital twins according to claim 1, characterized in that, The changes in the appearance of the same physical area of ​​the quantization box in step S2 include: Multi-channel feature layer extraction extracts three layers of feature matrices: RGB color channel, grayscale texture channel, and edge contour channel, to distinguish differences including color difference, texture difference, and contour difference; The pixel difference mask generation performs a difference operation on the feature matrices of the two images before and after the image to generate a difference mask matrix D(i,j), where the values ​​in the matrix represent the magnitude of the difference at the corresponding pixel positions. The region connected component segmentation binarizes the mask matrix to segment independent and differential connected components. Each connected component corresponds to a damaged / stained / clean area. The multi-dimensional quantitative index calculation calculates quantitative parameters for each connected component, including area quantization, damage degree quantization, location quantization, and rate of change quantization. The quantitative data import and visualization rendering process binds all quantitative indicators to the corresponding positions in the twin model, generating an labeled image comparison report.

7. The container file management method based on digital twins according to claim 1, characterized in that, The digital archives adopt a three-tiered linked storage architecture consisting of a structured database, object storage, and a 3D twin model; The structured database is used to store basic metadata of the archives, including box number, specifications, manufacturing information, timestamp of each collection, quantitative comparison indicators, maintenance records, responsible party for operation, and cleaning / repair work order number; The object storage is used to store the original acquired images, corrected textures, 3D twin files of various versions, visualization comparison report PDFs, and damage heat maps in a hierarchical manner. The twin model is used to automatically generate a new version of the 3D twin texture file after each data acquisition and update of the box, and archives it independently according to the timestamp, without overwriting the historical version.

8. A container file management system based on digital twins, characterized in that, include: The image processing module is used to perform container number recognition, container surface recognition, and angle correction on the acquired container images; The digital twin building module is used to paste processed images as textures onto a preset 3D model to build a digital twin of a shipping container; The image comparison module is used to perform difference comparison and quantitative analysis on images of the same container at different points in time. The database management module is used to store basic information about the containers, digital twins, historical images, and comparative analysis reports, forming a dynamically updated digital archive.

9. The container file management system based on digital twins according to claim 1, characterized in that, The image comparison module includes: The metadata hierarchical retrieval unit is used to quickly locate the target container and its surface based on the four-level metadata positioning logic; The stable structure anchor matching unit has a built-in pre-trained structure anchor detection model and LSH hash retrieval engine, and executes the steps of the local matching route of the stable structure anchor. The UV space unified registration unit projects the local / complete image onto the twin model's UV coordinate system to find the intersection, achieving pixel-level fine alignment; The multi-channel quantization analysis unit extracts the feature matrices of RGB, grayscale, and edge layers, generates the difference mask matrix D(i,j), segments the connected components, and calculates four types of quantification indicators: area, damage level, UV location, and rate of change.

10. The container file management system based on digital twins according to claim 1, characterized in that, The digital archives adopt a three-tiered linked storage architecture consisting of a structured database, object storage, and a 3D twin model; The structured database is used to store basic metadata of the archives, including box number, specifications, manufacturing information, timestamp of each collection, quantitative comparison indicators, maintenance records, responsible party for operation, and cleaning / repair work order number; The object storage is used to store the original acquired images, corrected textures, 3D twin files of various versions, visualization comparison report PDFs, and damage heat maps in a hierarchical manner. The twin model is used to automatically generate a new version of the 3D twin texture file after each data acquisition and update of the box, and archives it independently according to the timestamp, without overwriting the historical version.

Images