Container damage detection method and system, electronic device and medium

Abstract

The application relates to the technical field of digital door inspection, and discloses a container damage detection method and system, an electronic device and a medium. The method comprises the following steps: obtaining three-dimensional point cloud data of a container based on a laser radar; obtaining a container body surface image of the container based on an image acquisition device; calibrating and aligning the three-dimensional point cloud data and the container body surface image to obtain a transformation relationship; fusing and defect detecting the three-dimensional point cloud data and the container body surface image according to the transformation relationship and a heterogeneous modal fusion network to obtain a defect detection result; and generating a container damage detection report according to the defect detection result. Through key technologies such as pixel-level correlation of laser point cloud and video image alignment and laser-vision depth fusion defect detection, the application can stably operate container damage judgment under a complex environment, can make a fusion decision through combined detection of laser point cloud and video image in a double-path parallel mode, can realize high-precision quantification and all-weather robust detection, and can significantly improve the accuracy and efficiency of container damage identification under a complex environment.

Description

Technical Field

[0001] This invention relates to the field of digital door inspection technology, and in particular to a method, system, electronic device and medium for detecting damage to containers. Background Technology

[0002] With the continuous growth of railway freight volume, the demand for container inspection at railway freight yards is increasing. While traditional visual recognition technology has achieved automated detection of container damage to some extent, its accuracy and robustness are severely affected in complex environments, such as those with metallic reflections, shadows, and adverse weather conditions. Therefore, improving the container damage identification capabilities of gate inspection equipment at railway freight yards has become an urgent technical problem to be solved. Summary of the Invention

[0003] The main objective of this invention is to provide a container damage detection method, system, electronic device, and medium, aiming to solve at least one of the aforementioned technical problems.

[0004] In a first aspect, embodiments of the present invention provide a method for detecting container damage, comprising:

[0005] 3D point cloud data of containers are acquired using LiDAR;

[0006] Images of the container's surface are acquired using image acquisition equipment;

[0007] The 3D point cloud data and the box surface image are calibrated and aligned to obtain the transformation relationship;

[0008] Based on the transformation relationship and the heterogeneous modal fusion network, the three-dimensional point cloud data and the box surface image are fused to detect defects, and the defect detection results are obtained.

[0009] A box damage detection report is generated based on the defect detection results.

[0010] In some embodiments, the calibration and alignment of the three-dimensional point cloud data and the box surface image to obtain the transformation relationship includes:

[0011] Based on the three-dimensional point cloud data, contour segmentation and feature extraction are performed to obtain the three-dimensional coordinate information of the target container's body contour and key load-bearing structures.

[0012] Based on the three-dimensional coordinate information, the lidar-camera joint calibration board, and the extrinsic parameter calibration algorithm, the three-dimensional point cloud data and the box surface image are jointly calibrated to determine the transformation relationship between the lidar coordinate system and the camera pixel coordinate system.

[0013] In some embodiments, the step of performing contour segmentation and feature extraction based on the three-dimensional point cloud data to obtain the container body contour and three-dimensional coordinate information of key load-bearing structures includes:

[0014] The three-dimensional point cloud data is automatically identified and noise points and false targets are filtered out to obtain an effective point cloud;

[0015] The effective point cloud is denoised using height filtering, Euclidean clustering, and RANSAC plane fitting algorithms to separate the point cloud clusters of the target container.

[0016] Extract the outline of the point cloud cluster of the target container and the three-dimensional coordinate information of the key load-bearing structure.

[0017] In some embodiments, the automatic identification and filtering of noise points and false targets from the 3D point cloud data to obtain a valid point cloud includes:

[0018] The spatial density distribution, echo intensity continuity, and geometric rationality of the three-dimensional point cloud data are analyzed to obtain the analysis results.

[0019] Based on machine learning algorithms and the analysis results, false point clouds and false targets are identified and filtered to obtain clean point cloud data after noise reduction.

[0020] The denoised and clean point cloud data is used as the valid point cloud.

[0021] In some embodiments, the step of fusing the 3D point cloud data and the box surface image according to the transformation relationship and the heterogeneous modal fusion network to detect defects and obtain defect detection results includes:

[0022] Based on the transformation relationship, pixel-level aligned multimodal fusion data is constructed from the 3D point cloud data and the box surface image;

[0023] The multimodal fusion data is input into a heterogeneous modal fusion network, and geometric deformation features and visual texture features are analyzed simultaneously to obtain defect detection results.

[0024] In some embodiments, constructing pixel-level aligned multimodal fusion data based on the transformation relationship using the 3D point cloud data and the box surface image includes:

[0025] The depth value and echo intensity information of the three-dimensional point cloud data are used as additional feature channels;

[0026] Based on the transformation relationship, the additional feature channels are superimposed onto the RGB channels of the box surface image to construct multimodal fusion data including geometric information and visual texture information.

[0027] In some embodiments, generating a box damage detection report based on the defect detection results includes:

[0028] Based on the defect detection results, the lidar detection results and video image detection results are obtained;

[0029] Based on the decision-making logic that combines rule engine and weighted voting, a box damage detection report is generated by combining the lidar detection results and video image detection results.

[0030] Secondly, embodiments of the present invention provide a container damage detection system, comprising:

[0031] The laser point cloud acquisition module is used to acquire three-dimensional point cloud data of containers based on lidar.

[0032] The video image acquisition module is used to acquire images of the container's surface based on the image acquisition device;

[0033] The alignment module is used to calibrate and align the three-dimensional point cloud data and the box surface image to obtain the transformation relationship.

[0034] The fusion defect detection module is used to perform fusion defect detection on the three-dimensional point cloud data and the box surface image according to the transformation relationship and the heterogeneous modal fusion network, and obtain the defect detection result.

[0035] The multimodal fusion decision module is used to generate a box damage detection report based on the defect detection results.

[0036] Thirdly, embodiments of the present invention provide an electronic device, including:

[0037] One or more processors;

[0038] Memory, used to store one or more programs;

[0039] When the one or more programs are executed by the one or more processors, the one or more processors implement any of the methods described above.

[0040] Fourthly, embodiments of the present invention provide a computer-readable medium on which a computer program is stored, the computer program being executed by a processor to implement the steps of any of the methods described above.

[0041] This invention provides a method for detecting container damage, comprising: acquiring three-dimensional point cloud data of the container based on LiDAR; acquiring surface images of the container body based on an image acquisition device; calibrating and aligning the three-dimensional point cloud data and the container surface images to obtain a transformation relationship; performing fusion defect detection on the three-dimensional point cloud data and the container surface images according to the transformation relationship and a heterogeneous modal fusion network to obtain a defect detection result; and generating a container damage detection report based on the defect detection result. This invention utilizes key technologies such as pixel-level association through calibration and alignment of LiDAR point clouds and video images, and LiDAR-visual depth fusion defect detection to achieve stable container damage determination in complex environments. By combining parallel detection of LiDAR point clouds and video images with fusion decision-making, high-precision quantization and all-weather robust detection are achieved, significantly improving the accuracy and efficiency of container damage identification in complex environments. Attached Figure Description

[0042] Figure 1 A schematic flowchart of a container damage detection method provided in an embodiment of the present invention;

[0043] Figure 2 This is a schematic diagram of the key technical processes involved in the embodiments of the present invention;

[0044] Figure 3 This is a schematic diagram of the defect detection algorithm flow of laser-vision depth fusion involved in the embodiment of the present invention;

[0045] Figure 4 A structural block diagram of a container damage detection system provided in an embodiment of the present invention;

[0046] Figure 5 This is a structural block diagram of an electronic device provided in an embodiment of the present invention.

[0047] The realization of the objective, functional features and advantages of the present invention will be further explained in conjunction with the embodiments and with reference to the accompanying drawings. Detailed Implementation

[0048] To enable those skilled in the art to better understand the technical solutions of the present invention, exemplary embodiments of the present invention are described below in conjunction with the accompanying drawings, including various details of the embodiments of the present invention to aid understanding. These should be considered merely exemplary. Therefore, those skilled in the art should recognize that various changes and modifications can be made to the embodiments described herein without departing from the scope and spirit of the present invention. Similarly, for clarity and brevity, descriptions of well-known functions and structures are omitted in the following description.

[0049] Where there is no conflict, the various embodiments of the present invention and the features thereof may be combined with each other.

[0050] As used herein, the term “and / or” includes any and all combinations of one or more related enumerated entries.

[0051] The terminology used herein is for the purpose of describing particular embodiments only and is not intended to limit the invention. As used herein, the singular forms “a” and “the” are intended to include the plural forms as well, unless the context clearly indicates otherwise. It will also be understood that when the terms “comprising” and / or “made of” are used in this specification, the presence of the stated feature, integral, step, operation, element, and / or component is specified, but the presence or addition of one or more other features, integrals, steps, operations, elements, components, and / or groups thereof is not excluded. Terms such as “connected” or “linked” are not limited to physical or mechanical connections but can include electrical connections, whether direct or indirect.

[0052] Unless otherwise specified, all terms used herein (including technical and scientific terms) have the same meaning as commonly understood by one of ordinary skill in the art. It will also be understood that terms such as those defined in commonly used dictionaries should be interpreted as having the meaning consistent with their meaning in the context of the relevant art and the invention, and will not be interpreted as having an idealized or overly formal meaning unless expressly so defined herein.

[0053] In the technical solution of this invention, the collection, storage, use, processing, transmission, provision, and disclosure of user personal information all comply with relevant laws and regulations and do not violate public order and good morals. The use of user data in this technical solution follows relevant national laws and regulations (e.g., the "Information Security Technology - Personal Information Security Specification"). For example: appropriate measures are taken for personal information access control; restrictions are imposed on the display of personal information; the purpose of using personal information does not exceed the scope of direct or reasonable association; and explicit identity targeting is eliminated when using personal information to avoid precisely locating a specific individual.

[0054] In related technologies, railway freight yard gate inspection equipment mainly relies on single visual recognition technology, using industrial-grade cameras to capture images of containers and employing image processing algorithms to identify container damage. However, this approach is susceptible to interference in complex environments, leading to a high false alarm rate. Furthermore, these technologies lack effective utilization of the container's three-dimensional information, making it difficult to accurately quantify container damage. In addition, many systems in these technologies employ centralized computing architectures, resulting in significant data processing delays and making it difficult to meet real-time detection requirements.

[0055] To address at least one of the technical problems existing in the aforementioned related technologies, the present invention provides a method for detecting container damage. Figure 1 This is a flowchart illustrating a container damage detection method provided in an embodiment of the present invention.

[0056] As an embodiment of the present invention, such as Figure 1 As shown, the container damage detection method includes:

[0057] Step S100: Acquire 3D point cloud data of the container based on LiDAR;

[0058] Step S200: Acquire images of the container's surface using an image acquisition device;

[0059] Step S300: The three-dimensional point cloud data and the box surface image are calibrated and aligned to obtain the transformation relationship;

[0060] Step S400: Based on the transformation relationship and the heterogeneous modal fusion network, perform fusion defect detection on the three-dimensional point cloud data and the box surface image to obtain the defect detection result;

[0061] Step S500: Generate a box damage detection report based on the defect detection results.

[0062] It should be noted that the execution subject in this embodiment can be an electronic device, which can be a computer device with data processing function, or other devices that can achieve the same or similar functions. This embodiment does not limit this. In this embodiment, a computer device is used as an example for explanation.

[0063] It is understood that the container damage detection method proposed in this embodiment is a container damage detection method that fuses lidar and video images. It can be used for digital gate inspection of railway freight yards. This method is based on multimodal perception technology to identify container damage in railway freight yard gate inspection equipment. It aims to improve the accuracy and robustness of container damage identification, reduce the false alarm rate, and thus solve the technical problems in related technologies such as high false alarm rate, poor environmental adaptability and unclear responsibility definition caused by the limitations of a single sensor.

[0064] For example, this embodiment of the method utilizes key technologies such as pixel-level association between laser point clouds and video images, precise contour segmentation and feature extraction of the container based on laser point clouds, a defect detection algorithm based on laser-vision deep fusion, and multipath interference suppression and ghosting elimination in complex environments. This enables stable container damage assessment in complex environments. The method includes: acquiring a high-density 3D point cloud of the container; acquiring images of the container surface; multi-source data fusion processing; achieving pixel-level alignment between the point cloud and the image through an extrinsic parameter calibration algorithm; and simultaneously analyzing geometric deformation and visual texture features using a heterogeneous modal fusion network; and generating a container damage detection report based on the fusion results. This embodiment of the method achieves high-precision quantization and all-weather robust detection through a dual-path parallel detection + post-fusion decision architecture, significantly improving the accuracy and efficiency of container damage identification in complex environments. The specific steps are described below.

[0065] In some embodiments, three-dimensional point cloud data of the container is acquired based on lidar; and images of the container's surface are acquired based on image acquisition equipment.

[0066] Specifically, such as Figure 2 As shown, for the container to be inspected, the LiDAR data acquisition module is activated to scan the container and acquire high-density 3D point cloud data. The video image acquisition module (connected to an industrial camera) is activated to capture images of the container surface.

[0067] In some embodiments, the three-dimensional point cloud data and the container surface image are calibrated and aligned to obtain a transformation relationship, including: performing contour segmentation and feature extraction based on the three-dimensional point cloud data to obtain the container contour and three-dimensional coordinate information of key load-bearing structures of the target container; and performing joint calibration on the three-dimensional point cloud data and the container surface image based on the three-dimensional coordinate information, the lidar-camera joint calibration board, and the extrinsic parameter calibration algorithm to determine the transformation relationship between the lidar coordinate system and the camera pixel coordinate system.

[0068] Specifically, such as Figure 2 As shown, this embodiment employs pixel-level correlation technology between laser point clouds and video images. Using a joint LiDAR-camera calibration board and extrinsic parameter calibration algorithm, the high-density point cloud is projected onto the camera pixel coordinate system using the precise 3D coordinate information of the LiDAR. Sub-centimeter-level ranging accuracy achieves sub-pixel-level precision binding between the point cloud and pixels, providing a high-quality alignment foundation for subsequent deep fusion analysis of the point cloud and image. This embodiment addresses the problem of insufficient accuracy in calibration methods under complex scenarios in related technologies, significantly improving the reliability of multi-source data fusion. This process typically involves two steps: camera intrinsic parameter calibration and LiDAR-camera extrinsic parameter calibration.

[0069] For example, the core projection process and matrix formulas. The overall process includes: projecting a 3D point X in a LiDAR coordinate system... lidar Through a series of matrix transformations, the image is finally projected onto the pixel coordinate system of the camera's imaging plane to obtain the corresponding pixel point p. image The complete projection equation can usually be written as:

[0070] p image =K×[R|t]×X lidar

[0071] Among them, X lidar This represents a 3D point in the lidar coordinate system (usually represented as homogeneous coordinates [x, y, z, 1]). T[R|t] represents the extrinsic parameter matrix, consisting of a rotation matrix R (3×3) and a translation vector t (3×1), used to transform points from the lidar coordinate system to the camera coordinate system; K represents the camera intrinsic parameter matrix, used to project 3D points in the camera coordinate system onto a 2D pixel plane; p image The image pixel coordinates [u, v, 1] represent the projection results. T .

[0072] For example, the camera intrinsic parameter matrix (K) serves as a bridge between the camera coordinate system and the pixel coordinate system, including inherent attributes such as the camera's focal length and principal point position. Its standard form is:

[0073]

[0074] Among them, f x f y c represents the focal length in pixels along the x and y directions, respectively. x c y These are the pixel coordinates of the principal point of the image (usually the image center). The process of obtaining this camera intrinsic parameter matrix is ​​called camera calibration, which can be achieved using the Zhang Zhengyou calibration method by photographing a calibration board with a specific pattern (such as a checkerboard).

[0075] For example, the lidar-camera extrinsic parameter matrix ([R|t]) describes the rigid body transformation relationship between two different sensor coordinate systems. Its function is to transform the coordinates of a point in the lidar coordinate system to the camera coordinate system: X camera =R×X lidar +t; where R is a 3×3 rotation matrix and t is a 3×1 translation vector. This matrix (often combined into a 4x4 homogeneous transformation matrix) is the core parameter to be estimated in the joint calibration. The calibration process requires a specialized calibration board, which is solved by extracting and matching corresponding feature points in the point cloud and the image (e.g., corner points of the calibration board).

[0076] In one example, the process of obtaining subpixel-level precision binding includes: feature extraction and coarse calibration based on a calibration board; fine optimization to minimize reprojection error; and iterative refinement.

[0077] Specifically, feature extraction and coarse calibration based on the calibration board: First, the calibration board needs to be placed within the shared field of view of the camera and LiDAR. The algorithm extracts the corner points of the calibration board from the image (achieving sub-pixel accuracy) and segments the calibration board plane from the point cloud, calculating its corner point or plane parameters in 3D space. Then, using these 3D-2D corresponding point pairs, the initial values ​​of the extrinsic parameter matrix are solved using algorithms such as EPnP (Efficient Perspective n-Point).

[0078] Specifically, the fine-grained optimization for minimizing reprojection error involves obtaining the initial extrinsic parameters, followed by fine-grained optimization. The core idea is to minimize the reprojection error. This means reprojecting the 3D corner points of the calibration board onto the image using the currently estimated extrinsic and intrinsic parameters, and calculating the pixel distance between the projected points and the detected real sub-pixel corner points in the image. Through optimization algorithms such as nonlinear least squares (e.g., the Levenberg-Marquardt algorithm), the extrinsic parameter matrices R and t are iteratively adjusted to minimize the sum of squares of this reprojection error, ensuring that the calibration results achieve optimal alignment accuracy across the entire image.

[0079] Specifically, iterative refinement, in some more advanced calibration methods, involves this process iteratively. The optimized extrinsic parameters can be used to generate more accurate LiDAR images (projecting point cloud intensity or depth information onto the image plane) and perform more refined feature matching with the original camera images, which in turn optimizes the extrinsic parameters, thus achieving a coarse-to-fine calibration.

[0080] In some embodiments, contour segmentation and feature extraction are performed based on the three-dimensional point cloud data to obtain the container body contour and three-dimensional coordinate information of the key load-bearing structure of the target container. This includes: automatically identifying and filtering noise points and false targets from the three-dimensional point cloud data to obtain effective point clouds; performing denoising processing on the effective point clouds based on height filtering, Euclidean clustering, and RANSAC plane fitting algorithms to separate the point cloud clusters of the target container; and extracting the contour of the point cloud clusters of the target container and the three-dimensional coordinate information of the key load-bearing structure.

[0081] In some embodiments, automatically identifying and filtering noise points and false targets from the three-dimensional point cloud data to obtain a valid point cloud includes: analyzing the spatial density distribution, echo intensity continuity, and geometric rationality of the three-dimensional point cloud data to obtain analysis results; identifying and filtering false point clouds and false targets based on machine learning algorithms and the analysis results to obtain denoised clean point cloud data; and using the denoised clean point cloud data as a valid point cloud.

[0082] Specifically, such as Figure 2 As shown, this embodiment employs a precise container contour segmentation and feature extraction technique based on laser point clouds. Utilizing high-density 3D point clouds provided by LiDAR, combined with height filtering, Euclidean clustering, and RANSAC plane fitting algorithms, the target container point cloud is accurately segmented in the complex background of railway stations, extracting the container contour and locating the 3D coordinates of key load-bearing structures. For unstructured environments, a deep learning point cloud segmentation network (e.g., PointNet++) can be used for adaptive segmentation to further improve the accuracy and robustness of contour extraction.

[0083] Specifically, such as Figure 2As shown, this embodiment employs multipath interference suppression and ghost image elimination techniques in complex environments. For backscattered false point clouds generated by lidar under adverse weather conditions such as rain, fog, and dust, the spatial density distribution, echo intensity continuity, and geometric rationality of the point cloud are analyzed, and machine learning algorithms are used to automatically identify and filter noise points and false targets. This technology preserves the true geometric information of the container, significantly improving the purity and detection reliability of point cloud data in complex environments.

[0084] For example, multipath interference suppression is performed on laser point cloud data under complex environments. The spatial density distribution, echo intensity continuity, and geometric rationality of the point cloud are analyzed. Machine learning algorithms are used to identify and filter backscattered false point clouds (noise points) and false targets caused by adverse weather conditions such as rain, fog, and dust. The denoised clean point cloud data is retained. Precise container contour segmentation is performed on the clean point cloud data. Height filtering, Euclidean clustering, and RANSAC plane fitting algorithms are used to separate the point cloud clusters of the target container in the complex background of a railway station. If the environment is unstructured, a deep learning point cloud segmentation network is used for adaptive segmentation to extract the container contour and the 3D coordinates of key load-bearing structures. This yields the segmented 3D point cloud data of the target container, container contour features, 3D coordinates of key load-bearing structures, and the denoised clean point cloud.

[0085] In some embodiments, defect detection is performed on the 3D point cloud data and the box surface image based on the transformation relationship and the heterogeneous modal fusion network to obtain defect detection results. This includes: constructing pixel-level aligned multimodal fusion data based on the transformation relationship and the 3D point cloud data and the box surface image; inputting the multimodal fusion data into the heterogeneous modal fusion network, and simultaneously analyzing geometric deformation features and visual texture features to obtain defect detection results.

[0086] In some embodiments, constructing pixel-aligned multimodal fusion data based on the transformation relationship using the three-dimensional point cloud data and the box surface image includes: using the depth value and echo intensity information of the three-dimensional point cloud data as additional feature channels; and superimposing the additional feature channels onto the RGB channels of the box surface image based on the transformation relationship to construct multimodal fusion data including geometric information and visual texture information.

[0087] Specifically, such as Figure 2As shown, this embodiment employs a laser-vision depth fusion defect detection algorithm. A heterogeneous modal fusion network is designed, superimposing the depth values ​​and echo intensity information of the LiDAR point cloud as additional feature channels onto the RGB image and inputting them into the fusion neural network. By guiding the visual model to focus on analyzing the image features of geometrically abnormal regions, the algorithm accurately distinguishes between real structural damage and appearance interference such as surface stains and paint peeling. In this embodiment, the algorithm significantly reduces the false alarm rate while improving the detection capability of subtle damage.

[0088] Specifically, such as Figure 3 As shown, the defect detection algorithm in this embodiment adopts a strategy of combining dual-path parallel detection with fusion decision-making to give full play to the advantages of the two sensors.

[0089] For example, a LiDAR point cloud analysis algorithm (geometric dimension) is used to accurately detect dents, deformations, and holes. For instance, plane fitting and residual analysis: RANSAC plane fitting is performed on the container side panel area; the distance from each point to the fitted plane (residual) is calculated; if the residual exceeds a threshold (e.g., 5mm) and shows a clustered distribution, it is determined to be a dent; the maximum depth, area, and center coordinates of the dent are output. Normal vector change detection: Point cloud normal vectors are calculated, and regions of abrupt normal vector changes are identified to detect cracks or edge tears. Connectivity analysis: Holes (laser penetration or no-echo areas) are found in the point cloud depth map and determined to be holes. Template matching: ICP (Iterative Closest Point) registration is performed between the real-time point cloud and a standard container CAD model to identify overall distortion or bending.

[0090] For example, video image analysis algorithms (texture dimension) are employed to accurately detect rust, paint peeling, scratches, missing parts, and box number identification. For instance, object detection: the YOLOv8 / Faster R-CNN model is used, with training data including labeled defects of various types (rust, scratches, holes, missing corner pieces, etc.), outputting 2D bounding boxes (BBoxes) and category confidence scores for the defects. Semantic segmentation: the DeepLabV3+ / U-Net model is used to achieve pixel-level segmentation of rusted areas and irregular scratches, calculating the percentage of rusted area. Optical character recognition: the CRNN / PaddleOCR model is used to recognize box numbers and ISO codes, and these are linked to the detection results. Anomaly detection: for rare defects, unsupervised learning is used to reconstruct normal box images; areas with large reconstruction errors are identified as anomalies.

[0091] For example, a multimodal fusion decision algorithm. Spatial alignment: Mapping the 2D bounding boxes detected by vision to 3D point cloud space, or projecting the 3D clusters detected by the point cloud onto the image. Decision logic (rule engine + weighted voting): Scenario A (damage confirmation): The radar detects a depression with a depth > 10mm and the camera detects shadow / texture anomalies at the corresponding location, classifying it as a severe dent and recording the depth value. Scenario B (false positive removal): The radar does not detect geometric deformation, but the camera detects a rust-like patch. Combining the context, if the patch has smooth edges and no depth change, it may be a stain, reducing confidence or marking it as minor surface damage. Scenario C (complementary enhancement): The camera detects a fine crack (texture breakage), but the radar does not capture the depth change due to resolution limitations, retaining the camera detection result and marking it as a surface crack. Scenario D (hole confirmation): The radar detects penetration (no echo) and the camera sees the background perspective, confirming it as a hole.

[0092] For example, model training and optimization. Dataset construction: Collect real data on different lighting conditions, weather, box types, and damage types, and supplement rare defect samples with synthetic data. Transfer learning: Fine-tune the pre-trained model. Continuous learning: After system deployment, manually reviewed and corrected data are automatically added to the training set, and the model is retrained and updated periodically (e.g., weekly) in the background.

[0093] For example, algorithm optimization strategies include: Dynamic ROI cropping: Instead of processing the entire image or all point clouds, quickly locate the approximate ROI of the container using simple thresholding or old frame information. Run complex deep learning models and detailed point cloud analysis only within this area, which can improve processing speed by 3-5 times. Model quantization acceleration: On the edge control computer, convert YOLO and segmentation models to TensorRT Engine to reduce inference latency. Temporal fusion: Containers move continuously; for the same location, the camera may have captured 10 frames. Weighted voting or Kalman filtering can be used to combine the detection results of multiple frames, eliminating false alarms caused by light flicker or water droplets in a single frame. Adaptive thresholding: The threshold for determining the depth of the depression should not be fixed. For old containers (which are inherently uneven), the threshold should be appropriately relaxed; for new containers, the threshold should be tightened, and can be dynamically adjusted through historical data statistics.

[0094] In some embodiments, generating a box damage detection report based on the defect detection results includes: obtaining LiDAR detection results and video image detection results based on the defect detection results; and generating a box damage detection report by combining the LiDAR detection results and video image detection results based on a decision logic that combines a rule engine and weighted voting.

[0095] Specifically, such as Figure 2As shown, this embodiment employs a multimodal fusion decision algorithm. Spatial alignment maps the 2D bounding boxes detected by vision to a 3D point cloud space, or projects the 3D clusters detected by the point cloud onto the image. A decision logic combining a rule engine and weighted voting is used to integrate the detection results from LiDAR and video images to generate a final box damage assessment report. This algorithm effectively overcomes the limitations of single-sensor detection and improves decision accuracy in complex scenarios.

[0096] For example, the core principle of the algorithm and the bidirectional spatial mapping mechanism. Based on the calibrated camera intrinsic parameter matrix and radar-camera extrinsic parameter transformation matrix, a bidirectional mapping channel between 2D image and 3D point cloud is constructed to achieve accurate spatial correlation of the detection results of the two sensors, which is the prerequisite for fusion decision-making.

[0097] Specifically, the forward mapping is as follows: visual 2D detection bounding boxes → 3D point cloud space (image-guided point cloud filtering). The visual detection model outputs 2D bounding boxes with box loss in the image. By calibrating the projection inverse model, the 2D bounding box is mapped to the 3D point cloud space, thus locking the corresponding 3D region of the box. The execution logic includes: traversing all pixels within the bounding box using the pixel-to-point cloud correspondence, inversely solving for the corresponding radar 3D point set, and then filtering to obtain... 2D bounding boxes are associated with 3D point cloud clusters. This filters out background noise from a large area of ​​point clouds, accurately pinpoints the 3D physical regions where visual defects are suspected, and solves the problems of low efficiency and susceptibility to interference from irrelevant point clouds in global laser point cloud searches.

[0098] Specifically, the reverse mapping involves: 3D defect clusters in the point cloud → 2D image pixel space (point cloud-guided image verification). The laser point cloud algorithm uses curvature, normal vector, and elevation change features to cluster 3D point clusters of box-shaped defects. (Dents, protrusions, deformations, damage, etc.) Through a forward projection model, all 3D points are projected onto the image pixel coordinate system to generate... 3D clusters correspond to 2D pixel regions. They endow pure 3D defect points with image texture, color, and edge semantic information, distinguishing real damage from stains, scratches, and reflection interference, and solving problems such as the lack of semantics in laser point clouds and the easy misjudgment of non-structural defects.

[0099] Specifically, the multimodal association matching criterion is as follows: After completing the bidirectional mapping, the spatial overlap threshold is used to determine whether the two sets of detection results are the same defect target: if the IoU (Intersection over Union) of the 2D detection area and the 3D projected pixel area is greater than the preset threshold (default 0.6), it is determined to be a common defect and included in the fusion decision pool; otherwise, it is regarded as an independent interference target and directly filtered.

[0100] For example, the complete multimodal fusion decision-making process (end-to-end five-step process) includes five core stages: independent detection at both ends, cross-modal spatial association, feature fusion encoding, rule-based weighted voting decision-making, result output and report generation. The entire process is processed in parallel pipeline, taking into account both real-time performance and accuracy.

[0101] Specifically, Phase 1 involves independent defect detection using dual sensors (pre-input). Video images and laser point cloud data are received simultaneously, and independent defect detection is performed separately for each, outputting two sets of initial detection results: a visual image detection branch and a laser point cloud detection branch. The visual image detection branch, based on a deep learning detection model, identifies defects in the image such as box breakage, deformation, scratches, and paint peeling, outputting a 2D bounding box, defect type, and visual confidence score. (0~1) Defect texture features. Strong semantic recognition capability, able to distinguish defects from stains and reflections, but unable to acquire 3D size and depth deformation data, and easily affected by lighting. Laser point cloud detection branch: Based on point cloud clustering, normal vector mutation, and depth / height difference calculation, it identifies 3D structural defects in the box, outputting 3D defect point clusters, defect 3D size, deformation, and point cloud confidence score. (0~1). The ranging accuracy is sub-centimeter level, which can accurately quantify deformation and damage depth, but it has no semantic information and cannot distinguish between structural damage and surface stains.

[0102] Specifically, Phase 2 involves bidirectional spatial alignment and matching of objects from the same source. Based on the calibration matrix described earlier, cross-spatial mapping is completed to achieve the association and binding of results from both branches: all visual 2D detection boxes are back-mapped to 3D point clouds, and the corresponding 3D point clusters are extracted; all 3D defect clusters in the point cloud are forward-projected onto the 2D image to generate pixel regions; the IoU between each pair of regions is calculated to complete the matching of objects from the same source, and a 2D-3D paired defect dataset is constructed; isolated detection results without matching (false detections by a single sensor, environmental interference) are filtered out.

[0103] Specifically, Phase 3 involves multi-dimensional defect feature fusion encoding. For matched homologous defects, dual-modal core features are fused to construct a complete defect feature vector, providing data support for decision-making: visual features, including defect texture consistency, edge integrity, color anomaly, and image detection confidence; point cloud features, including defect depth deformation, 3D damaged area, point cloud dispersion, spatial normal vector deviation, and ranging accuracy weight; and alignment features, including dual-modal region matching IoU value, sub-pixel alignment error, and 3D coordinate residual.

[0104] Specifically, Stage 4 employs a fusion decision-making approach combining a rule engine and dynamic weighted voting. This embodiment abandons the single threshold judgment method and adopts a two-layer decision-making mechanism of hard rule filtering and dynamic weighted voting, taking into account both the general applicability of the scenario and the accurate judgment of special defects.

[0105] For example, the first-layer rule engine uses hard constraint filtering: based on the industry standard for container damage detection, rigid judgment rules are set to filter obvious false detection samples and lock in the effective defect candidate set. Among them, the size rule is: if the three-dimensional size of the defect is less than the minimum detection threshold of the system (sub-centimeter level), it is directly judged as no defect; the deformation rule is: if the point cloud detection has no depth change or three-dimensional deformation, and only the image has texture abnormalities, it is judged as stains or reflection interference, not real container damage; the semantic rule is: if the image has no defect semantic features, and only the point cloud has discrete noise, it is judged as point cloud noise interference; the alignment rule is: if the bimodal matching IoU is too low and the spatial association fails, invalid detection results are directly eliminated.

[0106] For example, the second-layer dynamic weighted voting decision: For candidate defects filtered by rules, a scene-based dynamic weight allocation mechanism is adopted, fusing the confidence scores of both sensors to calculate the final comprehensive confidence score and complete the defect classification. The dynamic weight allocation logic adaptively adjusts the weights according to the real-time scene state, avoiding single-sensor defects. For scenes with strong light, backlight, or box reflections, where visual imaging quality deteriorates, the image weight is reduced. Improve the weight of laser point cloud (Point clouds are unaffected by lighting); for weakly textured, clean box scenes, point clouds are prone to pseudo-defects, so the semantic weight of the image should be increased while the weight of the point cloud should be decreased; for standard scenes: the weights of both modalities should be evenly distributed to ensure fusion stability. Comprehensive confidence calculation formula:

[0107]

[0108] in: The weights are output in real time by the scene awareness module and are dynamically adaptive throughout the process. Voting criteria: If... If it is determined to be a deterministic box damage, the defect type, size, and location are marked; if If it is suspected of being damaged, it will be marked after a second verification; if If so, it is judged as no defect or a false positive.

[0109] Specifically, Phase 5 involves summarizing the results and generating a box damage report. After the fusion decision is completed, all valid defect information is integrated to generate a standardized box damage inspection report, including the two-dimensional location of the defect, the precise three-dimensional dimensions, the deformation depth, the defect type, the overall confidence level, the criteria for scene determination, and the raw data from the dual-modal inspection, so as to achieve traceability and quantification of the inspection results.

[0110] In this embodiment, pixel-level correlation technology between laser point clouds and video images achieves high-precision alignment of multi-source data, providing a reliable foundation for deep fusion analysis and solving the problem of insufficient accuracy in traditional calibration methods. The precise contour segmentation and feature extraction technology for containers based on laser point clouds significantly improves the accuracy and robustness of container contour extraction in complex backgrounds, adapting to the needs of different operational scenarios. The laser-vision deep fusion defect detection algorithm accurately distinguishes between real damage and surface interference through simultaneous analysis of geometric and visual features, reducing the false alarm rate to below 5%. Multipath interference suppression and ghost image elimination technology in complex environments effectively filters out false point clouds under adverse weather conditions, retaining true geometric information and improving detection reliability in complex environments. The multimodal fusion decision algorithm, by integrating the detection results of LiDAR and video images, overcomes the limitations of single-sensor detection, significantly improving decision accuracy in complex scenarios. Through a dual-path parallel detection + post-fusion decision architecture, high-precision quantization and all-weather robust detection are achieved, supporting accurate measurement of damage parameters such as indentation depth, providing a scientific basis for damage assessment.

[0111] In this embodiment, the localized processing design avoids uploading massive amounts of raw data, resulting in fast response times, adaptability to high-concurrency scenarios, and improved practicality in real-world operations. The fused results provide dual evidence of geometry and texture, significantly reducing false alarm rates, increasing the reliability of manual review, and minimizing the need for human intervention. It also features adaptive threshold adjustment capabilities, dynamically adjusting the damage assessment threshold based on the condition of the container, further enhancing the flexibility and accuracy of detection.

[0112] This embodiment provides a container damage detection method, including: acquiring three-dimensional point cloud data of the container based on LiDAR; acquiring surface images of the container body based on an image acquisition device; calibrating and aligning the three-dimensional point cloud data and the container body surface images to obtain a transformation relationship; performing fusion defect detection on the three-dimensional point cloud data and the container body surface images according to the transformation relationship and a heterogeneous modal fusion network to obtain a defect detection result; and generating a container damage detection report based on the defect detection result. This embodiment utilizes key technologies such as pixel-level association through calibration and alignment of LiDAR point clouds and video images, and LiDAR-visual deep fusion defect detection to stably perform container damage determination in complex environments. By combining parallel detection of LiDAR point clouds and video images with fusion decision-making, high-precision quantization and all-weather robust detection are achieved, significantly improving the accuracy and efficiency of container damage identification in complex environments.

[0113] Reference Figure 4 , Figure 4 This is a structural block diagram of an embodiment of the container damage detection system of the present invention. Figure 4 As shown, the container damage detection system includes:

[0114] The laser point cloud acquisition module 10 is used to acquire three-dimensional point cloud data of the container based on the lidar.

[0115] The video image acquisition module 20 is used to acquire images of the container surface based on the image acquisition device;

[0116] Alignment module 30 is used to calibrate and align the three-dimensional point cloud data and the box surface image to obtain the transformation relationship;

[0117] The fusion defect detection module 40 is used to perform fusion defect detection on the three-dimensional point cloud data and the box surface image according to the transformation relationship and the heterogeneous modal fusion network, and obtain the defect detection result.

[0118] The multimodal fusion decision module 50 is used to generate a box damage detection report based on the defect detection results.

[0119] Specifically, the container damage detection system provided in this embodiment includes: a laser point cloud acquisition module 10 (LiDAR data acquisition module), configured to acquire a high-density three-dimensional point cloud of the container; a video image acquisition module 20 (video image acquisition module), connected to an industrial camera, used to acquire images of the container surface; a multi-source data fusion processing module (including an alignment module 30 and a fusion defect detection module 40), connected to the LiDAR and video image acquisition modules, which achieves pixel-level alignment between the point cloud and the image through an external parameter calibration algorithm, and uses a heterogeneous modal fusion network to simultaneously analyze geometric deformation and visual texture features; and a decision output module (multi-modal fusion decision module 50), configured to generate a container damage detection report based on the fusion results.

[0120] For example, the entire system achieves high-precision quantization and all-weather robust detection through a dual-path parallel detection + post-fusion decision architecture, which significantly improves the accuracy and efficiency of container damage identification in complex environments.

[0121] The container damage detection system provided in this embodiment uses key technologies such as pixel-level correlation of laser point cloud and video image calibration and alignment, and laser-vision deep fusion defect detection to stably determine container damage in complex environments. By combining parallel detection of laser point cloud and video image and then fusing decision-making, it achieves high-precision quantification and all-weather robust detection, significantly improving the accuracy and efficiency of container damage identification in complex environments.

[0122] It should be noted that technical details not described in detail in this embodiment of the container damage detection system can be found in any embodiment of the present invention applied to the container damage detection method as described above, and will not be repeated here.

[0123] Based on the same inventive concept, embodiments of the present invention also provide an electronic device. Figure 5 This is a structural block diagram of an electronic device provided in an embodiment of the present invention. Figure 5 As shown, an embodiment of the present invention provides an electronic device including: one or more processors 101, a memory 102, and one or more I / O interfaces 103. The memory 102 stores one or more programs, which, when executed by the one or more processors, cause the one or more processors to implement any of the container damage detection methods described in the above embodiments; the one or more I / O interfaces 103 are connected between the processors and the memory, configured to enable information interaction between the processors and the memory.

[0124] The processor 101 is a device with data processing capabilities, including but not limited to a central processing unit (CPU); the memory 102 is a device with data storage capabilities, including but not limited to random access memory (RAM, more specifically SDRAM, DDR, etc.), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), and flash memory (FLASH); the I / O interface (read / write interface) 103 is connected between the processor 101 and the memory 102, and can realize information interaction between the processor 101 and the memory 102, including but not limited to a data bus (Bus).

[0125] In some embodiments, the processor 101, memory 102, and I / O interface 103 are interconnected via bus 104, and thus connected to other components of the computing device.

[0126] In some embodiments, the one or more processors 101 include a field-programmable gate array.

[0127] This invention also provides a computer-readable medium. The computer-readable medium stores a computer program, which, when executed by a processor, implements the steps of any of the container damage detection methods described in the above embodiments. The computer-readable storage medium can be volatile or non-volatile.

[0128] This invention also provides a computer program product, including computer-readable code, or a non-volatile computer-readable storage medium carrying computer-readable code. When the computer-readable code is run in the processor of an electronic device, the processor in the electronic device executes the above-described container damage detection method.

[0129] Those skilled in the art will understand that all or some of the steps, systems, and apparatuses disclosed above, and their functional modules / units, can be implemented as software, firmware, hardware, or suitable combinations thereof. In hardware implementations, the division between functional modules / units mentioned above does not necessarily correspond to the division of physical components; for example, a physical component may have multiple functions, or a function or step may be performed collaboratively by several physical components. Some or all physical components may be implemented as software executed by a processor, such as a central processing unit, digital signal processor, or microprocessor, or as hardware, or as an integrated circuit, such as an application-specific integrated circuit (ASIC). Such software can be distributed on a computer-readable storage medium, which may include computer storage media (or non-transitory media) and communication media (or transient media).

[0130] As is known to those skilled in the art, the term computer storage medium includes volatile and non-volatile, removable and non-removable media implemented in any method or technology for storing information, such as computer-readable program instructions, data structures, program modules, or other data. Computer storage media includes, but is not limited to, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM), static random access memory (SRAM), flash memory or other memory technologies, portable compact disc read-only memory (CD-ROM), digital versatile disc (DVD) or other optical disc storage, magnetic cartridges, magnetic tape, disk storage or other magnetic storage devices, or any other medium that can be used to store desired information and is accessible to a computer. Furthermore, it is known to those skilled in the art that communication media typically contain computer-readable program instructions, data structures, program modules, or other data in modulated data signals such as carrier waves or other transmission mechanisms, and may include any information delivery medium.

[0131] The computer-readable program instructions described herein can be downloaded from computer-readable storage media to various computing / processing devices, or downloaded via a network, such as the Internet, local area network, wide area network, and / or wireless network, to an external computer or external storage device. The network may include copper transmission cables, fiber optic transmission, wireless transmission, routers, firewalls, switches, gateway computers, and / or edge servers. A network adapter card or network interface in each computing / processing device receives the computer-readable program instructions from the network and forwards them to the computer-readable storage media in the respective computing / processing device.

[0132] The computer program instructions used to perform the operations of this invention may be assembly instructions, instruction set architecture (ISA) instructions, machine instructions, machine-dependent instructions, microcode, firmware instructions, state setting data, or source code or object code written in any combination of one or more programming languages, including object-oriented programming languages ​​such as Smalltalk, C++, etc., and conventional procedural programming languages ​​such as the "C" language or similar programming languages. The computer-readable program instructions may be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving a remote computer, the remote computer may be connected to the user's computer via any type of network—including a local area network (LAN) or a wide area network (WAN)—or may be connected to an external computer (e.g., via the Internet using an Internet service provider). In some embodiments, electronic circuitry, such as programmable logic circuitry, field-programmable gate arrays (FPGAs), or programmable logic arrays (PLAs), is personalized by utilizing state information from the computer-readable program instructions. This electronic circuitry can execute the computer-readable program instructions to implement various aspects of the invention.

[0133] The computer program product described herein can be implemented specifically through hardware, software, or a combination thereof. In one alternative embodiment, the computer program product is specifically embodied in a computer storage medium; in another alternative embodiment, the computer program product is specifically embodied in a software product, such as a software development kit (SDK), etc.

[0134] Various aspects of the present invention are described herein with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It should be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer-readable program instructions.

[0135] These computer-readable program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing apparatus to produce a machine such that, when executed by the processor of the computer or other programmable data processing apparatus, they create means for implementing the functions / actions specified in one or more blocks of the flowchart and / or block diagram. These computer-readable program instructions can also be stored in a computer-readable storage medium that causes a computer, programmable data processing apparatus, and / or other device to operate in a particular manner; thus, the computer-readable medium storing the instructions comprises an article of manufacture that includes instructions for implementing aspects of the functions / actions specified in one or more blocks of the flowchart and / or block diagram.

[0136] Computer-readable program instructions may also be loaded onto a computer, other programmable data processing apparatus, or other device to cause a series of operational steps to be performed on the computer, other programmable data processing apparatus, or other device to produce a computer-implemented process, thereby causing the instructions executed on the computer, other programmable data processing apparatus, or other device to perform the functions / actions specified in one or more boxes of a flowchart and / or block diagram.

[0137] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of the present invention. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of an instruction, which contains one or more executable instructions for implementing a specified logical function. In some alternative implementations, the functions marked in the blocks may occur in a different order than those shown in the drawings. For example, two consecutive blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, may be implemented using a dedicated hardware-based system that performs the specified function or action, or using a combination of dedicated hardware and computer instructions.

[0138] Example embodiments have been disclosed herein, and while specific terminology has been used, it is for illustrative purposes only and should be construed as such, and is not intended to be limiting. In some instances, it will be apparent to those skilled in the art that features, characteristics, and / or elements described in conjunction with particular embodiments may be used alone, or in combination with features, characteristics, and / or elements described in conjunction with other embodiments, unless otherwise expressly indicated. Therefore, those skilled in the art will understand that various changes in form and detail may be made without departing from the scope of the invention as set forth in the appended claims.

Claims

1. A method for detecting damage to containers, characterized in that, include: 3D point cloud data of containers are acquired using LiDAR; Images of the container's surface are acquired using image acquisition equipment; The 3D point cloud data and the box surface image are calibrated and aligned to obtain the transformation relationship; Based on the transformation relationship and the heterogeneous modal fusion network, the three-dimensional point cloud data and the box surface image are fused to detect defects, and the defect detection results are obtained. A box damage detection report is generated based on the defect detection results.

2. The method as described in claim 1, characterized in that, The calibration and alignment of the 3D point cloud data and the box surface image to obtain the transformation relationship includes: Based on the three-dimensional point cloud data, contour segmentation and feature extraction are performed to obtain the three-dimensional coordinate information of the target container's body contour and key load-bearing structures. Based on the three-dimensional coordinate information, the lidar-camera joint calibration board, and the extrinsic parameter calibration algorithm, the three-dimensional point cloud data and the box surface image are jointly calibrated to determine the transformation relationship between the lidar coordinate system and the camera pixel coordinate system.

3. The method as described in claim 2, characterized in that, The step of performing contour segmentation and feature extraction based on the three-dimensional point cloud data to obtain the container body contour and three-dimensional coordinate information of key load-bearing structures includes: The three-dimensional point cloud data is automatically identified and noise points and false targets are filtered out to obtain an effective point cloud; The effective point cloud is denoised using height filtering, Euclidean clustering, and RANSAC plane fitting algorithms to separate the point cloud clusters of the target container. Extract the outline of the point cloud cluster of the target container and the three-dimensional coordinate information of the key load-bearing structure.

4. The method as described in claim 3, characterized in that, The automatic identification and filtering of noise points and false targets from the 3D point cloud data to obtain a valid point cloud includes: The spatial density distribution, echo intensity continuity, and geometric rationality of the three-dimensional point cloud data are analyzed to obtain the analysis results. Based on machine learning algorithms and the analysis results, false point clouds and false targets are identified and filtered to obtain clean point cloud data after noise reduction. The denoised and clean point cloud data is used as the valid point cloud.

5. The method as described in claim 1, characterized in that, The step of fusing the 3D point cloud data and the box surface image based on the transformation relationship and the heterogeneous modal fusion network to detect defects and obtain defect detection results includes: Based on the transformation relationship, pixel-level aligned multimodal fusion data is constructed from the 3D point cloud data and the box surface image; The multimodal fusion data is input into a heterogeneous modal fusion network, and geometric deformation features and visual texture features are analyzed simultaneously to obtain defect detection results.

6. The method as described in claim 5, characterized in that, The step of constructing pixel-level aligned multimodal fusion data based on the transformation relationship and the 3D point cloud data and the box surface image includes: The depth value and echo intensity information of the three-dimensional point cloud data are used as additional feature channels; Based on the transformation relationship, the additional feature channels are superimposed onto the RGB channels of the box surface image to construct multimodal fusion data including geometric information and visual texture information.

7. The method according to any one of claims 1 to 6, characterized in that, The step of generating a box damage detection report based on the defect detection results includes: Based on the defect detection results, the lidar detection results and video image detection results are obtained; Based on the decision-making logic that combines rule engine and weighted voting, a box damage detection report is generated by combining the lidar detection results and video image detection results.

8. A container damage detection system, characterized in that, include: The laser point cloud acquisition module is used to acquire three-dimensional point cloud data of containers based on lidar. The video image acquisition module is used to acquire images of the container's surface based on the image acquisition device; The alignment module is used to calibrate and align the three-dimensional point cloud data and the box surface image to obtain the transformation relationship. The fusion defect detection module is used to perform fusion defect detection on the three-dimensional point cloud data and the box surface image according to the transformation relationship and the heterogeneous modal fusion network, and obtain the defect detection result. The multimodal fusion decision module is used to generate a box damage detection report based on the defect detection results.

9. An electronic device, characterized in that, include: One or more processors; Memory, used to store one or more programs; When the one or more programs are executed by the one or more processors, the one or more processors implement the method as described in any one of claims 1 to 7.

10. A computer-readable medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the steps of the method as described in any one of claims 1 to 7.

Images