System for inspecting shipping container and method thereof

Abstract

A system for inspecting a shipping container and a method are disclosed. The system comprises a plurality of cameras installed at a plurality of entry / exit gates, wherein the plurality of cameras captures one or more images of the container passing through the plurality of the entry gates. The system further comprises an inspection device operatively coupled to the plurality of cameras to receive captured images and process the one or more images using a machine learning model, wherein the machine learning model is configured to determine: a container number recognition, a marking code sequence, presence of a seal on the container, presence of a hazardous sign on the container, and presence of a damage on the container.

Description

SYSTEM FOR INSPECTING SHIPPING CONTAINER AND METHOD THEREOF

[0001] The present disclosure generally relates to logistic management and more particularly to system and method for inspecting a shipping container.

[0002] Logistic management systems deal with transportation of shipping containers from one point to another. Transportation of the shipping containers from one point to another involves passing of the container from one or more entry / exit gates of the ports. Usually, the transportation of the containers is performed by third parties where the shipping containers are passed on by one party (e.g., a consignor) to another party (e.g., a consignee). The consignee, while taking possession of the shipping container needs to ensure that the shipping container is in good shape and is without any damage.

[0003] The shipping container are stopped at inspection gates at the facility of the consignee for various checks such as container number, container marking codes, damage, presence of seal and hazardous sign. These inspections are often done manually and therefore, they are time-consuming. Existing solutions for determination of the various parameters related to inspections have complex architectures and are not cost effective.

[0004] PTL 1: Japanese Unexamined Patent Application Publication No. 2023-076323

[0005] Hence, there is a need in the art which can automate the inspection of the shipping container while the shipping container is passing through the various entry / exit gates at the consignee facility such that the need for stopping the shipping container at the entry / exit points can be avoided.

[0006] One of the objects to be accomplished by example embodiments disclosed herein is to at least partially address one or more of the above needs and / or issues.

[0007] The following presents a simplified summary of the subject matter in order to provide a basic understanding of some of the aspects of subject matter embodiments. This summary is not an extensive overview of the subject matter. It is not intended to identify key / critical elements of the embodiments or to delineate the scope of the subject matter. Its sole purpose to present some concepts of the subject matter in a simplified form as a prelude to the more detailed description that is presented later.

[0008] The implementations of the present disclosure provide a system for inspecting a shipping container. The system comprises a plurality of cameras installed at a plurality of entry gates, wherein the plurality of cameras captures one or more images of the container passing through the plurality of the entry gates, an inspection device operatively coupled to the plurality of cameras to receive captured images and process the one or more images using a machine learning model, wherein the machine learning model is configured to determine: a container number recognition, a marking code sequence, presence of a seal on the container, presence of a hazardous sign on the container, and presence of a damage on the container.

[0009] In another embodiment, the implementation of the present disclosure provides a method for inspecting a shipping container. The method comprises installing a plurality of cameras at a plurality of entry gates, wherein the plurality of cameras captures one or more images of the container passing through the plurality of the entry gates, receiving, by an inspection device, the plurality of captured device from the plurality of cameras and processing the one or more images using a machine learning model and determining, using the machine learning model, a container number recognition, a marking code sequence, presence of a seal on the container, presence of a hazardous sign on the container, and presence of a damage on the container.

[0010] The details of one or more examples are set forth in the accompanying drawings and the description below. Other features, objects, and advantages will be apparent from the description, drawings, and claims.

[0011] According to the aspects described above, it is possible to provide a system and a method that contributes to at least partially address one or more of the above needs and / or issues.

[0012] The foregoing and further objects, features, and advantages of the present subject matter will become apparent from the following description of exemplary embodiments with reference to the accompanying drawings, wherein like numerals are used to represent like elements.

[0013] It is to be noted, however, that the appended drawings illustrate only typical embodiments of the present subject matter, and are, therefore, not to be considered for limiting of its scope, for the subject matter may admit to other equally effective embodiments.

[0014] For a better understanding of the present disclosure, reference is made to the following description of an exemplary embodiment thereof, considered in conjunction with the accompanying drawings, in which:

[0015] Fig. 1 illustrates an environment showing a shipping container passing through entry / exit gates in accordance with an embodiment of the present subject matter.Fig. 2 illustrates a detailed aspect of a system in accordance with an embodiment of the present subject matter.Fig. 3 illustrates a process flow performed by the inspection device with an embodiment of the present subject matter.Fig. 4 illustrates a process flow performed by the container number recognition model in accordance with an embodiment of the present subject matter.Fig. 5 illustrates an example embodiment for determining marking code sequence in accordance with an embodiment of the present subject matter.Fig. 6 discloses an example embodiment showing e-seals present on the shipping container in accordance with an embodiment of the present subject matter.Fig. 7 discloses different stages for determining a hazardous sign in accordance with an embodiment of the present subject matter.Fig. 8 discloses an example embodiment disclosing different categories of damage in accordance with an embodiment of the present subject matter.Fig. 9 illustrates a flowchart of a method for inspecting a shipping container in accordance with an embodiment of the present subject matter.Fig. 10 illustrates a block diagram of the inspection device in accordance with an embodiment of the present subject matter.

[0016] Although specific features of various embodiments may be shown in some drawings and not in others, this is for convenience only. Any feature of any drawing may be referenced and / or claimed in combination with any feature of any other drawing.

[0017] The embodiments of the present subject matter are described in detail with reference to the accompanying drawings. However, the present subject matter is not limited to these embodiments which are only provided to explain more clearly the present subject matter to the ordinarily skilled in the art of the present disclosure. In the accompanying drawings, like reference numerals are used to indicate like components.

[0018] The present disclosure is not limited in its application to the details of construction and the arrangement of components set forth in the following description or illustrated in the drawings. The present disclosure is capable of other embodiments and of being practiced or of being carried out in various ways. Also, the phraseology and terminology used herein is for the purpose of description and should not be regarded as limiting. The use of "including," "comprising," or "having," "containing," "involving," and variations thereof herein, is meant to encompass the items listed thereafter and equivalents thereof as well as additional items.

[0019] Various aspects of the proposed system and method are described fully hereinafter with reference to the accompanying drawings. The same reference numbers in different drawings may identify the same or similar elements. The teachings disclosed may, however, be embodied in many different models with variations and should not be construed as limited to any specific structure or function presented throughout this disclosure. Rather, these aspects are provided so that this disclosure will be thorough and complete, and will fully convey the scope of the disclosure to those skilled in the art. It should be understood that any aspect disclosed herein may be embodied by one or more elements of a claim, and also that the following detailed description does not limit the claims.

[0020] Also, all logical units described and depicted in the figures include the software and / or hardware components required for the unit to function. Further, each unit may comprise within itself one or more components which are implicitly understood. These components may be operatively coupled to each other and be configured to communicate with each other to perform the function of the said unit.

[0021] Fig. 1 illustrates an environment showing a shipping container 104 passing through entry / exit gates 106 in accordance with an embodiment of the present subject matter. The entry / exit gates are shown in the form of boom barriers. The present disclosure ensures smooth operation of the shipping container when the shipping container passes through the entry / exit gates. The smooth operation includes determining various parameters related to inspection of the shipping container such as determination of container recognition number, marking code sequence, presence of seal, hazard sign and damage detection at the entry / exit gates in real-time. Further, although a 20 feet container is shown, the shipping container 104 can be of any size / capacity.

[0022] The determination of various parameters related to inspection of the shipping container 104 helps the shipping container pass through the entry / exit gate effortlessly without the manual inspection by an operator. The various features of the present disclosure enables automated operation of the entry / exit gates such that there is no requirement of manual operation of the shipping container by the operator. With the help of a plurality of cameras and the image processing techniques, smooth operation for the shipping container can be performed.

[0023] The plurality of cameras are placed at various locations across the passing lane of the shipping container 104. The plurality of cameras are numbered as 1-11 where the cameras numbered as 1, 2, 7, 11, 4, 8, 5 and 9 are configured to take image of the shipping container 104 across the side of the shipping container while the cameras numbered as 10 is configured to take the image of the shipping container from the backside (i.e., rear image) of the shipping container 104. Similarly, there are cameras configured to capture images of the shipping container from the front and the top of the shipping container. All the images from the various sides of the shipping container are processed based on the image processing techniques and the machine learning techniques to determine various parameters (container recognition number, marking code sequence, seal presence, hazard sign detection and damage detection) related to the inspection of the shipping container 104.

[0024] Fig. 2 illustrates a detailed aspect of a system 200 in accordance with an embodiment of the present subject matter. The system 200 comprises the plurality of cameras 202 installed at the plurality of entry / exit gate 204 and an inspection device 206. The inspection device 206 is operatively coupled to the plurality of cameras 202 to receive the captured images and process the captured images using a machine learning model. The machine learning model is configured to determine: a container number recognition, a marking code sequence, presence of a seal on the container, presence of a hazardous sign on the container, and presence of a damage on the container.

[0025] The plurality of cameras 202 captures the images of the shipping container 104 from different sides. For example, the plurality of cameras 202 captures the images of the shipping container from top, sides, back and front. Capturing of images of different portions of the shipping container 104 ensures that determination of various parameters related to the inspection of the shipping container 104 can be performed more accurately and efficiently. The capturing of images may occur in real time as the shipping container passes through the entry / exit gates 204 with cameras. In one embodiment, the different portions of the shipping container includes images of the container's rear, front, sides and / or top of the shipping container.

[0026] As explained above, the images captured by the cameras 202 are transmitted to the inspection device 206 for further processing. The inspection device 206 processes the images of the different portions of the shipping container to identify different parameters related to the inspection of the shipping container 104. For example, the inspection device 206 may process the images of the shipping container 104 captured from back side to determine presence of the seal on the back doors of the shipping container 104. Similarly, the images of the back side of the shipping container may also be processed to determine hazard sign, container code sequence and / or container recognition number. Although, the images of the rear side is mentioned here, the position of the container number, code sequence can be present at the sides of the shipping container as well.

[0027] Fig. 3 illustrates a process flow 300 performed by the inspection device 206 with an embodiment of the present subject matter. The inspection device 206 contains a machine leaning model 302 for determining parameters related to the damage of the shipping container 104. The machine learning model 302 contains various other sub-models such as the machine learning model 302 can further include a machine learning damage detection model 304 for determining damage of the shipping container, a machine learning container number recognition model 306 for determining container number of the shipping container, a machine learning ISO model 308 for determining marking code sequence present on the shipping container, a machine learning e-seal model 310 for determining presence of seal on the shipping container and a machine learning hazard sign detection model 312 for determining hazard sign present on the shipping container.

[0028] Each of the machine learning models 302-312 can use a custom convolution neural network (CNN) model. In one embodiment, the CNN model is a ResNet50 (residual network) that is built using TensorFlow. The images of the different portions of the shipping container can be fed to the convolution layers of the CNN model for machine learning. The layers of the CNN networks are assigned weights for learning according to the layers input.

[0029] The machine learning model 302 receives as input the plurality of images of the shipping container labelled with type and area. In other words, the type of shipping container may include the type / capacity of the shipping container and the area of the shipping container may include, for example, top, rear, sides, underside, etc. In one embodiment, the machine learning 302 is trained using these images received from the plurality of cameras 202.

[0030] In one embodiment, the machine learning model 302 may also receive images of the shipping container taken manually from yard or taken from other places. For example, in addition to the entry / exit gates, the plurality of cameras may also be installed in the lane from where the shipping container may pass. These additional cameras may be configured to capture images of the shipping container from different sides. In one embodiment, the additional cameras may also help ensure an added level of accuracy in training of the machine learning model when images from different sides are captured. In one embodiment, the plurality of cameras 202 also captures live camera feed from the entry / exit gates 204 as the shipping container 104 passes through them. The live feeds of the cameras can also help to ensure attention to details while determining parameters relating to the shipping container.

[0031] The machine learning model 302 contains various other sub-models, as discussed above. Training of machine learning model 302 includes training of the various other machine learning sub-models such as the machine learning damage detection model 304, machine learning container number recognition model 306, machine learning ISO model 308, machine learning e-seal model 310 and machine learning hazard sign detection model 312. Each of these machine learning models 304-312 may use one or more images of different portions of the shipping container for training. For example, the machine learning container number recognition model 306 uses images of at least one side of the shipping container for determining the container number of the shipping container. The objective of the machine learning container number recognition model 306 is to automate the reading and recognition of the container number.

[0032] Similarly, the machine learning ISO model 308 is trained to determine length, height and type of container in an easy-to-read sequence. The marking code sequence generally appear right under the container number. The machine learning ISO model 308 can be trained using the images captured of the shipping container from the rear end, i.e., back side. In one embodiment, the container code sequence can be present on any portion of the shipping container and different portions of the images can be used.

[0033] Also, the machine learning e-seal model 310 may be trained by using images of the doors of the container. The doors of the containers are generally present at the rear side of the shipping container. Hence, the machine learning e-seal model 310 can be used to determine whether the seal present on the shipping container is intact and record the position of the missing seals, if present. The machine learning e-seal model 310 can be trained in a way such that the seals can be linked with the container number.

[0034] Similarly, the machine learning hazardous sign detection model 312 is trained using the images from all the sides of the shipping container. The images of the top and the bottom of the shipping container can be eliminated since the hazardous sign is not present on the top of the shipping container. The machine learning hazard sign detection model 312 can be trained to detect and then classify the hazardous symbol into various categories. For example, the hazard sign can be categorized based on the degree of severity of the hazard.

[0035] Next the machine learning damage detection model 304 can be trained using the images from all the sides of the shipping container. The machine learning damage detection model 304 is used to determine whether the container is damaged and the type of damage present on the shipping container. The type of damage can include tear, rust, hole, scratch, dent. Apart from this these damages can be further divided based on the severity and location of the damage, e.g., high priority damage or low priority damage. The machine learning damage detection model 304 can be trained to perform a different set of actions for each of the different types of the damage as will be explained below. The machine learning damage detection model 304 can be trained based on supervised learning techniques and un-supervised learning techniques. Under supervised learning techniques, the images are labelled by the machine learning damage detection model 304 and the damage determination is performed from the labelled images. Under unsupervised learning techniques, the determination of the damage on the shipping container is performed without labelling the images.

[0036] Fig. 4 illustrates a process flow performed by the container number recognition (CNR) model in accordance with an embodiment of the present subject matter. The live feed from the plurality of cameras installed at the entry / exit gates are provided to the machine learning container number recognition model 306. The machine learning container number recognition model 306 comprises of a pre-processing stage 402 where it is checked whether the image captured by the plurality of cameras contains the shipping container 104 in it. This pre-filtering may be performed using custom Tensorflow model. If the captured image contains the image of the container (YES), identification stage 404 is performed where boundaries of the container number present on the container is identified. This identification stage 404 may be performed using TensorFlow MobileNet-SSD model. The identification stage 404 is also used to detect the presence and location of the container number on different sides of a container. One single model may be trained to detect the numbers on the front and the back of the container.

[0037] Next stage is the recognition stage 406, where the characters of the container number are recognized using OCR TensorFlow AttentionOCR model This recognition stage 406 is used to recognize the unique characters of the container number. Also, the model for recognizing the container number may use an available NN architecture called AttentionOCR, that was built using TensorFlow. If the number of characters that are recognized are more than a threshold, i.e., if the confidence is higher, inference is stored in memory. Further, a checksum stage 408 is performed to verify the check digits of the recognized container number. It is checked whether the value of the check digit of the recognized container number match the checksum in maximum frames of the video feed / image captured by the plurality of cameras.

[0038] If there is a match, it is determined whether the checksum is matched and report number identification success. However, if the check digits do not match, perform character / number replacement (step 410). Checksum is used to check errors or damage to data when the data is transmitted or stored. Matching of the check digits involves comparing check digits with the checksum performed on the digits stored in the memory. In other words, the container number recognition is determined using convolution neural network (CNN) by matching the container number recognition with a checksum and performing selection of highest occurring characters(with above threshold probability) in the same position in different frames.

[0039] Due to the critical nature of the determination process for the container number, high availability is a crucial requirement. But as there are multiple gates or lanes at the entrance of the ports, even if a one unit is down, it only results in closure of a single lane. The rest of the lanes can continue processing and can handle the excess load for a brief period.

[0040] Due to the relatively low cost of these devices, replacement units can be used to replace faulty units. An application will be designed such that a reboot of inference hardware will reinitialize the application and prepare it to process the video feeds. As the application does not store any transactional data, there is no fear of data corruption or data loss. Further, due to its modularized design and containerized deployment architecture, the application has the ability to scale linearly. The application can be scaled vertically or horizontally based on the specific requirements and constraints of each deployment.

[0041] Fig. 5 illustrates an example embodiment for determining marking code sequence in accordance with an embodiment of the present subject matter. As explained above, the machine learning ISO model 308 determines the making code sequence present on the shipping container. The container marking codes depicts the length, height, and type of container in an easy-to-read sequence. This sequence is composed of four letters or digits that commonly appear right under the container identification sequence. In one of the examples shown in the figure, interpretation of the marking code is shown. The first character is related to the length of the container. The second character is relative to its height while the remaining two elements of the sequence identify the container type and characteristics.

[0042] For determining the marking sequence, the rear (doors), left and right side images of the shipping container are fed into the machine learning ISO model 308. The machine learning ISO model 308 may work in different stages. In the first stage, the machine learning ISO model 308 may detect and localize the marking code sequence from container image using Object detection algorithm like ssd-mobilenet v2. The model also captures the area of marking code and passes this cropped image to the second stage for optical character recognition (OCR) task. The second stage has an OCR algorithm which reads the marking code sequence.

[0043] Fig. 6 discloses an example embodiment showing e-seals present on the shipping container in accordance with an embodiment of the present subject matter. The E-seals are generally present on the rear, i.e., doors of the shipping containers. E-seals are container seals on doors of the shipping containers that are put on international shipping containers once a shipment is loaded. This seal is meant to stay on through to the container's final destination and is removed by the consignee. It is important for the consignee to check the E-seal present on the doors before taking the delivery of the shipping container. The presence of E-seals on the shipping containers can prevent tampering of the goods present in the shipping container.

[0044] The objective of the machine learning e-seal model 310 is to detect and record the presence of seals on the container, record the position of the missing seals, store the image of each of the seals and link them to the container number, raise an exception if the minimum criteria, to secure the container, is not met. The criteria for such a determination is: "At least one bottle seal should be present on right side door". The machine learning E-seal model can raise an alert if the latches are not visible (reverse position or hidden).

[0045] E-seals are very small and difficult to detect, so to improve accuracy detection the machine learning model can work in two stages. In the first stage, the model can identify the latch area from the rear image of the shipping container which is fed into this model. The identification of the latch area includes identifying where the seal would be in the image and when the seal is identified, performing image processing techniques to zoom- in the image and pass the zoomed image to the second stage for e-seal detection. The second stage can identify the presence of the E-seal from the zoomed-in images based on the trained model. In both the stages MobileNetSSD models may be used, but both the models are with different configurations and are customized accordingly.

[0046] Fig. 7 discloses different stages for determining a hazardous sign in accordance with an embodiment of the present subject matter. The presence of the 'Hazardous Cargo' sign on the shipping container indicates that the shipping container requires special attention in storage & handling. The container handling facility may want to ensure that each and every container which has cargo that requires special care needs to have hazardous sign present on all visible sides of the container. The objective of the machine learning hazardous sign detection model is to detect and verify the presence of hazardous signs on the shipping container.

[0047] As shown in the figure, the machine learning model determines the hazardous sign in two stages. The first stage 702 is hazardous sign detection stage, and the second stage 704 is the hazardous sign classification stage. The first stage 702 is used to detect the presence and location of the hazard symbol on different sides of a container or tanker. The presence and location of the hazard symbol is determined from the side and rear images of the shipping container which are fed into the machine learning hazardous sign determination model. The first stage may use MobileNet v1-SSD as the model architecture. In one embodiment, the determination of the hazardous sign involves labelling a plain rectangular box over each hazard symbol in the image fed into the model and passing the labelled image to the second stage 704.

[0048] The second stage 704 is classification of the hazardous sign determined in the first stage. The second stage 704 is used to classify the detected hazard symbol in the previous stage. The second stage704 uses the convolutional Neural network (CNN) architecture called ResNet50, that was built using TensorFlow. The model may be trained from scratch. The second stage extracts the hazard symbols from the images received from the first stage 702 and manually segregate them into different folders.

[0049] The machine learning damage detection model 304 will now be discussed. Shipping containers are handled by multiple parties during their transportation. Therefore, damages to a shipping container need to be tracked at each stage, so that any damage can be attributed to the correct party and compensation claimed from them. Most of the ports have mandated that every container entering its port needs to be visually assessed for external physical damage. If detected, the damage needs to be recorded. This data will be used for multiple purposes like proactively informing the shipping company, absolving of any blame for the damage, proof for any insurance claims etc. It also helps in ensuring that damaged containers are handled cautiously to avoid accidents.

[0050] Also, the damages present in the shipping containers can be categorized into 5 main types named tear, rust, hole, scratch, dent. Apart from this these damages can be further divided based on the severity and location of the damage. Further, based on customers' inputs, some damage patterns have been identified that need extra attention. These inputs are fed at the time of training the machine learning damage detection model. For example, damages that require extra attention can include damage to crane latching points present at the 4 corners on the top side of the container.

[0051] The machine learning damage detection model is configured to classify (yes / no) if any externally visible physical damage is present on the shipping container and raise an alert. The damages are determined from the images of the shipping container fed to the model. The model also identifies externally visible physical damage to the containers and classify them to a damage type. Identify if the damage matches any of the 'high priority damage' category. The model may also generate images marking the position of the damage.

[0052] Before transferring the container from one entity to another entity (usually at entry or exit place), the container handling facility may want to ensure that the container is not damaged and if it's damaged does it happened in their yard or did it previously happen. This check is to ensure the liability transfer and claim process. The objective of the model is to detect the damage from all the outside visible side of the container.

[0053] The determination of the damage on the shipping container by the machine learning damage detection model is performed into two models- the supervised model and the unsupervised model. Both the supervised model and the unsupervised model run in parallel for determination of the damage on the shipping container. The supervised techniques includes labelling the one or more images of the shipping container and determining the type of damage based on the labelling of the one or more images and the unsupervised techniques includes comparing one or more captured images of the shipping container with one or more images of the shipping container without damage.

[0054] Under the supervised model, a single stage 'CenterNet Hourglass' model is used. In this model, the labels are provided to each type of damage and the model is trained so that it learns the actual damage type and gives the result from one of the pre-defined labels lists. This model runs on the edge device with 1 frame per second speed. For determination of the damage on multiple containers passing from the multiple lanes simultaneously, an on-premise server is used which can run process multiple images together using the supervised model for determination of the damage on the shipping container. This can be with or without GPU server and can handle multiple lanes.

[0055] The un-Supervised model helps determine damages which cannot be determined easily, i.e., rare type of damages which can occur on the shipping container. These damages are very subjective in nature and all the type of damages cannot be defined beforehand, other than that few of the damages are rare in nature and procuring or simulating the labelled images for all the combinations of that is not feasible. To overcome this, the unsupervised model uses a GAN based Anomaly detection techniques. In this model we are providing good container images, and the damage is identified as an anomaly at the inference time. This model is to ensure that we are not missing any major and severe damages which is not identified by the CenterNet model due to undefined label type or unrecognized pattern of existing defined damage type. Combining these two approaches gives a holistic view of damage detection and Improves the reliability of the system.

[0056] Fig. 8 discloses an example embodiment disclosing different categories of damage in accordance with an embodiment of the present subject matter. As discussed above, the damage can be categorized based on a level of damage in the shipping container. In one embodiment, the category of damages includes a first category of damage, a second category of damage, a third category of damage and a fourth category of damage. Under the first category of damage, the shipping container is categorized as non-damaged. If the container is judged as non-damaged, the entry / exit gate can automatically open for the shipping container and a report can be automatically generated. The report highlights that the container is non-damaged and should be allowed inside the container facility.

[0057] Under the second category of damage, the container is categorized as slightly and mid-level damaged. If the shipping container is identified has having slight and mid-level damage, the shipping container is asked to park in the designated area and a manual check may be performed of the shipping container. In this case, a damage report is generated, and notification is sent to the container facility. The report in this case can highlight the type of damages present on the shipping container, the level of damage and the location of the damage on the shipping container. The shipping container can then take the necessary actions according to the type of damage detected on the shipping container. In one embodiment, the slight and mid-level damage includes scratch, rust, wide-shallow dent.

[0058] Similarly, under the third category, the shipping container is categorized as seriously damaged. Seriously damaged shipping containers may include the damages such as deep dent on the shipping container or the presence of a hole on the shipping container can be categorized as serious damages. They can also be categorized as high priority damages which needs special attention. For these types of damages, manual inspection is required to be performed and a special care is required. The shipping facility is accordingly informed about such types of damages. The shipping facility can accordingly perform repair operations as required.

[0059] Further, the fourth category of damages are the ones which cannot be categorized under the first, second and third category of damages. These are some damages which are hard to classify. The machine learning damage detection model may find it hard to label such types of damages. These can be rare type of damages which may not have been seen before. Hence, the machine learning damage detection model may categorize these damages as having unknown classes and inform the shipping facility accordingly. Further, the aim of the present disclosure is to reduce the manual inspection of the shipping container.

[0060] Fig. 9 illustrates a flowchart of a method for inspecting a shipping container in accordance with an embodiment of the present subject matter. At step 902, the method comprises installing a plurality of cameras at a plurality of entry gates. The plurality of cameras capture one or more images of the shipping container when the shipping container passes through the entry / exit gates. The plurality of cameras can capture the images of the shipping container in real-time as the container passes through the entry / exit gates. The captured images are passed on to the inspection device.

[0061] At step 904, the method comprises receiving, by the inspection device, the plurality of captured images from the plurality of cameras. In one embodiment, the inspection device may be present at a remote location from the entry / exit gates. The inspection device may be in wireless communication with the plurality of cameras present at the entry / exit gates. The inspection device stores the machine learning model to determine a container number recognition, a marking code sequence, presence of a seal on the container, presence of a hazardous sign on the container, and presence of a damage on the container.

[0062] At step 906, the method comprises determining, by the machine learning model, container number recognition, a marking code sequence, presence of a seal on the container, presence of a hazardous sign on the container, and presence of a damage on the container. The machine learning model may be first trained by the plurality of images of different portions of the shipping container captured by the plurality of cameras at the entry / exit gates. The machine learning model can then perform the determination based on the training.

[0063] Fig. 10 illustrates a block diagram of the inspection device 206 in accordance with an embodiment of the present subject matter. The device 206 may be a server computer, comprising at least one memory 1010 and at least one processor 1020, for implementing the determination of a container number recognition, a marking code sequence, presence of a seal on the container, presence of a hazardous sign on the container, and presence of a damage on the container. The various modules in the inspection device can be embodied as a hardware that includes, without limitation, the at least one processor 1020, the at least one memory 1010, transmitter / receiver circuitry 1030, and programmable logic or software.

[0064] The at least one memory 1010, which may include both read-only memory (ROM) and random access memory (RAM), can provide instructions and data to the at least one processor 1020. The at least one memory 1010 and the at least one processor 1020 may be operatively coupled. The at least one memory 1010 may store computer readable instructions / computer program code. The at least one processor 1020 in the device 206 may train the first and second machine learning model.

[0065] In the context of this document, the "memory" (also referred to as "computer-readable media" or "computer-readable medium") may be any non-transitory media or medium or means that can contain, store, communicate, propagate or transport the instructions for use by or in connection with an instruction execution system, apparatus, or device, such as a computer. The term "non-transitory," as used herein, is a limitation of the medium itself (i.e., tangible, not a signal) as opposed to a limitation on data storage persistency (e.g., RAM vs. ROM).

[0066] The transmitter / receiver (TX / RX) circuitry 1030 may comprise a transmitter and a receiver that can enable the device 700 to transmit data to or receive data (e.g., the input image of the crop) from the plurality of cameras.

[0067] The at least one processor 1020 can be a general purpose processor, a Digital Signal Processor (DSP), an Application Specific Integrated Circuit (ASIC), a Field Programmable Gate Array (FPGA) or other programmable logic device, discrete gate or transistor logic, discrete hardware components, or any combination thereof designed to perform the functions described herein. A general purpose processor may be a microprocessor, but in the alternative, the processor may be any conventional processor, controller, microcontroller, or state machine. The processor may also be implemented as a combination of computing devices, e.g., a combination of a DSP and a microprocessor, a plurality of microprocessors, one or more microprocessors in conjunction with a DSP core, or any other such configuration. The processor can include the logic circuitry with hardware, firmware, and software architecture frameworks for facilitating image processing.

[0068] The steps of a method (e.g., method 900) described in connection with the embodiments disclosed herein may be embodied directly in hardware (e.g., device 206), in a software module executed by the at least one processor 1020, or in a combination of the two. If implemented in software, the functions may be stored on or transmitted over as one or more instructions or code on a tangible, non-transitory computer-readable medium (e.g., the at least one memory 1010). A software module may reside in Random Access Memory (RAM), flash memory, Read Only Memory (ROM), Electrically Programmable ROM (EPROM), Electrically Erasable Programmable ROM (EEPROM), registers, hard disk, a removable disk, a CD ROM, or any other form of storage medium known in the art. A storage medium is coupled to the processor such that the processor can read information from, and write information to, the storage medium.

[0069] In the several embodiments provided in this application, the disclosed system, device, and method may be implemented in another manner. For example, some features of the method embodiments described above may be ignored or not performed. The described device embodiments are merely examples.

[0070] The term based on is not exclusive and allows for being based on additional factors not described unless the context clearly dictates otherwise. In addition, throughout the specification, the meaning of "a", "an" and "the" include plural references. The meaning of "in" includes "in" and "on".

[0071] As used herein the terms "and" and "or" may be used interchangeably to refer to a set of items in both the conjunctive and disjunctive in order to encompass the full description of combinations and alternatives of the items. In either case, the set is to be interpreted as meaning each of the items singularly as alternatives, as well as any combination of the listed items.

[0072] The description above merely illustrating the technical spirit of the present disclosure, and various changes and modifications may be made by those skilled in the art without departing from the essential characteristics of the present disclosure. Therefore, the embodiments of the present disclosure described above may be implemented separately or in combination with each other.

[0073] The embodiments disclosed in the present disclosure are intended to illustrate rather than limit the scope of the present disclosure, and the scope of the technical spirit of the present disclosure is not limited by these embodiments. The scope of the present disclosure should be construed by claims below, and all technical spirits within a range equivalent to claims should be construed as being included in the right scope of the present disclosure.

[0074] While only certain features have been illustrated and described herein, many modifications and changes will occur to those skilled in the art. It is, therefore, to be understood that the appended claims are intended to cover all such modifications and changes as fall within the true spirit of the disclosure.

[0075] Further, each of the drawings or figures is merely an example to illustrate one or more example embodiments. Each figure may not be associated with only one particular example embodiment, but may be associated with one or more other example embodiments. As those of ordinary skill in the art will understand, various features or steps described with reference to any one of the figures can be combined with features or steps illustrated in one or more other figures, for example, to produce example embodiments that are not explicitly illustrated or described. Not all of the features or steps illustrated in any one of the figures to describe an example embodiment are necessarily essential, and some features or steps may be omitted. The order of the steps described in any of the figures may be changed as appropriate.

[0076] Further, some or all of the above-described example embodiments may be described in supplementary notes below, but are not limited thereto. 　　(Supplementary Note 1) 　　A system for inspecting a shipping container, the system comprising: 　　a plurality of cameras installed at a plurality of entry gates, wherein the plurality of cameras captures one or more images of the container passing through the plurality of the entry gates; 　　an inspection device operatively coupled to the plurality of cameras to receive captured images and process the one or more images using a machine learning model; 　　wherein the machine learning model is configured to determine: a container number recognition, a marking code sequence, presence of a seal on the container, presence of a hazardous sign on the container, and presence of a damage on the container. 　　(Supplementary Note 2) 　　The system according to supplementary note 1, wherein: 　　the one or more images of the container includes images of different portions of the container, 　　the different portions of the container includes images of container's rear, front, sides and / or top. 　　(Supplementary Note 3) 　　The system according to supplementary note 1, wherein the container number recognition is determined using convolution neural network (CNN) by matching the container number recognition with a checksum and performing selection of highest occurring characters(with above threshold probability) in the same position in different frames. 　　(Supplementary Note 4) 　　The system according to supplementary note 1, wherein the damage on the container includes type of damage such as at least one of tear, hole, scratch, rust, and dent. 　　(Supplementary Note 5) 　　The system according to supplementary note 1, wherein the determination of the damage includes determination of the degree of the container damage from a first category, a second category, a third category and a fourth category, wherein the first category indicates that the container is non-damaged, the second category indicates that the container is slightly damaged, the third category indicates that the container is seriously damaged, and the forth category indicates a category other than the first, second and third categories. 　　(Supplementary Note 6) 　　The system according to supplementary note 1, wherein the presence of the damage of the container is determined by: 　　providing a supervised techniques and unsupervised techniques, 　　the supervised techniques includes labelling the one or more images of the shipping container and determining the type of damage based on the labelling of the one or more images, 　　the unsupervised techniques includes comparing one or more captured images of the shipping container with one or more images of the shipping container without damage. 　　(Supplementary Note 7) 　　The system according to supplementary note 1, wherein each character of the marking code sequence depicts length, height, and type of the shipping container. 　　(Supplementary Note 8) 　　The system according to supplementary note 1, wherein the presence of hazardous sign is determined by classifying the hazardous sign using convolution neural network (CNN). 　　(Supplementary Note 9) 　　The system according to supplementary note 1, wherein the presence of the damage on the container is determined by classifying the damage into high priority damage. 　　(Supplementary Note 10) 　　The system according to supplementary note 9, wherein for high priority damage, the inspection device is configured to send an alert to an operator. 　　(Supplementary Note 11) 　　A method performed by a system for inspecting a shipping container, the method comprising: 　　installing a plurality of cameras at a plurality of entry gates, wherein the plurality of cameras captures one or more images of the container passing through the plurality of the entry gates; 　　receiving, by an inspection device, the plurality of captured device from the plurality of cameras and processing the one or more images using a machine learning model; 　　determining, using the machine learning model, a container number recognition, a marking code sequence, presence of a seal on the container, presence of a hazardous sign on the container, and presence of a damage on the container. 　　(Supplementary Note 12) 　　The method according to supplementary note 11, wherein: 　　the one or more images of the container includes images of different portions of the container, 　　the different portions of the container includes images of container's rear, front, sides and / or top. 　　(Supplementary Note 13) 　　The method according to supplementary note 11, wherein the container number recognition is determined using convolution neural network (CNN) by matching the container number recognition with a checksum and performing selection of highest occurring characters(with above threshold probability) in the same position in different frames. 　　(Supplementary Note 14) 　　The method according to supplementary note 11, wherein the damage on the container includes type of damage such as at least one of tear, hole, scratch, rust, and dent. 　　(Supplementary Note 15) 　　The method according to supplementary note 11, wherein the determination of the damage includes determination of the degree of the container damage from a first category, a second category, a third category and a fourth category, wherein the first category indicates that the container is non-damaged, the second category indicates that the container is slightly damaged, the third category indicates that the container is seriously damaged, and the forth category indicates a category other than the first, second and third categories. 　　(Supplementary Note 16) 　　The method according to supplementary note 11, wherein the presence of a damage of the container is determined by: 　　providing a supervised techniques and unsupervised techniques, 　　the supervised techniques includes labelling the one or more images of the shipping container and determining the type of damage based on the labelling of the one or more images, 　　the unsupervised techniques includes comparing one or more captured images of the shipping container with one or more images of the shipping container without damage. 　　(Supplementary Note 17) 　　The method according to supplementary note 11, wherein each character of the marking code sequence depicts length, height, and type of the shipping container. 　　(Supplementary Note 18) 　　The method according to supplementary note 11, wherein the presence of hazardous sign is determined by classifying the hazardous sign using convolution neural network (CNN). 　　(Supplementary Note 19) 　　The method according to supplementary note 11, wherein the presence of the damage on the container is determined by classifying the damage into high priority damage. 　　(Supplementary Note 20) 　　The method according to supplementary note 19, wherein for high priority damage, sending an alert to an operator.

[0077] This application is based upon and claims the benefit of priority from Indian Patent Application No. 202341079152, filed on November 21, 2023, the disclosure of which is incorporated herein in its entirety by reference.

[0078] 104　　SHIPPING CONTAINER 106　　GATE 200　　SYSTEM 202　　CAMERA 204　　ENTRY / EXIT GATE 206　　INSPECTION DEVICE 302　　MACHINE LEANING MODEL 304　　MACHINE LEARNING DAMAGE DETECTION MODE 306　　MACHINE LEARNING CONTAINER NUMBER RECOGNITION MODEL 308　　MACHINE LEARNING ISO MODEL 310　　MACHINE LEARNING E-SEAL MODEL 312　　MACHINE LEARNING HAZARD SIGN DETECTION MODEL 1010　　MEMORY 1020　　PROCESSOR 1030　　TRANSMITTER / RECEIVER CIRCUITRY

Claims

1. A system for inspecting a shipping container, the system comprising: 　　a plurality of cameras installed at a plurality of entry gates, wherein the plurality of cameras captures one or more images of the container passing through the plurality of the entry gates; 　　an inspection device operatively coupled to the plurality of cameras to receive captured images and process the one or more images using a machine learning model; 　　wherein the machine learning model is configured to determine: a container number recognition, a marking code sequence, presence of a seal on the container, presence of a hazardous sign on the container, and presence of a damage on the container.

2. The system as claimed in claim 1, wherein: 　　the one or more images of the container includes images of different portions of the container, 　　the different portions of the container includes images of container's rear, front, sides and / or top.

3. The system as claimed in claim 1, wherein the container number recognition is determined using convolution neural network (CNN) by matching the container number recognition with a checksum and performing selection of highest occurring characters(with above threshold probability) in the same position in different frames.

4. The system as claimed in claim 1, wherein the damage on the container includes type of damage such as at least one of tear, hole, scratch, rust, and dent.

5. The system as claimed in claim 1, wherein the determination of the damage includes determination of the degree of the container damage from a first category, a second category, a third category and a fourth category, wherein the first category indicates that the container is non-damaged, the second category indicates that the container is slightly damaged, the third category indicates that the container is seriously damaged, and the forth category indicates a category other than the first, second and third categories.

6. The system as claimed in claim 1, wherein the presence of the damage of the container is determined by: 　　providing a supervised techniques and unsupervised techniques, 　　the supervised techniques includes labelling the one or more images of the shipping container and determining the type of damage based on the labelling of the one or more images, 　　the unsupervised techniques includes comparing one or more captured images of the shipping container with one or more images of the shipping container without damage.

7. The system as claimed in claim 1, wherein each character of the marking code sequence depicts length, height, and type of the shipping container.

8. The system as claimed in claim 1, wherein the presence of hazardous sign is determined by classifying the hazardous sign using convolution neural network (CNN).

9. The system as claimed in claim 1, wherein the presence of the damage on the container is determined by classifying the damage into high priority damage.

10. The system as claimed in claim 9, wherein for high priority damage, the inspection device is configured to send an alert to an operator.

11. A method performed by a system for inspecting a shipping container, the method comprising: 　　installing a plurality of cameras at a plurality of entry gates, wherein the plurality of cameras captures one or more images of the container passing through the plurality of the entry gates; 　　receiving, by an inspection device, the plurality of captured device from the plurality of cameras and processing the one or more images using a machine learning model; 　　determining, using the machine learning model, a container number recognition, a marking code sequence, presence of a seal on the container, presence of a hazardous sign on the container, and presence of a damage on the container.

12. The method as claimed in claim 11, wherein: 　　the one or more images of the container includes images of different portions of the container, 　　the different portions of the container includes images of container's rear, front, sides and / or top.

13. The method as claimed in claim 11, wherein the container number recognition is determined using convolution neural network (CNN) by matching the container number recognition with a checksum and performing selection of highest occurring characters(with above threshold probability) in the same position in different frames.

14. The method as claimed in claim 11, wherein the damage on the container includes type of damage such as at least one of tear, hole, scratch, rust, and dent.

15. The method as claimed in claim 11, wherein the determination of the damage includes determination of the degree of the container damage from a first category, a second category, a third category and a fourth category, wherein the first category indicates that the container is non-damaged, the second category indicates that the container is slightly damaged, the third category indicates that the container is seriously damaged, and the forth category indicates a category other than the first, second and third categories.

16. The method as claimed in claim 11, wherein the presence of a damage of the container is determined by: 　　providing a supervised techniques and unsupervised techniques, 　　the supervised techniques includes labelling the one or more images of the shipping container and determining the type of damage based on the labelling of the one or more images, 　　the unsupervised techniques includes comparing one or more captured images of the shipping container with one or more images of the shipping container without damage.

17. The method as claimed in claim 11, wherein each character of the marking code sequence depicts length, height, and type of the shipping container.

18. The method as claimed in claim 11, wherein the presence of hazardous sign is determined by classifying the hazardous sign using convolution neural network (CNN).

19. The method as claimed in claim 11, wherein the presence of the damage on the container is determined by classifying the damage into high priority damage.

20. The method as claimed in claim 19, wherein for high priority damage, sending an alert to an operator.

Images