A container damage inspection system with lead seal mark

Abstract

The present invention discloses a container damage inspection system with a seal mark, which belongs to the technical field of container damage inspection, and includes an image acquisition module, a preprocessing module, a seal recognition module, a damage recognition module, a damage classification module, a database module, and an early warning module. The present invention can conveniently obtain RGB three-channel color images and depth images of each surface of the container with the same parameters through the image acquisition module; through the seal recognition module, the seal mark number of the container can be conveniently identified, which is conducive to timely understanding of the damage status of each container; through the damage recognition module, the type of the damaged area can be conveniently determined by calculating the depth value of the pixel points in the damaged area, thereby improving the accuracy of damage recognition and classification, and is worthy of being promoted and used.

Description

Technical Field

[0001] The invention relates to the technical field of container damage inspection, and in particular to a container damage inspection system with a lead seal mark. Background Art

[0002] Artificial intelligence is a product of social development and technological innovation, and is an important technological form that promotes human progress. Since its development, artificial intelligence has become the core driving force of a new round of scientific and technological revolution and industrial transformation, and is having a profound impact on the world economy, social progress and people's lives. For the world economy, artificial intelligence is a strategic technology that leads the future. Major countries and regions in the world have regarded the development of artificial intelligence as a major strategy to enhance national competitiveness and promote national economic growth; for daily life, artificial intelligence technologies such as deep learning, image recognition, and voice recognition have been widely used in smart terminals, smart homes, mobile payments and other fields. In the future, artificial intelligence technology will also play a more significant role in education, medical care, travel and other fields that are closely related to people's lives, providing ordinary people with wider coverage, better experience, and more convenient life services.

[0003] At present, multiple machine vision applications based on deep learning, such as container number recognition, internal and external truck number recognition, and dangerous goods recognition, have been successfully deployed in ports and have been running stably for many years. The successful commissioning of these applications has provided new technologies and ideas for port operations, effectively reducing operating costs, improving operating efficiency, and reducing operating interference.

[0004] The existing container damage inspection system has a high recognition rate for damage caused by holes on the top of the container. However, the damage inspection only targets the top of the container and is temporarily unable to identify damage on the sides and door of the container. In addition, it is unable to distinguish the types of damage, making it difficult to identify and classify damage such as holes, rust, and dents. The above problems need to be solved urgently. To this end, a container damage inspection system with a lead seal mark is proposed. Summary of the invention

[0005] The technical problem to be solved by the present invention is: how to solve the problem in the prior art that damage on the sides and door sides of the container cannot be identified, and the types of damage cannot be distinguished, and it is difficult to identify and classify damage such as holes, rust, and dents. A container damage inspection system with a lead seal mark is provided.

[0006] The present invention solves the above technical problems through the following technical solutions, which include: an image acquisition module, a preprocessing module, a seal recognition module, a damage recognition module, a damage classification module, a database module, and an early warning module;

[0007] The image acquisition module is used to acquire RGB three-channel color images and depth images of the top surface, two end surfaces, two side surfaces, and bottom surface of the container;

[0008] The preprocessing module is used to perform size standardization, noise reduction and enhancement processing on the RGB three-channel color image and the depth image;

[0009] The seal identification module is used to identify the container seal identification and identification number;

[0010] The damage recognition module is used to perform target recognition on the damaged area in the RGB three-channel color image;

[0011] The damage classification module is used to obtain the damaged area in the depth image according to the damaged area recognition result in the RGB three-channel color image, and calculate the depth value of each pixel in the damaged area in the depth image to obtain the damage type of each damaged area, and then obtain the number of damaged areas of the current container and the corresponding type, that is, the damage recognition result;

[0012] The database module is used to store the damage identification results of each container based on the container seal identification number;

[0013] The early warning module is used to score the damage status of the current container according to the damage identification result, obtain the damage status score of each container, and when the damage status score of the container exceeds the early warning set threshold, warn the staff and remind the staff to handle the container.

[0014] Furthermore, the image acquisition module includes a first image acquisition unit and a second image acquisition unit; the first image acquisition unit is used to acquire RGB three-channel color images of the top surface, two end surfaces, two side surfaces, and bottom surface of the container; the second image acquisition unit is used to acquire depth images of the top surface, two end surfaces, two side surfaces, and bottom surface of the container.

[0015] Furthermore, the first image acquisition unit includes six groups of high-definition cameras arranged on the top surface, two end surfaces, two side surfaces and bottom surface of the container, and the optical axes of the six groups of high-definition cameras are all arranged perpendicular to the corresponding surfaces of the container; the second image acquisition unit includes six groups of depth cameras arranged on the top surface, two end surfaces, two side surfaces and bottom surface of the container, and the optical axes of the six groups of depth cameras are all arranged perpendicular to the corresponding surfaces of the container; the parameters of the high-definition camera and the depth camera are the same, the sizes of the RGB three-channel color image and the depth image of the corresponding surface acquired are the same, and the positions of each pixel point in the RGB three-channel color image and the depth image correspond to the same.

[0016] Furthermore, the preprocessing module includes a size standardization unit, an image noise reduction unit, and an image enhancement unit. The size standardization unit is used to crop the RGB three-channel color image and depth image of the corresponding surface of the container to a set size; the image noise reduction unit is used to perform noise reduction processing on the cropped RGB three-channel color image and depth image of the corresponding surface of the container; and the image enhancement unit is used to enhance the RGB three-channel color image and depth image that have been subjected to noise reduction processing.

[0017] Furthermore, the seal recognition module includes a first target detection unit, a second target detection unit, and a result library establishment unit; the first target detection unit is used to use the trained first target detection network to perform target recognition on the container seal mark in the RGB three-channel color image, obtain the container seal mark target detection frame, and then cut out the container seal mark target detection frame to obtain the container seal mark target detection frame image; the second target detection unit is used to use the trained second target detection network to identify the identification number in the container seal mark target detection frame image, and obtain the current container identification number information; the result library establishment unit is used to establish the current container damage inspection result library in the database module according to the identification number of the current container.

[0018] Furthermore, the damage recognition module includes a third target detection unit and a contour detection unit; the third target detection unit is used to use the trained third target detection network to perform target recognition on the damaged area in the RGB three-channel color image, and obtain the target detection frame of the damaged area in the RGB three-channel color image; the contour detection unit is used to perform outer contour recognition on the damaged area target detection frame in the RGB three-channel color image, and crop the damaged area based on the outer contour line to obtain the corresponding damaged area picture, and then send each damaged area picture to the damage classification module.

[0019] Furthermore, the damage classification module includes a defect area acquisition unit, a depth value calculation unit, and a damage judgment unit; the defect area acquisition unit is used to obtain the corresponding damaged area in the depth image according to the position of the damaged area picture in the RGB three-channel color image; the depth value calculation unit is used to calculate the mean value of each pixel point in each corresponding damaged area in the depth image; the damage judgment unit is used to compare the mean value of each pixel point in each damaged area with the set threshold value of hole damage, rust damage, and dent damage to obtain the number and corresponding types of damaged areas of the current container, that is, the damage recognition result, and send the damage recognition result of the current container to the damage inspection result library.

[0020] Furthermore, the specific processing process of the depth value calculation unit is as follows:

[0021] S1: Calculate the mean value Xi of each pixel in each corresponding damaged area in the depth image, where i represents the sequence number of each damaged area;

[0022] S2: Compare the mean value Xi of each pixel point in each damaged area with the hole damage setting threshold value Y1, the rust damage setting threshold value Y2, and the dent damage setting threshold value Y3;

[0023] S3: When the mean value Xi of each pixel point in the damaged area falls within the range of the hole damage setting threshold value Y1, the damaged area is judged to be hole damage; when the mean value Xi of each pixel point in the damaged area falls within the range of the rust damage setting threshold value Y2, the damaged area is judged to be rust damage; when the mean value Xi of each pixel point in the damaged area falls within the range of the concave damage setting threshold value Y3, the damaged area is judged to be concave damage.

[0024] Furthermore, in step S2, the hole damage setting threshold value Y1, the rust damage setting threshold value Y2, and the dent damage setting threshold value Y3 are all range values, wherein the lower limit value of the hole damage setting threshold value Y1 is greater than the upper limit value of the dent damage setting threshold value Y3, and the lower limit value of the dent damage setting threshold value Y3 is greater than the upper limit value of the rust damage setting threshold value Y2.

[0025] Furthermore, the scoring formula of the early warning module for the damage status of the current container is as follows:

[0026] T=w1*D+w2*S+w3*A

[0027] Among them, T is the damage status score of the current container, D is the number of holes on the current container, S is the number of rust damage on the current container, A is the number of dents on the current container, and w1, w2, and w3 are the weights of the corresponding damage types in the damage status score.

[0028] Compared with the prior art, the present invention has the following advantages: through the set image acquisition module, the RGB three-channel color image and depth image of each surface of the container with the same parameters can be easily obtained; through the set seal recognition module, the seal identification number of the container can be easily identified, which is conducive to timely understanding of the damage status of each container; through the set damage recognition module, the type of the damaged area can be conveniently determined by calculating the depth value of the pixel points in the damaged area, thereby improving the accuracy of damage recognition and classification, and is worthy of popularization and use. BRIEF DESCRIPTION OF THE DRAWINGS

[0029] Figure 1 1 is a schematic diagram of the structure of a container damage inspection system with a lead seal mark according to an embodiment of the present invention;

[0030] Figure 2 is a structural schematic diagram of a seal identification module in an embodiment of the present invention;

[0031] Figure 3 is a schematic diagram of the structure of a damage identification module in an embodiment of the present invention;

[0032] Figure 4 Schematic diagram of the structure of the damage classification module in the embodiment of the present invention. DETAILED DESCRIPTION

[0033] The following is a detailed description of an embodiment of the present invention. This embodiment is implemented on the premise of the technical solution of the present invention, and a detailed implementation method and a specific operation process are given, but the protection scope of the present invention is not limited to the following embodiment.

[0034] like Figure 1-4 As shown, this embodiment provides a technical solution: a container damage inspection system with a lead seal mark, including: an image acquisition module, a preprocessing module, a lead seal recognition module, a damage recognition module, a damage classification module, a database module, and an early warning module;

[0035] The image acquisition module includes a first image acquisition unit and a second image acquisition unit; the first image acquisition unit is used to acquire RGB three-channel color images of the top surface, two end surfaces, two side surfaces and the bottom surface of the container, and transmit the RGB three-channel color images to the preprocessing module for preprocessing by the preprocessing module; the second image acquisition unit is used to acquire depth images of the top surface, two end surfaces, two side surfaces and the bottom surface of the container, and transmit the depth images to the preprocessing module for preprocessing by the preprocessing module;

[0036] The preprocessing module includes an image noise reduction unit and an image enhancement unit; the image noise reduction unit is used to perform noise reduction processing on the RGB three-channel color image and the depth image of the corresponding surface of the container, reduce the noise in the RGB three-channel color image and the depth image, and improve the accuracy of subsequent recognition. After the noise reduction processing is completed, the RGB three-channel color image and the depth image are sent to the image enhancement unit for the image enhancement unit to perform enhancement processing on the image; the image enhancement unit is used to enhance the RGB three-channel color image and the depth image after the noise reduction processing, improve the contrast of the image, make the image clearer, and improve the accuracy of subsequent recognition. After the enhancement processing is completed, the RGB three-channel color image is sent to the seal recognition module for the seal recognition module to recognize the container seal mark in the RGB three-channel color image, and the depth image is sent to the damage classification module at the same time;

[0037] The seal recognition module includes a first target detection unit, a second target detection unit, and a result library establishment unit; the first target detection unit is used to use the trained first target detection network to perform target recognition on the container seal mark in the RGB three-channel color image, obtain the container seal mark target detection frame, and then cut out the container seal mark target detection frame to obtain the container seal mark target detection frame image and send it to the second target detection unit; the second target detection unit is used to use the trained second target detection network to identify the identification number in the container seal mark target detection frame image, obtain the current container identification number information, and send the current container identification number information to the result library establishment unit; the result library establishment unit is used to establish the current container damage inspection result library in the database module according to the identification number of the current container;

[0038] The damage recognition module includes a third target detection unit and a contour detection unit; the third target detection unit is used to use the trained third target detection network to perform target recognition on the damaged area in the RGB three-channel color image, and obtain the target detection frame of the damaged area in the RGB three-channel color image; the contour detection unit is used to perform outer contour recognition on the target detection frame of the damaged area in the RGB three-channel color image (using the contour recognition function in OpenCV to realize contour recognition), and cut out the damaged area based on the outer contour line, obtain the corresponding damaged area picture, and then send each damaged area picture to the damage classification module;

[0039] The damage classification module includes a defect area acquisition unit, a depth value calculation unit, and a damage judgment unit; the defect area acquisition unit is used to acquire the corresponding damaged area in the depth image according to the position of the damaged area picture in the RGB three-channel color image; the depth value calculation unit is used to calculate the mean value of each pixel point in each damaged area corresponding to the depth image; the damage judgment unit is used to compare the mean value of each pixel point in each damaged area with the set threshold value of hole damage, rust damage, and concave damage, and judge the damaged area as hole damage if it falls within the set threshold range of hole damage, judge the damaged area as rust damage if it falls within the set threshold range of rust damage, and judge the damaged area as concave damage if it falls within the set threshold range of concave damage, and obtain the number of damaged areas and the corresponding types of the current container, that is, the damage recognition result, and send the damage recognition result of the current container to the damage inspection result library;

[0040] The database module is used to store damage identification results of each container based on the container seal identification number;

[0041] The early warning module is used to score the damage status of the current container according to the damage identification result, obtain the damage status score of each container, and when the damage status score of the container exceeds the early warning set threshold, warn the staff and remind the staff to handle the container.

[0042] In this embodiment, the first image acquisition unit includes six groups of high-definition cameras arranged on the top surface, two end surfaces, two side surfaces and the bottom surface of the container, and the optical axes of the six groups of high-definition cameras are arranged perpendicular to the corresponding surfaces of the container.

[0043] In this embodiment, the second image acquisition unit includes six groups of depth cameras arranged on the top surface, two end surfaces, two side surfaces and the bottom surface of the container, and the optical axes of the six groups of depth cameras are arranged perpendicular to the corresponding surfaces of the container.

[0044] In this embodiment, the parameters of the high-definition camera and the depth camera are the same, the sizes of the acquired RGB three-channel color image and depth image of the corresponding surface are the same, and the positions of each pixel in the RGB three-channel color image and the depth image are the same.

[0045] It should be noted that during shooting, the high-definition camera first reaches the designated position for shooting under the control of a group of manipulators. After shooting, the high-definition camera is retracted, and another group of manipulators moves the depth camera to the shooting position of the high-definition camera to continue shooting, and successively obtains RGB three-channel color images and depth images of the top, two end faces, two side faces, and bottom of the container.

[0046] In this embodiment, the image noise reduction unit uses a Gaussian noise reduction method to reduce the noise of the image, and the image enhancement unit uses a histogram equalization method to enhance the image.

[0047] In this embodiment, the preprocessing module also includes a size standardization unit, which is used to crop the RGB three-channel color image and the depth image of the corresponding surface of the container to a set size, so that the positions of each pixel point of the RGB three-channel color image and the depth image can be completely corresponding, which can improve the subsequent recognition accuracy while ensuring the calculation speed.

[0048] In this embodiment, the first target detection network, the second target detection network, the third target detection network, and the fourth target detection network all adopt any one of Faster RCNN and Yolo V3 target detection networks.

[0049] In this embodiment, the processing process of the first target detection unit is specifically as follows:

[0050] S101: using the trained first target detection network to perform target recognition on the container seal mark in the RGB three-channel color image to obtain a container seal mark target detection frame;

[0051] S102: cropping the container seal mark target detection frame, obtaining a container seal mark target detection frame image, and sending the image to a second target detection unit.

[0052] In this embodiment, the processing process of the second target detection unit is as follows:

[0053] S201: using the trained second target detection network to identify the identification number in the container seal identification target detection frame image, and obtaining the current container identification number information;

[0054] S202: Send the current container identification number information to the result database establishing unit.

[0055] In this embodiment, the specific processing process of the depth value calculation unit is as follows:

[0056] S301: Calculate the mean value Xi of each pixel in each corresponding damaged area in the depth image, where i represents the sequence number of each damaged area;

[0057] S302: comparing the mean value Xi of each pixel point in each damaged area with the hole damage setting threshold value Y1, the rust damage setting threshold value Y2, and the dent damage setting threshold value Y3;

[0058] S303: When the mean value Xi of each pixel point in the damaged area falls within the hole damage setting threshold value Y1, the damaged area is judged to be hole damage; when the mean value Xi of each pixel point in the damaged area falls within the rust damage setting threshold value Y2, the damaged area is judged to be rust damage; when the mean value Xi of each pixel point in the damaged area falls within the concave damage setting threshold value Y3, the damaged area is judged to be concave damage.

[0059] In this embodiment, the hole damage setting threshold value Y1, the rust damage setting threshold value Y2, and the dent damage setting threshold value Y3 are all range values, wherein the lower limit value of the hole damage setting threshold value Y1 is greater than the upper limit value of the dent damage setting threshold value Y3, and the lower limit value of the dent damage setting threshold value Y3 is greater than the upper limit value of the rust damage setting threshold value Y2.

[0060] It should be noted that in the depth image, the pixel value of each pixel indicates its distance from the depth camera, which makes it easy to distinguish the damage category.

[0061] In this embodiment, the scoring formula used by the early warning module to score the damage status of the current container is as follows:

[0062] T=w1*D+w2*S+w3*A

[0063] Among them, T is the damage status score of the current container, D is the number of holes on the current container, S is the number of rust damage on the current container, A is the number of dents on the current container, and w1, w2, and w3 are the weights of the corresponding damage types in the damage status score.

[0064] The use of this system can achieve an accuracy rate of 95% in identifying damage on all sides of the container, and the warning error rate is less than 15%. For the item of identifying damage types, the classification accuracy rate is not less than 90% for damage types that can be clearly classified manually.

[0065] Although the embodiments of the present invention have been shown and described above, it is to be understood that the above embodiments are exemplary and are not to be construed as limitations of the present invention. A person skilled in the art may change, modify, replace and vary the above embodiments within the scope of the present invention.

Claims

1. A container damage inspection system with a lead seal mark, characterized in that: include: Image acquisition module, preprocessing module, seal recognition module, damage recognition module, damage classification module, database module, early warning module; The image acquisition module is used to acquire RGB three-channel color images and depth images of the top surface, two end surfaces, two side surfaces, and bottom surface of the container; The preprocessing module is used to perform size standardization, noise reduction and enhancement processing on the RGB three-channel color image and the depth image; The seal identification module is used to identify the container seal identification and identification number; The damage recognition module is used to perform target recognition on the damaged area in the RGB three-channel color image; The damage classification module is used to obtain the damaged area in the depth image according to the damaged area recognition result in the RGB three-channel color image, and calculate the depth value of each pixel in the damaged area in the depth image to obtain the damage type of each damaged area, and then obtain the number of damaged areas of the current container and the corresponding type, that is, the damage recognition result; The database module is used to store the damage identification results of each container based on the container seal identification number; The early warning module is used to score the damage status of the current container according to the damage identification result, obtain the damage status score of each container, and when the damage status score of the container exceeds the early warning set threshold, warn the staff to remind the staff to handle the container; The image acquisition module includes a first image acquisition unit and a second image acquisition unit; the first image acquisition unit is used to acquire RGB three-channel color images of the top surface, two end surfaces, two side surfaces, and the bottom surface of the container; the second image acquisition unit is used to acquire depth images of the top surface, two end surfaces, two side surfaces, and the bottom surface of the container; The first image acquisition unit includes six groups of high-definition cameras arranged on the top surface, two end surfaces, two side surfaces and the bottom surface of the container, and the optical axes of the six groups of high-definition cameras are all arranged perpendicular to the corresponding surfaces of the container; the second image acquisition unit includes six groups of depth cameras arranged on the top surface, two end surfaces, two side surfaces and the bottom surface of the container, and the optical axes of the six groups of depth cameras are all arranged perpendicular to the corresponding surfaces of the container; the parameters of the high-definition cameras and the depth cameras are the same, the sizes of the RGB three-channel color images and depth images of the corresponding surfaces obtained are the same, and the positions of the pixels in the RGB three-channel color images and the depth images correspond to the same; The preprocessing module includes a size standardization unit, an image noise reduction unit, and an image enhancement unit, wherein the size standardization unit; Used to crop the RGB three-channel color image and depth image of the corresponding surface of the container to a set size; the image denoising unit is used to perform denoising on the cropped RGB three-channel color image and depth image of the corresponding surface of the container; the image enhancement unit is used to enhance the RGB three-channel color image and depth image after denoising; The seal recognition module includes a first target detection unit, a second target detection unit, and a result library establishment unit; the first target detection unit is used to use the trained first target detection network to perform target recognition on the container seal mark in the RGB three-channel color image, obtain the container seal mark target detection frame, and then cut out the container seal mark target detection frame to obtain the container seal mark target detection frame image; The second target detection unit is used to use the trained second target detection network to identify the identification number in the container seal identification target detection frame image to obtain the current container identification number information; the result library establishment unit is used to establish the damage inspection result library of the current container in the database module according to the identification number of the current container; The damage recognition module includes a third target detection unit and a contour detection unit; the third target detection unit is used to use the trained third target detection network to perform target recognition on the damaged area in the RGB three-channel color image, and obtain the target detection frame of the damaged area in the RGB three-channel color image; the contour detection unit is used to perform outer contour recognition on the damaged area target detection frame in the RGB three-channel color image, and cut out the damaged area based on the outer contour line, obtain the corresponding damaged area picture, and then send each damaged area picture to the damage classification module; The damage classification module includes a defect area acquisition unit, a depth value calculation unit, and a damage judgment unit; The defect area acquisition unit is used to acquire the corresponding damaged area in the depth image according to the position of the damaged area picture in the RGB three-channel color image; the depth value calculation unit is used to calculate the mean value of each pixel point in each corresponding damaged area in the depth image; the damage judgment unit is used to compare the mean value of each pixel point in each damaged area with the set threshold value of hole damage, rust damage, and dent damage to obtain the number of damaged areas and the corresponding types of the current container, that is, the damage recognition result, and send the damage recognition result of the current container to the damage inspection result library.

2. A container damage inspection system with a lead seal mark according to claim 1, characterized in that: The specific processing process of the depth value calculation unit is as follows: S1: Calculate the mean value Xi of each pixel in each corresponding damaged area in the depth image, where i represents the sequence number of each damaged area; S2: Compare the mean value Xi of each pixel point in each damaged area with the hole damage setting threshold value Y1, the rust damage setting threshold value Y2, and the dent damage setting threshold value Y3; S3: When the mean value Xi of each pixel point in the damaged area falls within the range of the hole damage setting threshold value Y1, the damaged area is judged to be hole damage; when the mean value Xi of each pixel point in the damaged area falls within the range of the rust damage setting threshold value Y2, the damaged area is judged to be rust damage; when the mean value Xi of each pixel point in the damaged area falls within the range of the concave damage setting threshold value Y3, the damaged area is judged to be concave damage.

3. The container damage inspection system with lead seal mark according to claim 2 is characterized by: In step S2, the hole damage setting threshold value Y1, the rust damage setting threshold value Y2, and the dent damage setting threshold value Y3 are all range values, wherein the lower limit value of the hole damage setting threshold value Y1 is greater than the upper limit value of the dent damage setting threshold value Y3, and the lower limit value of the dent damage setting threshold value Y3 is greater than the upper limit value of the rust damage setting threshold value Y2.

4. A container damage inspection system with a lead seal mark according to claim 1 or 2, characterized in that: The scoring formula of the early warning module for the damage status of the current container is as follows: T=w1*D+w2*S+w3*A Among them, T is the damage status score of the current container, D is the number of holes on the current container, S is the number of rust damage on the current container, A is the number of dents on the current container, and w1, w2, and w3 are the weights of the corresponding damage types in the damage status score.

Images