Container detection method, system and device based on machine vision and storage medium

Abstract

The application discloses a container detection method, system and device based on machine vision and a storage medium, relates to the technical field of containers, and comprises the following steps: obtaining colored detection parts and non-colored detection parts based on a seven-item inspection method of containers; building an external contour model; performing one-time filling on the external contour model based on machine vision; obtaining a virtual container model; and identifying defects of the container based on the virtual container model. The application is used for solving the problem that, in the existing container detection method, when there are defects inside the container or defects in the blind area of the container outside, the defects cannot be detected in a targeted manner, and after the defect parts are obtained, the identification of the defects is relatively fuzzy, thereby affecting the maintenance of the management personnel and the use of the container.

Description

Technical Field

[0001] This invention relates to the field of container inspection technology, specifically to container inspection methods, systems, equipment, and storage media based on machine vision. Background Technology

[0002] A container is a standardized means of transport used to load packaged or unpackaged goods, facilitating loading, unloading, and handling by machinery. It is a key element in the modern logistics system and is widely used in various modes of transport, including sea, land, and air. Container inspection refers to a series of checks and tests performed on containers during transport to ensure they meet safety standards and transport requirements. The main purpose of container inspection is to ensure the structural integrity, sealing, cleanliness, and compliance with specific transport requirements of the container.

[0003] Existing methods for container inspection typically rely on scanning the container surface and using real-world images to identify defects and flaws. These defects are then compared to a standard model to obtain the inspection result. While this method can identify external defects, it cannot effectively detect defects inside the container or in areas where external scanning is limited. Furthermore, the identification of defects after acquisition is often vague, leading to issues that hinder maintenance by management personnel and impact container usability. For example, patent application CN115937185A discloses a container inspection system and method. This solution involves acquiring real-world images, point cloud files, and container information of the container surface. It then uses these images to identify defects in the container surface and compares them with a baseline model of defect-free units to determine the defects on each surface. Other improvements in container inspection typically focus on the container's condition and performance. However, in detecting container defects, there are still limitations. When defects exist inside the container or in areas outside the scanning blind spots, it's impossible to target the specific location of the defect. Furthermore, the identification of defects after acquisition is often vague, impacting maintenance by management personnel and the use of the container. Therefore, it is necessary to improve existing container inspection methods. Summary of the Invention

[0004] This invention aims to at least partially solve one of the technical problems in the prior art by proposing a machine vision-based container inspection method, system, equipment, and storage medium. This addresses the shortcomings of existing container inspection methods, which fail to specifically detect defects when they exist inside the container or in areas of blind spots on the outside of the container. Furthermore, the identification of defects after acquisition is often vague, leading to problems that affect maintenance by management personnel and the use of the container.

[0005] To achieve the above objectives, in a first aspect, this application provides a machine vision-based container inspection method, comprising the following steps:

[0006] Multiple inspection parts are obtained based on the seven-item inspection method for containers, and these parts are classified. Colored and colorless inspection parts are obtained based on the classification results. An external contour model is built based on the external dimension data of the container.

[0007] Based on machine vision, defects are detected in multiple classified parts to be detected, and the external contour model is filled once based on the defect detection results; the external contour model after filling is denoted as the part filling model.

[0008] The interior of the container is scanned using LiDAR, and the part filling model is filled in a second time based on the scan data to obtain a virtual container model; the defects of the container are marked based on the virtual container model.

[0009] Furthermore, multiple areas to be detected are classified, and based on the classification results, colored and colorless detection areas are obtained, including:

[0010] Based on the seven-item inspection method for containers, all inspection parts of the container in the seven-item inspection method are obtained and recorded as the parts to be inspected; for any part to be inspected: the inspection description of the part to be inspected in the seven-item inspection method for containers is recorded as the inspection identification data of the part to be inspected; the inspection identification data of all parts to be inspected are obtained and word segmentation is performed on all inspection identification data.

[0011] All color names are obtained based on big data and stored in the color database. For any detection and recognition data: when there is a word in the detection and recognition data that is the same as any word in the color database, the part to be detected corresponding to the detection and recognition data is recorded as a colored detection part; when all words in the detection and recognition data are different from all words in the color database, the part to be detected corresponding to the detection and recognition data is recorded as a colorless detection part.

[0012] Furthermore, building an external contour model based on the external dimensional data of the container includes:

[0013] The external dimensions of the container are acquired using machine vision. The external dimensions include the length and width of each side of the container. A spatial rectangular coordinate system with units of meters for the X, Y, and Z axes is constructed and denoted as the detection coordinate system. Based on the external dimensions of the container, a rectangular body is constructed within the detection coordinate system and denoted as the external contour model.

[0014] Arrange any vertex of the external contour model to coincide with the origin of the coordinate system, and make the front view, left view and top view of the external contour model parallel to the XZ plane, YZ plane and XY plane respectively. The vertex that coincides with the origin of the coordinate system is called the feature vertex.

[0015] Furthermore, defect detection includes both colored defect detection and colorless defect detection. Colored defect detection includes:

[0016] For any colored detection area, a camera is used to capture the colored detection area, and the resulting image is recorded as a colored detection image; the outline of the colored detection area in the colored detection image is recorded as a colored detection outline; the colors within the colored detection image are identified, and based on the identification results, the boundary lines between different colors within the colored detection image are obtained and recorded as colored boundary lines; the colored boundary lines are marked within the colored detection outline.

[0017] The detection standards corresponding to the colored detection areas are obtained based on the seven detection methods, and the colored detection outlines are colored based on the detection standards. When the boundary line between any two colored areas and any colored boundary line do not coincide for any length during the coloring process, the colored detection area is recorded as a colored defect area.

[0018] When the dividing lines between all different colors coincide with all colored dividing lines during the coloring process, the colored detection outlines and colored detection images are superimposed. If the color of the colored detection outline in any overlapping area is different from that in the colored detection image, the colored detection area is recorded as a colored defect area. If the color of the colored detection outline in all overlapping areas is the same as that in the colored detection image, the colored detection area is recorded as a qualified area.

[0019] Furthermore, the colorless defect detection includes: for any colorless detection area, using a camera to capture the colorless detection area and recording the resulting image as a colorless detection image; recording the area of ​​the colorless detection area in the colorless detection image as a colorless detection region; obtaining the colorless detection region corresponding to the flawless colorless detection area based on the seven-item detection method and recording it as a colorless comparison region.

[0020] For any pixel within the colorless detection area: obtain the gray value of the pixel and the gray values ​​of all pixels in its eight neighboring regions; obtain the difference between the gray value of the pixel and the gray values ​​of all pixels in its eight neighboring regions, and record the largest difference as the neighbor difference value; obtain the neighbor difference values ​​corresponding to all pixels.

[0021] The mode of all neighbor differences is recorded as the maximum neighbor difference value; all pixels with the maximum neighbor difference value are marked as feature pixels within the colorless detection region; for any feature pixel α: the distance between feature pixel α and the feature pixel farthest from feature pixel α is recorded as the limiting feature parameter of feature pixel α; the limiting feature parameters of all feature pixels are obtained, and the feature algorithm is used to obtain the detection parameters of the colorless detection region. The feature algorithm is as follows: Where F is the detection parameter, G i G represents the limiting feature parameter of the i-th feature pixel among all feature pixels. sq is the average of all limiting feature parameters, c is the number of feature pixels, g is the number of neighboring feature pixels, and neighboring feature pixels are feature pixels that have feature pixels in their eight neighborhoods.

[0022] When the detection parameters of the colorless comparison area are the same as those of the colorless detection area, the colorless detection area is recorded as a qualified area; when the detection parameters of the colorless comparison area are different from those of the colorless detection area, the colorless detection area is recorded as a defective area.

[0023] Furthermore, the process of filling the external contour model based on the defect detection results includes:

[0024] One-time filling is as follows: obtain the position of all parts to be detected in the container, and mark the position of all parts to be detected in the outer contour model based on the positional relationship between all parts to be detected and the feature apex; based on the positional relationship between the parts to be detected and the outer contour model and the actual position of each part to be detected, the parts to be detected that are inside the outer contour model and in contact with the inner wall of the container are recorded as internal positions.

[0025] The external contour model at this point is denoted as the part filling model.

[0026] Furthermore, the interior of the container is scanned using LiDAR, and the area filling model is re-filled based on the scan data to obtain a virtual container model; based on the virtual container model, defects in the container are identified, including:

[0027] The interior of the container is scanned using a LiDAR scanner, and the space corresponding to the scanned data is recorded as the actual internal space. Based on the positional relationship between the actual internal space and the feature apex, the actual internal space is placed inside the part-filling model; the part-filling model at this time is recorded as the virtual container model.

[0028] When any internal part does not fit with the actual internal space, the internal part is recorded as a defective part and the container is recorded as a misaligned container; when all internal parts fit with the actual internal space, the container is recorded as a container with intact space.

[0029] Mark all defective parts within the virtual container model.

[0030] Secondly, this application also provides a machine vision-based container inspection system, including an external inspection and analysis module, a defect filling module, and a virtual model comparison module;

[0031] The external inspection and analysis module is used to obtain multiple parts to be inspected based on the seven-item inspection method for containers, classify the multiple parts to be inspected, and obtain colored and colorless inspection parts based on the classification results; and build an external contour model based on the external dimension data of the container.

[0032] The defect filling module is used to perform defect detection on multiple classified parts to be detected based on machine vision, and to fill the outer contour model once based on the defect detection results; the outer contour model after one filling is denoted as the part filling model;

[0033] The virtual model comparison module is used to scan the interior of the container using LiDAR, and to perform secondary filling of the part filling model based on the scan data to obtain a virtual container model; it performs uniform thickness detection and overlap detection on the virtual container simulation, and identifies the defects of the container based on the detection results.

[0034] Thirdly, this application provides an electronic device including a processor and a memory, the memory storing computer-readable instructions, which, when executed by the processor, perform the steps of the method described above.

[0035] Fourthly, this application provides a storage medium having a computer program stored thereon, which, when executed by a processor, performs the steps of the method described above.

[0036] The beneficial effects of this invention are as follows: First, this application obtains multiple parts to be inspected based on the seven-item inspection method for containers, and classifies these parts. Based on the classification results, colored and colorless inspection parts are obtained. An external contour model is built based on the external dimension data of the container. Then, machine vision is used to detect defects in the multiple classified parts to be inspected. The advantage of this is that by obtaining multiple parts to be inspected based on the seven-item inspection method for containers, it can be ensured that the parts analyzed later are those that may have defects inside the container, thereby avoiding invalid detection and missed detection. By building an external contour model, a model foundation can be provided for the subsequent virtual container model, making the identification of container defects clearer and more accurate. By performing defect detection on the parts to be inspected after classification, during image analysis, the influence of color on different parts to be inspected can be differentiated, thereby ensuring that the image analysis is more in line with the actual detection standards of each part to be inspected, making the detection results more accurate.

[0037] This application also performs a first filling of the external contour model based on the defect detection results; the external contour model after the first filling is recorded as the part filling model; finally, the interior of the container is scanned using LiDAR, and the part filling model is filled a second time based on the scan data to obtain a virtual container model; the defects of the container are marked based on the virtual container model. The advantage of this is that by obtaining a virtual container model through the first and second filling, and marking the defects of the container based on the virtual container model, all defects in the container obtained through detection in this application can be accurately marked, which helps to improve the maintenance efficiency of management personnel and avoid potential problems in the actual use of the container. Attached Figure Description

[0038] Figure 1 This is a schematic diagram of the system of the present invention;

[0039] Figure 2 This is a flowchart illustrating the steps of the method of the present invention;

[0040] Figure 3 This is a schematic diagram of the colored detection profile of the present invention;

[0041] Figure 4 This is a schematic diagram of the electronic device of the present invention. Detailed Implementation

[0042] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0043] Example 1, please refer to Figure 1 As shown, this application provides a machine vision-based container inspection system, including an external inspection and analysis module, a defect filling module, and a virtual model comparison module;

[0044] The external inspection and analysis module is used to obtain multiple parts to be inspected based on the seven-item inspection method for containers, classify the multiple parts to be inspected, and obtain colored and colorless inspection parts based on the classification results; and build an external contour model based on the external dimension data of the container.

[0045] The external detection and analysis module includes a part classification and contour construction unit, which in turn includes part classification and contour construction strategies. These strategies include:

[0046] Based on the seven-item inspection method for containers, all inspection parts of the container in the seven-item inspection method are obtained and recorded as the parts to be inspected; for any part to be inspected: the inspection description of the part to be inspected in the seven-item inspection method for containers is recorded as the inspection identification data of the part to be inspected; the inspection identification data of all parts to be inspected are obtained and word segmentation is performed on all inspection identification data.

[0047] All color names are obtained based on big data and stored in the color database. For any detection and recognition data: when there is a word in the detection and recognition data that is the same as any word in the color database, the part to be detected corresponding to the detection and recognition data is recorded as a colored detection part; when all words in the detection and recognition data are different from all words in the color database, the part to be detected corresponding to the detection and recognition data is recorded as a colorless detection part.

[0048] In the specific implementation process, for example, during a data processing, if the detection and identification data of a part to be detected, "rivet", is "there must be no colored filler around the rivet", it means that the detection direction of the "rivet" is related to the color when it is detected. In this case, the "rivet" can be recorded as a colored detection part and colored defect detection can be performed in subsequent analysis.

[0049] The external dimensions of the container are acquired using machine vision. The external dimensions include the length and width of each side of the container. A spatial rectangular coordinate system with units of meters for the X, Y, and Z axes is constructed and denoted as the detection coordinate system. Based on the external dimensions of the container, a rectangular body is constructed within the detection coordinate system and denoted as the external contour model.

[0050] Align any vertex of the external contour model with the origin of the coordinate system, and align the front view, left view, and top view of the external contour model with the XZ plane, YZ plane, and XY plane, respectively. The vertex that coincides with the origin of the coordinate system is designated as the feature vertex. In practice, adjusting the position of the external contour model helps to make the overall model more regular after marking the defective parts, and also makes it easier for staff to find the defective parts, thus improving the efficiency of defect repair.

[0051] The defect filling module is used to perform defect detection on multiple classified parts to be detected based on machine vision, and to fill the outer contour model once based on the defect detection results; the outer contour model after one filling is denoted as the part filling model;

[0052] The defect filling module includes a defect detection and location filling unit, which in turn includes defect detection and location filling strategies. These strategies include:

[0053] Defect detection includes both colored defect detection and colorless defect detection. Colored defect detection includes:

[0054] For any colored detection area, a camera is used to capture the colored detection area, and the resulting image is recorded as a colored detection image; the outline of the colored detection area in the colored detection image is recorded as a colored detection outline; the colors within the colored detection image are identified, and based on the identification results, the boundary lines between different colors within the colored detection image are obtained and recorded as colored boundary lines; the colored boundary lines are marked within the colored detection outline.

[0055] In specific implementation, for example, the detection process for a colored detection area during a data processing session is as follows: Figure 3 As shown, FF1 is the colored detection image, FF2 is the colored detection contour, and YF1 to YF3 are the colored boundary lines in the colored detection image. By obtaining the colored boundary lines and coloring the colored detection contour, the colored detection area in the standard state after color division can be obtained, which helps to identify and detect the color distribution in the colored detection area.

[0056] The detection standards corresponding to the colored detection areas are obtained based on the seven detection methods, and the colored detection outlines are colored based on the detection standards. When the boundary line between any two colored areas and any colored boundary line do not coincide for any length during the coloring process, the colored detection area is recorded as a colored defect area.

[0057] When the dividing lines between all different colors coincide with all colored dividing lines during the coloring process, the colored detection outlines and colored detection images are superimposed. If the color of the colored detection outline in any overlapping area is different from that in the colored detection image, the colored detection area is recorded as a colored defect area. If the color of the colored detection outline in all overlapping areas is the same as that in the colored detection image, the colored detection area is recorded as a qualified area.

[0058] Colorless defect detection includes: for any colorless detection area, using a camera to capture the colorless detection area and recording the resulting image as a colorless detection image; recording the area of ​​the colorless detection area in the colorless detection image as a colorless detection region; obtaining the colorless detection region corresponding to the flawless colorless detection area based on the seven-item detection method and recording it as a colorless comparison region.

[0059] For any pixel within the colorless detection area: obtain the gray value of the pixel and the gray values ​​of all pixels in its eight neighboring regions; obtain the difference between the gray value of the pixel and the gray values ​​of all pixels in its eight neighboring regions, and record the largest difference as the neighbor difference value; obtain the neighbor difference values ​​corresponding to all pixels.

[0060] In a specific implementation process, for example, during a data processing session, if the grayscale value of a pixel is 100, and the grayscale values ​​of all pixels in the eight neighboring regions are 130, 120, 100, 95, 21, 120, 170, and 110 respectively, then through analysis, the maximum difference is found to be 79, i.e., the neighbor difference value is 79. By obtaining the neighbor difference value and subsequently analyzing to obtain the maximum neighbor difference value, we can identify areas with large color differences in the colorless detection area, thereby performing targeted feature analysis and improving the accuracy of detection.

[0061] The mode of all neighbor differences is recorded as the maximum neighbor difference value; all pixels with the maximum neighbor difference value are marked as feature pixels within the colorless detection region; for any feature pixel α: the distance between feature pixel α and the feature pixel farthest from feature pixel α is recorded as the limiting feature parameter of feature pixel α; the limiting feature parameters of all feature pixels are obtained, and the feature algorithm is used to obtain the detection parameters of the colorless detection region. The feature algorithm is as follows: Where F is the detection parameter, G i G represents the limiting feature parameter of the i-th feature pixel among all feature pixels. sq is the average of all limiting feature parameters, c is the number of feature pixels, g is the number of neighboring feature pixels, and neighboring feature pixels are feature pixels that have feature pixels in their eight neighborhoods.

[0062] In a specific implementation process, for example, during a data processing session, the number of feature pixels is 7, the number of neighboring feature pixels is 14, the average value of all extreme feature parameters is 90, and the differences between the extreme feature parameters of all feature pixels and 90 are 20, 10, 10, 20, 5, 1, and 4, respectively. Therefore, the detection parameter can be calculated to be 20. By obtaining the detection parameter, the features of areas with large color differences in the colorless comparison area can be obtained, ensuring that there are no detection omissions during detection.

[0063] When the detection parameters of the colorless comparison area are the same as those of the colorless detection area, the colorless detection area is recorded as a qualified area; when the detection parameters of the colorless comparison area are different from those of the colorless detection area, the colorless detection area is recorded as a defective area.

[0064] One-time filling is as follows: obtain the position of all parts to be detected in the container, and mark the position of all parts to be detected in the outer contour model based on the positional relationship between all parts to be detected and the feature apex; based on the positional relationship between the parts to be detected and the outer contour model and the actual position of each part to be detected, the parts to be detected that are inside the outer contour model and in contact with the inner wall of the container are recorded as internal positions.

[0065] The external contour model at this time is denoted as the part filling model;

[0066] In the specific implementation process, by obtaining the part filling model and then obtaining the virtual container model in the subsequent analysis, all defects in the container obtained through detection can be accurately identified, which helps to improve the maintenance efficiency of managers and avoid potential problems in the actual use of the container.

[0067] The virtual model comparison module is used to scan the interior of the container using LiDAR, and to perform secondary filling of the part filling model based on the scan data to obtain a virtual container model; it performs uniform thickness detection and overlap detection on the virtual container simulation, and identifies the defects of the container based on the detection results.

[0068] The virtual model comparison module includes a defect detection and labeling unit, which is configured with a defect detection and labeling strategy. The defect detection and labeling strategy includes:

[0069] The interior of the container is scanned using a LiDAR scanner, and the space corresponding to the scanned data is recorded as the actual internal space. Based on the positional relationship between the actual internal space and the feature apex, the actual internal space is placed inside the part-filling model; the part-filling model at this time is recorded as the virtual container model.

[0070] When any internal part does not fit with the actual internal space, the internal part is recorded as a defective part and the container is recorded as a misaligned container; when all internal parts fit with the actual internal space, the container is recorded as a container with intact space.

[0071] In the specific implementation process, when any internal position does not fit with the actual internal space, it indicates that there is a gap between the internal position and the inside of the container, that is, the container body may be misaligned. Therefore, it should be recorded as a space misalignment container to remind the staff.

[0072] Mark all defective parts within the virtual container model.

[0073] Example 2, please refer to Figure 2 As shown, this application also provides a machine vision-based container inspection method, including the following steps:

[0074] Step S1: Based on the seven-item inspection method for containers, obtain multiple parts to be inspected, classify the multiple parts to be inspected, and obtain colored and colorless inspection parts based on the classification results; build an external contour model based on the external dimension data of the container.

[0075] Step S1 includes: Step S101, obtaining all inspection parts of the container in the seven-item inspection method and recording them as parts to be inspected; for any part to be inspected: recording the inspection description of the part to be inspected in the seven-item inspection method as the inspection identification data of the part to be inspected; obtaining the inspection identification data of all parts to be inspected and performing word segmentation processing on all inspection identification data.

[0076] Step S102: Obtain all color names based on big data and store them in the color database; for any detection and recognition data: when there is a word in the detection and recognition data that is the same as any word in the color database, the part to be detected corresponding to the detection and recognition data is recorded as a colored detection part; when all words in the detection and recognition data are different from all words in the color database, the part to be detected corresponding to the detection and recognition data is recorded as a colorless detection part.

[0077] Step S103: Obtain the external dimension data of the container based on machine vision. The external dimension data includes the length and width data of each side of the container. Construct a spatial rectangular coordinate system with the X, Y and Z axes in meters and denote it as the detection coordinate system. Construct a rectangular body in the detection coordinate system based on the external dimension data of the container and denote it as the external contour model.

[0078] Step S104: Align any vertex of the external contour model with the origin of the coordinate system, and align the front view, left view, and top view of the external contour model with the XZ plane, YZ plane, and XY plane, respectively. The vertex that coincides with the origin of the coordinate system is denoted as the feature vertex.

[0079] Step S2: Based on machine vision, perform defect detection on multiple classified parts to be detected, and fill the outer contour model once based on the defect detection results; the outer contour model after filling once is recorded as the part filling model;

[0080] Defect detection includes colored defect detection and colorless defect detection. Step S201, colored defect detection includes:

[0081] Step S2011: For any colored detection area, use a camera to capture the colored detection area and record the resulting image as a colored detection image; record the outline of the colored detection area in the colored detection image as a colored detection outline; identify the colors in the colored detection image and obtain the boundary lines between different colors in the colored detection image based on the identification results, and record them as colored boundary lines; mark the colored boundary lines within the colored detection outline;

[0082] Step S2012: Obtain the detection standard corresponding to the colored detection area based on the seven detection methods, and color the colored detection outline based on the detection standard; when the boundary line between any two colors and any colored boundary line do not coincide for any length during the coloring process, the colored detection area is recorded as a colored defect area.

[0083] Step S2013: When the dividing lines between all different colors coincide with all colored dividing lines during the coloring process; the colored detection outline and the colored detection image are overlapped. When the color of the colored detection outline in any overlapping area is different from the color of the colored detection image, the colored detection area is recorded as a colored defect area; when the color of the colored detection outline and the colored detection image are the same in all overlapping areas, the colored detection area is recorded as a qualified area.

[0084] Step S202, colorless defect detection includes: Step S2021, for any colorless detection area, use a camera to take a picture of the colorless detection area and record the obtained image as a colorless detection image; record the area of ​​the colorless detection area in the colorless detection image as a colorless detection area; obtain the colorless detection area corresponding to the colorless detection area of ​​the defect-free colorless detection area based on the seven-item detection method, and record it as a colorless comparison area;

[0085] Step S2022: For any pixel within the colorless detection area: obtain the gray value of the pixel and the gray values ​​of all pixels in the pixel's eight neighboring regions; obtain the difference between the gray value of the pixel and the gray values ​​of all pixels in the eight neighboring regions, and record the largest difference as the neighbor difference value; obtain the neighbor difference values ​​corresponding to all pixels.

[0086] Step S2023: Record the mode of all neighbor differences as the maximum neighbor difference value; mark all pixels with the maximum neighbor difference value as feature pixels within the colorless detection region; for any feature pixel α: record the distance between feature pixel α and the feature pixel farthest from feature pixel α as the limiting feature parameter of feature pixel α; obtain the limiting feature parameters of all feature pixels, and use the feature algorithm to obtain the detection parameters of the colorless detection region. The feature algorithm is as follows: Where F is the detection parameter, G i G represents the limiting feature parameter of the i-th feature pixel among all feature pixels. sq is the average of all limiting feature parameters, c is the number of feature pixels, g is the number of neighboring feature pixels, and neighboring feature pixels are feature pixels that have feature pixels in their eight neighborhoods.

[0087] Step S2024: When the detection parameters of the colorless comparison area are the same as those of the colorless detection area, the colorless detection area is recorded as a qualified area; when the detection parameters of the colorless comparison area are different from those of the colorless detection area, the colorless detection area is recorded as a defective area.

[0088] Step S2 also includes: Step S203, one filling is: obtaining the position of all the parts to be detected in the container, and marking the position of all the parts to be detected in the outer contour model based on the positional relationship between all the parts to be detected and the feature apex; based on the positional relationship between the parts to be detected and the outer contour model and the actual position of each part to be detected, the parts to be detected that are inside the outer contour model and are in contact with the inner wall of the container are recorded as the inner position.

[0089] Step S204: Record the external contour model at this time as the part filling model.

[0090] Step S3: Use LiDAR to scan the inside of the container, and perform secondary filling of the part filling model based on the scan data to obtain a virtual container model; mark the defects of the container based on the virtual container model;

[0091] Step S3 includes: Step S301, using a lidar to scan the inside of the container, and recording the space corresponding to the scan data as the actual internal space, and placing the actual internal space inside the part filling model based on the positional relationship between the actual internal space and the feature apex; and recording the part filling model at this time as the virtual container model.

[0092] Step S302: When any internal part does not fit with the actual internal space, the internal part is recorded as a part with fitting defects, and the container is recorded as a container with spatial misalignment; when all internal parts fit with the actual internal space, the container is recorded as a container with intact space.

[0093] Step S303: Mark all defective parts within the virtual container model.

[0094] Example 3, please refer to Figure 4 As shown, Figure 4 A schematic diagram of an electronic device is provided, which may include a processor, a communication interface, a memory, and a communication bus. The processor, communication interface, and memory communicate with each other via the communication bus. The memory stores computer-readable instructions, and the processor can call the instructions in the memory. When the computer-readable instructions are executed by the processor, steps such as those in a machine vision-based container inspection method are performed to achieve the following functions: First, multiple parts to be inspected are obtained based on the seven-item container inspection method, and these parts are classified. Based on the classification results, colored and colorless inspection parts are obtained. An external contour model is built based on the external dimension data of the container. Then, defects are detected on the multiple classified parts based on machine vision, and the external contour model is filled once based on the defect detection results. The external contour model after the first filling is recorded as the part filling model. Finally, the interior of the container is scanned using a LiDAR scanner, and the part filling model is filled a second time based on the scan data to obtain a virtual container model. Defects in the container are identified based on the virtual container model.

[0095] Furthermore, the logical instructions in the aforementioned memory can be implemented as software functional units and sold or used as independent products, and can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0096] Example 4: This application also provides a computer-readable storage medium storing a computer program. When the computer program is executed by a processor, it performs the steps of the above-mentioned machine vision-based container inspection method to achieve the following functions: First, multiple parts to be inspected are obtained based on the seven-item container inspection method, and these parts are classified. Based on the classification results, colored and colorless inspection parts are obtained. An external contour model is built based on the external dimension data of the container. Then, defects are detected on the multiple classified parts based on machine vision, and the external contour model is filled once based on the defect detection results. The external contour model after the first filling is recorded as the part filling model. Finally, the interior of the container is scanned using a lidar, and the part filling model is filled a second time based on the scan data to obtain a virtual container model. Defects in the container are identified based on the virtual container model.

[0097] Based on the above description of the embodiments, the embodiments of the present invention can be provided as methods, systems, or computer program products. Based on this understanding, the technical solutions described above, or the parts that contribute to the prior art, can be embodied in the form of software products. These computer software products can be stored in computer-readable storage media, such as ROM / RAM, magnetic disks, optical disks, etc., and include several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods described in the various embodiments or certain parts of the embodiments.

[0098] In the embodiments provided in this application, it should be understood that the disclosed system or method can be implemented in other ways. The embodiments described above are merely illustrative. For example, the division of modules or units is only a logical functional division, and there may be other division methods in actual implementation. Furthermore, multiple modules or units may be combined or integrated into another system, or some features may be ignored or not executed. Additionally, the coupling or direct coupling or communication connection shown or discussed may be through some communication interfaces. The indirect coupling or communication connection between systems, modules, and units may be electrical, mechanical, or other forms.

[0099] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application.

Claims

1. A machine vision-based container inspection method, characterized in that, Includes the following steps: Multiple inspection parts are obtained based on the seven-item inspection method for containers, and these parts are classified. Colored and colorless inspection parts are obtained based on the classification results. An external contour model is built based on the external dimension data of the container. Based on machine vision, defects are detected in multiple classified parts to be detected, and the external contour model is filled once based on the defect detection results; the external contour model after filling is denoted as the part filling model. The interior of the container is scanned using LiDAR, and the part filling model is then filled in a second time based on the scan data to obtain a virtual container model. Defects in containers are identified based on virtual container models; Defect detection includes both colored defect detection and colorless defect detection. Colored defect detection includes: For any colored detection area, a camera is used to capture the colored detection area, and the resulting image is recorded as a colored detection image; the outline of the colored detection area in the colored detection image is recorded as a colored detection outline; the colors within the colored detection image are identified, and based on the identification results, the boundary lines between different colors within the colored detection image are obtained and recorded as colored boundary lines; the colored boundary lines are marked within the colored detection outline. The detection standards corresponding to the colored detection areas are obtained based on the seven detection methods, and the colored detection outlines are colored based on the detection standards. When the boundary line between any two colored areas and any colored boundary line do not coincide for any length during the coloring process, the colored detection area is recorded as a colored defect area. When the dividing lines between all different colors coincide with all colored dividing lines during the coloring process; the colored detection outlines are superimposed on the colored detection images. When the color of the colored detection outline in any overlapping area is different from the color of the colored detection image, the colored detection area is recorded as a colored defect area; when the color of the colored detection outlines in all overlapping areas is the same as the color of the colored detection image, the colored detection area is recorded as a qualified area. Colorless defect detection includes: for any colorless detection area, using a camera to capture the colorless detection area and recording the resulting image as a colorless detection image; recording the area of ​​the colorless detection area in the colorless detection image as a colorless detection region; obtaining the colorless detection region corresponding to the flawless colorless detection area based on the seven-item detection method and recording it as a colorless comparison region. For any pixel within the colorless detection area: obtain the gray value of the pixel and the gray values ​​of all pixels in its eight neighboring regions; obtain the difference between the gray value of the pixel and the gray values ​​of all pixels in its eight neighboring regions, and record the largest difference as the neighbor difference value; obtain the neighbor difference values ​​corresponding to all pixels. The mode of all neighbor differences is recorded as the maximum neighbor difference value; all pixels with the maximum neighbor difference value are marked as feature pixels within the colorless detection region; for any feature pixel α: the distance between feature pixel α and the feature pixel farthest from feature pixel α is recorded as the limiting feature parameter of feature pixel α; the limiting feature parameters of all feature pixels are obtained, and the feature algorithm is used to obtain the detection parameters of the colorless detection region. The feature algorithm is as follows: Where F is the detection parameter, and G i G represents the limiting feature parameter of the i-th feature pixel among all feature pixels. sq is the average of all limiting feature parameters, c is the number of feature pixels, g is the number of neighboring feature pixels, and neighboring feature pixels are feature pixels that have feature pixels in their eight neighborhoods. When the detection parameters of the colorless comparison area are the same as those of the colorless detection area, the colorless detection area is recorded as a qualified area; when the detection parameters of the colorless comparison area are different from those of the colorless detection area, the colorless detection area is recorded as a defective area.

2. The machine vision-based container inspection method according to claim 1, characterized in that, Multiple areas to be detected are classified, and based on the classification results, colored and colorless detection areas are obtained, including: Based on the seven-item inspection method for containers, all inspection parts of the container in the seven-item inspection method are obtained and recorded as the parts to be inspected; for any part to be inspected: the inspection description of the part to be inspected in the seven-item inspection method for containers is recorded as the inspection identification data of the part to be inspected; the inspection identification data of all parts to be inspected are obtained and word segmentation is performed on all inspection identification data. All color names are obtained based on big data and stored in the color database. For any detection and recognition data: when there is a word in the detection and recognition data that is the same as any word in the color database, the part to be detected corresponding to the detection and recognition data is recorded as a colored detection part; when all words in the detection and recognition data are different from all words in the color database, the part to be detected corresponding to the detection and recognition data is recorded as a colorless detection part.

3. The machine vision-based container inspection method according to claim 2, characterized in that, Building an external contour model based on the external dimensions of a shipping container includes: The external dimensions of the container are acquired using machine vision. The external dimensions include the length and width of each side of the container. A spatial rectangular coordinate system with units of meters for the X, Y, and Z axes is constructed and denoted as the detection coordinate system. Based on the external dimensions of the container, a rectangular body is constructed within the detection coordinate system and denoted as the external contour model. Arrange any vertex of the external contour model to coincide with the origin of the coordinate system, and make the front view, left view and top view of the external contour model parallel to the XZ plane, YZ plane and XY plane respectively. The vertex that coincides with the origin of the coordinate system is called the feature vertex.

4. The machine vision-based container inspection method according to claim 1, characterized in that, The process of filling the outer contour model based on the defect detection results includes: One-time filling is as follows: obtain the position of all parts to be detected in the container, and mark the position of all parts to be detected in the outer contour model based on the positional relationship between all parts to be detected and the feature apex; based on the positional relationship between the parts to be detected and the outer contour model and the actual position of each part to be detected, the parts to be detected that are inside the outer contour model and in contact with the inner wall of the container are recorded as internal positions. The external contour model at this point is denoted as the part filling model.

5. The machine vision-based container inspection method according to claim 1, characterized in that, The interior of the container is scanned using LiDAR, and the part filling model is then filled in a second time based on the scan data to obtain a virtual container model. Identifying defects in containers based on virtual container models includes: The interior of the container is scanned using a LiDAR scanner, and the space corresponding to the scanned data is recorded as the actual interior space. Based on the positional relationship between the actual interior space and the feature apex, the actual interior space is placed inside the part filling model. The part filled in at this time is denoted as the virtual container model; When any internal part does not fit with the actual internal space, the internal part is recorded as a defective part and the container is recorded as a misaligned container; when all internal parts fit with the actual internal space, the container is recorded as a container with intact space. Mark all defective parts within the virtual container model.

6. A machine vision-based container inspection system, used to implement the machine vision-based container inspection method according to any one of claims 1-5, characterized in that, It includes an external detection and analysis module, a defect filling module, and a virtual model comparison module; The external inspection and analysis module is used to obtain multiple parts to be inspected based on the seven-item inspection method for containers, classify the multiple parts to be inspected, and obtain colored and colorless inspection parts based on the classification results; and build an external contour model based on the external dimension data of the container. The defect filling module is used to perform defect detection on multiple classified parts to be detected based on machine vision, and to fill the outer contour model once based on the defect detection results; the outer contour model after one filling is denoted as the part filling model; The virtual model comparison module is used to scan the interior of the container using LiDAR, and to perform secondary filling of the part filling model based on the scan data to obtain a virtual container model; it performs uniform thickness detection and overlap detection on the virtual container simulation, and identifies the defects of the container based on the detection results.

7. An electronic device, characterized in that, It includes a processor and a memory, the memory storing computer-readable instructions that, when executed by the processor, perform the steps of the method as described in any one of claims 1-5.

8. A storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it performs the steps of the method as described in any one of claims 1-5.

Images