Container detection method, system and equipment based on machine vision and storage medium

Abstract

The invention discloses a container detection method, system and equipment based on machine vision and a storage medium, and relates to the technical field of containers, and the method comprises the steps: obtaining a colored detection part and a colorless detection part based on a seven-item container inspection method; building an external contour model; performing primary filling on the external contour model based on machine vision; obtaining a virtual container model; identifying flaws of the container based on the virtual container model; the method is used for solving the problems that in an existing container detection method, when flaws exist in a container or flaws exist in a scanning blind area outside the container, the flaws cannot be detected in a targeted mode, marks of the flaws are fuzzy after the flaws are obtained, and the detection accuracy is poor. Therefore, the problems of influence on maintenance of managers and influence on use of containers are solved.

Description

Technical Field

[0001] The present invention relates to the technical field of container detection, and specifically to a container detection method, system, device and storage medium based on machine vision. Background Art

[0002] A container is a standardized transportation tool for loading packaged or unpackaged goods, facilitating mechanical equipment for loading and unloading; it is a key element in the modern logistics system and is widely used in various transportation modes such as sea transportation, land transportation, and air transportation; container detection refers to a series of inspections and tests on containers during the container transportation process to ensure that they meet safety standards and transportation requirements; the main purpose of container detection is to ensure the structural integrity, tightness, cleanliness of the container, and whether it meets specific transportation requirements.

[0003] Existing methods for container detection usually identify defects and damages on the container surface based on box surface scanning and real-scene pictures, and compare the identified defects with a standard model to obtain the detection result. Although this detection method can identify the defects on the outside of the container, when there are defects inside the container or in the areas of the container outside the scanning blind area, it is impossible to specifically detect the areas where the defects exist only through box surface scanning and real-scene pictures, and the marking of the defects is relatively vague after obtaining the defect areas, resulting in problems that affect the maintenance by management personnel and the use of the container. For example, in the patent application with the publication number CN115937185A, a container detection system and method are disclosed. This solution obtains the real-scene pictures, point cloud files, and container information of the container surface, identifies the defects on the container surface through the real-scene pictures, and compares them with the defect-free unit reference model to obtain the defect results of each box surface. Other improvements in container detection usually focus on the detection of the container state and performance. There are still problems in the detection of container defects. When there are defects inside the container or in the areas of the container outside the scanning blind area, it is impossible to specifically detect the areas where the defects exist, and the marking of the defects is relatively vague after obtaining the defect areas, resulting in problems that affect the maintenance by management personnel and the use of the container. In view of this, it is necessary to improve the existing container detection methods. Summary of the Invention

[0004] The present invention aims to solve at least one of the technical problems in the prior art to some extent. By providing a container detection method, system, device and storage medium based on machine vision, it is used to solve the problems in the existing container detection methods. When there are defects inside the container or in the parts with scanning blind areas outside the container, it is impossible to detect the parts where the defects exist specifically, and the identification of the defects is relatively vague after obtaining the defective parts, resulting in problems that affect the maintenance of management personnel and the use of containers.

[0005] To achieve the above object, in the first aspect, the present application provides a container detection method based on machine vision, including the following steps:

[0006] Obtain multiple parts to be detected based on the seven-item inspection method for containers, classify the multiple parts to be detected, and obtain colored detection parts and colorless detection parts based on the classification results; build an external contour model based on the external dimension data of the container;

[0007] Perform defect detection on the multiple classified parts to be detected based on machine vision, and perform a first filling on the external contour model based on the defect detection results; record the external contour model after the first filling as the part filling model;

[0008] Use lidar to scan the inside of the container, perform a second filling on the part filling model based on the scan data, and obtain a virtual container model; identify the defects of the container based on the virtual container model.

[0009] Further, classifying the multiple parts to be detected and obtaining colored detection parts and colorless detection parts based on the classification results includes:

[0010] Obtain all the inspection parts of the container in the seven-item inspection method based on the seven-item inspection method for containers, and record them as the parts to be detected; for any one part to be detected: record the detection description of the part to be detected in the seven-item container detection method as the detection identification data of the part to be detected; obtain the detection identification data of all the parts to be detected, and perform word segmentation processing on all the detection identification data;

[0011] Obtain all color names based on big data and store them in the color database; for any one detection identification data: when there is a word in the detection identification data that is the same as any word in the color database, record the part to be detected corresponding to the detection identification data as a colored detection part; when all the words in the detection identification data are different from all the words in the color database, record the part to be detected corresponding to the detection identification data as a colorless detection part.

[0012] Further, building an external contour model based on the external dimension data of the container includes:

[0013] Obtain the external dimension data of the container based on machine vision, where the external dimension data includes the length data and width data of each side of the container; establish a spatial rectangular coordinate system with the units of the X-axis, Y-axis, and Z-axis all being m, and denote it as the detection coordinate system; build a rectangular parallelepiped within the detection coordinate system based on the external dimension data of the container, and denote it as the external contour model.

[0014] Coincide any one vertex angle of the external contour model with the coordinate origin, and make the front view, left view, and top view of the external contour model parallel to the X-Z plane, Y-Z plane, and X-Y plane respectively, where the vertex angle coinciding with the coordinate origin is denoted as the characteristic vertex angle.

[0015] Furthermore, the defect detection includes colored defect detection and colorless defect detection. The colored defect detection includes:

[0016] For any colored detection part, use a camera to take a picture of the colored detection part, and denote the obtained image as the colored detection image; denote the contour of the colored detection part in the colored detection image as the colored detection contour; identify the colors in the colored detection image, and obtain the dividing line between the different colors in the colored detection image based on the identification result, denoted as the colored dividing line; mark the colored dividing line within the colored detection contour.

[0017] Obtain the detection standard corresponding to the colored detection part based on the seven-item detection method, and paint the colored detection contour based on the detection standard; when there is any length of non-coincidence between the dividing line of any two already painted colors and any colored dividing line during the painting process, denote the colored detection part as a colored defect part.

[0018] When all the dividing lines between the different colors coincide with all the colored dividing lines during the painting process; coincide the painted colored detection contour with the colored detection image. When the colors of the colored detection contour and the colored detection image in any one overlapping area are different, denote the colored detection part as a colored defect part; when the colors of the colored detection contour and the colored detection image in all overlapping areas are the same, denote the colored detection part as a qualified part.

[0019] Furthermore, the colorless defect detection includes: for any colorless detection part, use a camera to take a picture of the colorless detection part, and denote the obtained image as the colorless detection image; denote the area of the colorless detection part in the colorless detection image as the colorless detection area; obtain the colorless detection area corresponding to the colorless detection part without defects based on the seven-item detection method, and denote it as the colorless comparison area.

[0020] For any pixel point within the colorless detection area: Obtain the grayscale value of the pixel point and the grayscale values of all pixel points in the eight-neighborhood of the pixel point. Respectively obtain the differences between the grayscale value of the pixel point and the grayscale values of all pixel points in the eight-neighborhood, and record the largest difference as the neighborhood difference; Obtain the neighborhood differences corresponding to all pixel points;

[0021] Record the mode of all neighborhood differences as the maximum neighborhood difference; Mark the pixel points with all neighborhood differences being the maximum neighborhood difference as characteristic pixel points within the colorless detection area; For any characteristic pixel point α: Record the distance between the characteristic pixel point α and the characteristic pixel point farthest from the characteristic pixel point α as the limit characteristic parameter of the characteristic pixel point α; Obtain the limit characteristic parameters of all characteristic pixel points, and use a characteristic algorithm to obtain the detection parameter of the colorless detection area. The characteristic algorithm is: where F is the detection parameter, G i is the limit characteristic parameter of the i-th characteristic pixel point among all characteristic pixel points, G sq is the average value of all limit characteristic parameters, c is the number of characteristic pixel points, g is the number of neighboring characteristic pixel points, and a neighboring characteristic pixel point is a characteristic pixel point with a characteristic pixel point in its eight-neighborhood;

[0022] When the detection parameter of the colorless comparison area is the same as the detection parameter of the colorless detection area, record the colorless detection part as a qualified part; When the detection parameter of the colorless comparison area is different from the detection parameter of the colorless detection area, record the colorless detection part as a defective part.

[0023] Furthermore, a primary filling of the external contour model based on the defect detection result includes:

[0024] The primary filling is: Obtain the positions of all parts to be detected in the container, and mark the positions of all parts to be detected in the external contour model based on the positional relationship between all parts to be detected and the characteristic apex angles; Based on the positional relationship between the parts to be detected and the external contour model and the actual positions of each part to be detected, record the parts to be detected that are inside the external contour model and in contact with the inner wall of the container as internal parts;

[0025] Record the external contour model at this time as the part filling model.

[0026] Furthermore, use a lidar to scan the inside of the container, and perform a secondary filling on the part filling model based on the scan data and obtain a virtual container model; Marking the defects of the container based on the virtual container model includes:

[0027] Use lidar to scan the interior of the container, and record the space corresponding to the scan data as the actual interior space. Place the actual interior space inside the part filling model based on the positional relationship between the actual interior space and the characteristic apex angles. Denote the part filling model at this time as the virtual container model.

[0028] When any interior part does not fit the actual interior space, mark the interior part as a fitting defect part, and mark the container as a space misaligned container. When all interior parts fit the actual interior space, mark the container as a space intact container.

[0029] Mark all defect parts in the virtual container model.

[0030] In a second aspect, the present application also provides a container detection system based on machine vision, including an external detection and analysis module, a defect filling module, and a virtual model comparison module.

[0031] The external detection and analysis module is used to obtain multiple parts to be detected based on the seven-item inspection method for containers, classify the multiple parts to be detected, and obtain colored detection parts and colorless detection parts based on the classification results. Build an external contour model based on the external dimension data of the container.

[0032] The defect filling module is used to detect defects in the multiple classified parts to be detected based on machine vision, and perform a first filling on the external contour model based on the defect detection results. Denote the external contour model after the first filling as the part filling model.

[0033] The virtual model comparison module is used to scan the interior of the container using lidar, perform a second filling on the part filling model based on the scan data and obtain the virtual container model. Perform uniform thickness detection and overlap detection on the virtual container simulation, and identify the defects of the container based on the detection results.

[0034] In a third aspect, the present application provides an electronic device, including a processor and a memory. The memory stores computer-readable instructions, and when the computer-readable instructions are executed by the processor, the steps in the above method are run.

[0035] In a fourth aspect, the present application provides a storage medium, on which a computer program is stored. When the computer program is executed by a processor, the steps in the above method are run.

[0036] Advantages of the present invention: First, based on the seven-item inspection method for containers, multiple parts to be detected are obtained, and the multiple parts to be detected are classified. Based on the classification results, colored detection parts and colorless detection parts are obtained; an external contour model is built based on the external dimension data of the container; then, based on machine vision, defect detection is performed on the multiple classified parts to be detected. The advantage of this is that by obtaining multiple parts to be detected based on the seven-item inspection method for containers, it can be ensured that the parts in subsequent analysis are all parts where defects may exist in the container, thus avoiding ineffective detection and missed detection; by building an external contour model, a model basis can be provided for the subsequent virtual container model, making the identification of container defects clearer and more accurate; by performing defect detection on the parts to be detected after classification, during image analysis, differential analysis can be performed based on the degree of influence of different parts to be detected by color, so as to ensure that the image analysis better conforms to the actual detection standards of each part to be detected and make the detection results more accurate;

[0037] The present application also performs a first filling on the external contour model based on the defect detection results; the external contour model after the first filling is denoted as the part filling model; finally, a lidar is used to scan the interior of the container, and based on the scan data, a second filling is performed on the part filling model to obtain a virtual container model; the defects of the container are identified based on the virtual container model. The advantage of this is that by obtaining a virtual container model through the first filling and the second filling and identifying the defects of the container based on the virtual container model, all the defects in the container obtained through detection in the present application can be accurately identified, which helps to improve the maintenance efficiency of the management personnel and avoid potential hazards in the actual application of the container. Brief Description of the Drawings

[0038] Figure 1 It is the principle block diagram of the system of the present invention;

[0039] Figure 2 It is the step flow chart of the method of the present invention;

[0040] Figure 3 It is the schematic diagram of the colored detection contour of the present invention;

[0041] Figure 4 It is the structural schematic diagram of the electronic device of the present invention. Detailed Embodiments

[0042] Next, the technical solutions in the embodiments of the present invention will be clearly and completely described in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the protection scope of the present invention.

[0043] Example 1, please refer to Figure 1 As shown, the present application provides a container detection system based on machine vision, including an external detection and analysis module, a defect filling module, and a virtual model comparison module;

[0044] The external detection and analysis module is used to obtain multiple parts to be detected based on the seven-item inspection method for containers, classify the multiple parts to be detected, and obtain colored detection parts and colorless detection parts based on the classification results; build an external contour model based on the external dimension data of the container;

[0045] The external detection and analysis module includes a part classification and contour building unit, and the part classification and contour building unit includes a part classification and contour building strategy, and the part classification and contour building strategy includes:

[0046] Obtain all inspection parts of the container in the seven-item inspection method based on the seven-item inspection method for containers, and record them as parts to be detected; for any one part to be detected: record the detection description of the part to be detected in the seven-item detection method of the container as the detection identification data of the part to be detected; obtain the detection identification data of all parts to be detected, and perform word segmentation processing on all detection identification data;

[0047] Obtain all color names based on big data and store them in the color database; for any one detection identification data: when there is a word in the detection identification data that is the same as any word in the color database, record the part to be detected corresponding to the detection identification data as a colored detection part; when all words in the detection identification data are different from all words in the color database, record the part to be detected corresponding to the detection identification data as a colorless detection part;

[0048] In the specific implementation process, for example, in a data processing, the detection identification data of a part to be detected "rivet" is "there should be no colored filler around the rivet", which means that the detection direction of the "rivet" is related to color during detection, so the "rivet" can be recorded as a colored detection part and colored defect detection can be performed in subsequent analysis;

[0049] Obtain the external dimension data of the container based on machine vision, where the external dimension data includes the length data and width data of each side of the container; build a three-dimensional rectangular coordinate system with the units of the X-axis, Y-axis, and Z-axis all being m, and record it as the detection coordinate system; build a cuboid in the detection coordinate system based on the external dimension data of the container, and record it as the external contour model;

[0050] Coincide any vertex angle of the external contour model with the origin of coordinates, and make the front view, left view, and top view of the external contour model parallel to the X-Z plane, Y-Z plane, and X-Y plane respectively. Among them, the vertex angle that coincides with the origin of coordinates is denoted as the characteristic vertex angle; in the specific implementation process, by adjusting the position of the external contour model, it helps to make the overall modeling more regular after marking the defective parts in the subsequent process, and at the same time facilitates the staff to find the defective parts and improve the defect repair efficiency of the staff.

[0051] The defect filling module is used to detect defects in multiple classified parts to be detected based on machine vision, and perform a first filling on the external contour model based on the defect detection results; the external contour model after the first filling is denoted as the part filling model;

[0052] The defect filling module includes a defect detection and part filling unit, and the defect detection and part filling unit includes a defect detection and part filling strategy, and the defect detection and part filling strategy includes:

[0053] Defect detection includes colored defect detection and colorless defect detection, and colored defect detection includes:

[0054] For any colored detection part, use a camera to take a picture of the colored detection part, and denote the obtained image as the colored detection image; denote the contour of the colored detection part in the colored detection image as the colored detection contour; identify the colors in the colored detection image, and based on the recognition results, obtain the dividing line between the different colors in the colored detection image, denoted as the colored dividing line; mark the colored dividing line within the colored detection contour;

[0055] In the specific implementation process, for example, the detection process of a colored detection part during a data processing is as Figure 3 shown, where FF1 is the colored detection image, FF2 is the colored detection contour, and YF1 to YF3 are the colored dividing lines in the colored detection image; by obtaining the colored dividing line and coloring the colored detection contour, the colored detection part in the standard state after color division can be obtained, which helps to identify and detect the color distribution in the colored detection part;

[0056] Obtain the detection standard corresponding to the colored detection part based on the seven-item detection method, and color the colored detection contour based on the detection standard; when there is any length of non-coincidence between the dividing line of any two already painted colors and any colored dividing line during the coloring process, the colored detection part is denoted as a colored defective part;

[0057] When the boundaries between all distinct colors during the coloring process coincide with all the colored boundaries; overlay the colored detection contour that has been colored with the colored detection image. When the colors of the colored detection contour and the colored detection image in any one of the overlapping regions are different, mark the colored detection part as a colored defect part; when the colors of the colored detection contour and the colored detection image in all the overlapping regions are the same, mark the colored detection part as a qualified part.

[0058] Colorless defect detection includes: For any colorless detection part, use a camera to take a picture of the colorless detection part, and record the obtained image as the colorless detection image; mark the area of the colorless detection part in the colorless detection image as the colorless detection area; based on the seven-item detection method, obtain the colorless detection area corresponding to the colorless detection part without defects, and record it as the colorless comparison area;

[0059] For any pixel point within the colorless detection area: Obtain the gray value of the pixel point and the gray values of all pixel points in the eight-neighborhood of the pixel point. Respectively obtain the differences between the gray value of the pixel point and the gray values of all pixel points in the eight-neighborhood, and record the largest difference as the neighborhood difference; obtain the neighborhood differences corresponding to all pixel points;

[0060] In a specific implementation process, for example, during a data processing, the gray value of the obtained pixel point is 100, and the gray values of all pixel points in the eight-neighborhood of the pixel point are 130, 120, 100, 95, 21, 120, 170, and 110 respectively. Then, through analysis, it can be obtained that the largest difference is 79, that is, the neighborhood difference is 79; by obtaining the neighborhood differences and then obtaining the maximum neighborhood difference through subsequent analysis, it is possible to obtain the areas with relatively large color differences in the colorless detection area, thereby conducting targeted feature analysis and improving the accuracy of detection;

[0061] Record the mode of all neighborhood differences as the maximum neighborhood difference; mark the pixel points with neighborhood differences equal to the maximum neighborhood difference in the colorless detection area as characteristic pixel points; for any characteristic pixel point α: Record the distance between the characteristic pixel point α and the characteristic pixel point farthest from the characteristic pixel point α as the limit characteristic parameter of the characteristic pixel point α; obtain the limit characteristic parameters of all characteristic pixel points, and use a characteristic algorithm to obtain the detection parameter of the colorless detection area. The characteristic algorithm is: where, F is the detection parameter, G i is the limit characteristic parameter of the i-th characteristic pixel point among all characteristic pixel points, G sq is the average value of all limit characteristic parameters, c is the number of characteristic pixel points, g is the number of neighboring characteristic pixel points, and a neighboring characteristic pixel point is a characteristic pixel point with a characteristic pixel point in its eight-neighborhood;

[0062] In the specific implementation process, for example, during a data processing operation, the number of feature pixels is 7, the number of adjacent feature pixels is 14, the average value of all extreme feature parameters is 90, and the differences between the extreme feature parameters of all feature pixels and 90 are 20, 10, 10, 20, 5, 1, and 4 respectively. Through calculation, the detection parameter is obtained as 20. By obtaining the detection parameter, the characteristics of the regions with significant color differences in the colorless comparison region can be acquired, ensuring that there are no oversights during the detection process.

[0063] When the detection parameter of the colorless comparison region is the same as that of the colorless detection region, the colorless detection part is marked as a qualified part; when the detection parameter of the colorless comparison region is different from that of the colorless detection region, the colorless detection part is marked as a defective part.

[0064] The first filling operation is as follows: Obtain the positions of all parts to be detected in the container, and mark the positions of all parts to be detected in the external contour model based on the positional relationship between all parts to be detected and the characteristic apex angles; Based on the positional relationship between the parts to be detected and the external contour model and the actual positions of each part to be detected, mark the parts to be detected that are inside the external contour model and in contact with the inner wall of the container as internal parts.

[0065] Mark the external contour model at this time as the part filling model.

[0066] In the specific implementation process, by obtaining the part filling model and acquiring the virtual container model during subsequent analysis, all defects in the container obtained through detection can be accurately identified, which helps improve the maintenance efficiency of management personnel and avoid potential hazards during the actual application of the container.

[0067] The virtual model comparison module is used to scan the interior of the container using lidar, perform secondary filling on the part filling model based on the scan data, and obtain the virtual container model; perform uniform thickness detection and overlap detection on the virtual container simulation, and mark the defects of the container based on the detection results.

[0068] The virtual model comparison module includes a defect detection and marking unit, and the defect detection and marking unit is configured with a defect detection and marking strategy, which includes:

[0069] Scan the interior of the container using lidar, and mark the space corresponding to the scan data as the actual internal space. Place the actual internal space inside the part filling model based on the positional relationship between the actual internal space and the characteristic apex angles; Mark the part filling model at this time as the virtual container model.

[0070] When any internal part does not fit the actual internal space, mark the internal part as a fitting defect part and mark the container as a space misaligned container; when all internal parts fit the actual internal space, mark the container as a space intact container.

[0071] In the specific implementation process, when any internal part does not fit the actual internal space, it indicates that there is a gap between the internal part and the inside of the container, that is, the container body may be misaligned. Therefore, it should be marked as a space misaligned container to remind the staff.

[0072] Mark all defect parts in the virtual container model.

[0073] Embodiment 2, please refer to Figure 2 As shown, the present application also provides a container detection method based on machine vision, including the following steps:

[0074] Step S1, obtain multiple parts to be detected based on the seven-item inspection method for containers, classify the multiple parts to be detected, and obtain colored detection parts and colorless detection parts based on the classification results; build an external contour model based on the external dimension data of the container.

[0075] Step S1 includes: Step S101, obtain all inspection parts of the container in the seven-item inspection method based on the seven-item inspection method for containers, and mark them as parts to be detected; for any part to be detected: mark the detection description of the part to be detected in the seven-item detection method for containers as the detection identification data of the part to be detected; obtain the detection identification data of all parts to be detected, and perform word segmentation processing on all detection identification data.

[0076] Step S102, obtain all color names based on big data and store them in the color database; for any detection identification data: when there is a word in the detection identification data that is the same as any word in the color database, mark the part to be detected corresponding to the detection identification data as a colored detection part; when all words in the detection identification data are different from all words in the color database, mark the part to be detected corresponding to the detection identification data as a colorless detection part.

[0077] Step S103, obtain the external dimension data of the container based on machine vision, where the external dimension data includes the length data and width data of each side of the outside of the container; build a three-dimensional rectangular coordinate system with the units of the X-axis, Y-axis, and Z-axis all being m, and mark it as the detection coordinate system; build a rectangular parallelepiped in the detection coordinate system based on the external dimension data of the container, and mark it as the external contour model.

[0078] Step S104: Coincide any vertex angle of the external contour model with the origin of coordinates, and make the front view, left view, and top view of the external contour model parallel to the X-Z plane, Y-Z plane, and X-Y plane respectively. Herein, the vertex angle that coincides with the origin of coordinates is denoted as the characteristic vertex angle.

[0079] Step S2: Based on machine vision, perform defect detection on multiple classified parts to be detected, and perform a first filling on the external contour model based on the defect detection results; denote the external contour model after the first filling as the part filling model.

[0080] The defect detection includes colored defect detection and colorless defect detection. Step S201, the colored defect detection includes:

[0081] Step S2011: For any colored detection part, use a camera to take a picture of the colored detection part, and denote the obtained image as the colored detection image; denote the contour of the colored detection part in the colored detection image as the colored detection contour; identify the colors in the colored detection image, and based on the identification results, obtain the dividing line between the different colors in the colored detection image, denoted as the colored dividing line; mark the colored dividing line within the colored detection contour.

[0082] Step S2012: Obtain the detection standard corresponding to the colored detection part based on the seven-item detection method, and paint the colored detection contour based on the detection standard; when there is any non-coincidence of any length between the dividing line of any two painted colors and any colored dividing line during the painting process, denote the colored detection part as a colored defect part.

[0083] Step S2013: When all the dividing lines between all different colors coincide with all the colored dividing lines during the painting process; coincide the painted colored detection contour with the colored detection image. When the colors of the colored detection contour and the colored detection image in any coincidence area are different, denote the colored detection part as a colored defect part; when the colors of the colored detection contour and the colored detection image in all coincidence areas are the same, denote the colored detection part as a qualified part.

[0084] Step S202, the colorless defect detection includes: Step S2021: For any colorless detection part, use a camera to take a picture of the colorless detection part, and denote the obtained image as the colorless detection image; denote the area of the colorless detection part in the colorless detection image as the colorless detection area; obtain the colorless detection area corresponding to the colorless detection part without defects based on the seven-item detection method, and denote it as the colorless comparison area.

[0085] Step S2022: For any pixel point within the colorless detection area, obtain the gray value of the pixel point and the gray values of all pixel points in the eight-neighborhood of the pixel point. Respectively obtain the differences between the gray value of the pixel point and the gray values of all pixel points in the eight-neighborhood, and record the maximum difference as the neighborhood difference. Obtain the neighborhood differences corresponding to all pixel points.

[0086] Step S2023: Denote the mode of all neighborhood differences as the maximum neighborhood difference. Mark the pixel points with neighborhood differences equal to the maximum neighborhood difference as characteristic pixel points within the colorless detection area. For any characteristic pixel point α, denote the distance between the characteristic pixel point α and the characteristic pixel point farthest from the characteristic pixel point α as the limit characteristic parameter of the characteristic pixel point α. Obtain the limit characteristic parameters of all characteristic pixel points, and use a characteristic algorithm to obtain the detection parameter of the colorless detection area. The characteristic algorithm is as follows: where F is the detection parameter, G i is the limit characteristic parameter of the i-th characteristic pixel point among all characteristic pixel points, G sq is the average value of all limit characteristic parameters, c is the number of characteristic pixel points, and g is the number of neighboring characteristic pixel points. A neighboring characteristic pixel point is a characteristic pixel point with a characteristic pixel point in its eight-neighborhood.

[0087] Step S2024: When the detection parameter of the colorless comparison area is the same as the detection parameter of the colorless detection area, mark the colorless detection part as a qualified part. When the detection parameter of the colorless comparison area is different from the detection parameter of the colorless detection area, mark the colorless detection part as a defective part.

[0088] Step S2 further includes: Step S203. The first filling: Obtain the positions of all parts to be detected in the container, and mark the positions of all parts to be detected in the external contour model based on the positional relationship between all parts to be detected and the characteristic apex angles. Based on the positional relationship between the parts to be detected and the external contour model and the actual positions of each part to be detected, mark the parts to be detected that are inside the external contour model and in contact with the inner wall of the container as inner parts.

[0089] Step S204: Denote the external contour model at this time as the part filling model.

[0090] Step S3: Use a lidar to scan the interior of the container, and perform secondary filling on the part filling model based on the scan data and obtain a virtual container model. Identify the defects of the container based on the virtual container model.

[0091] Step S3 includes: Step S301, scanning the interior of the container using a lidar, and denoting the space corresponding to the scanned data as the actual interior space. Place the actual interior space inside the part filling model based on the positional relationship between the actual interior space and the characteristic apex angles; denote the part filling model at this time as the virtual container model.

[0092] Step S302, when any interior part does not fit the actual interior space, denote the interior part as a fitting defect part and denote the container as a space misaligned container; when all interior parts fit the actual interior space, denote the container as a space intact container.

[0093] Step S303, mark all defect parts in the virtual container model.

[0094] Example 3, please refer to Figure 4 as shown Figure 4 illustrates a schematic structural diagram of an electronic device. The electronic device may include: a processor, a communication interface, a memory, and a communication bus. Among them, the processor, the communication interface, and the memory complete mutual communication through the communication bus. The memory stores computer-readable instructions. The processor can call the instructions in the memory. When the computer-readable instructions are executed by the processor, the steps in the container detection method based on machine vision are run to achieve the following functions: First, obtain multiple parts to be detected based on the seven-item inspection method for containers, and classify the multiple parts to be detected. Based on the classification results, obtain the colored parts to be detected and the colorless parts to be detected; build an external contour model based on the external dimension data of the container; then perform defect detection on the multiple classified parts to be detected based on machine vision, and perform a first filling on the external contour model based on the defect detection results; denote the external contour model after the first filling as the part filling model; finally, scan the interior of the container using a lidar, perform a second filling on the part filling model based on the scan data, and obtain a virtual container model; identify the defects of the container based on the virtual container model.

[0095] In addition, when the logical instructions in the above-mentioned memory are implemented in the form of software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on such an understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or a part of this technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the methods described in various embodiments of this application. The aforementioned storage medium includes: various media such as USB flash drives, mobile hard disks, read-only memories (ROM, Read-Only Memory), random access memories (RAM, Random Access Memory), magnetic disks, or optical discs that can store program codes.

[0096] Embodiment 4, this application also provides a computer-readable storage medium. This application provides a storage medium on which a computer program is stored. When the computer program is executed by a processor, it runs the steps in the above-mentioned container detection method based on machine vision to achieve the following functions: First, obtain multiple parts to be detected based on the seven-item inspection method for containers, and classify the multiple parts to be detected. Based on the classification results, obtain colored detection parts and colorless detection parts; build an external contour model based on the external dimension data of the container; then perform defect detection on the multiple classified parts to be detected based on machine vision, and perform a first filling on the external contour model based on the defect detection results; record the externally contour model after the first filling as the part filling model; finally, use lidar to scan the inside of the container, and perform a second filling on the part filling model based on the scan data and obtain a virtual container model; identify the defects of the container based on the virtual container model.

[0097] Through the description of the above embodiments, the embodiments of the present invention can be provided as a method, a system, or a computer program product. Based on such an understanding, the above technical solution, in essence, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disks, optical discs, etc., and includes several instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in various embodiments or certain parts of the embodiments.

[0098] In the embodiments provided in the present application, it should be understood that the disclosed system or method can be implemented in other ways. The embodiments described above are merely illustrative. For example, the division of modules or units is only a logical functional division, and there may be other division methods in actual implementation. For another example, multiple modules or units can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the displayed or discussed coupling or direct coupling or communication connection between each other can be through some communication interfaces. The indirect coupling or communication connection of systems, modules, and units can be electrical, mechanical, or other forms.

[0099] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present application and are not intended to limit them. Although the present application has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that they can still modify the technical solutions recorded in the foregoing embodiments or equivalently replace some of the technical features; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present application.

Claims

1. A container detection method based on machine vision, characterized in that, It includes the following steps: Based on the seven-item inspection method for containers, obtain multiple parts to be detected, classify the multiple parts to be detected, and based on the classification results, obtain colored detection parts and colorless detection parts; build an external contour model based on the external dimension data of the container; Based on machine vision, perform defect detection on the multiple classified parts to be detected, and based on the defect detection results, perform a first filling on the external contour model; denote the external contour model after the first filling as the part filling model; Use lidar to scan the interior of the container, and based on the scan data, perform a second filling on the part filling model and obtain a virtual container model; identify the defects of the container based on the virtual container model.

2. The container detection method based on machine vision according to claim 1, characterized in that Classify the multiple parts to be detected, and obtaining colored detection parts and colorless detection parts based on the classification results includes: Based on the seven-item inspection method for containers, obtain all the inspection parts of the container in the seven-item inspection method and denote them as parts to be detected; for any part to be detected: denote the detection description of the part to be detected in the seven-item inspection method for containers as the detection identification data of the part to be detected; obtain the detection identification data of all parts to be detected, and perform word segmentation on all the detection identification data; Obtain all color names based on big data and store them in a color database; for any detection identification data: when there is a word in the detection identification data that is the same as any word in the color database, denote the part to be detected corresponding to the detection identification data as a colored detection part; when all the words in the detection identification data are different from all the words in the color database, denote the part to be detected corresponding to the detection identification data as a colorless detection part.

3. The container detection method based on machine vision according to claim 2, characterized in that, Building an external contour model based on the external dimension data of the container includes: Obtain the external dimension data of the container based on machine vision, where the external dimension data includes the length data and width data of each side of the container exterior; build a three-dimensional rectangular coordinate system with the unit of m for the X-axis, Y-axis, and Z-axis, and denote it as the detection coordinate system; build a cuboid in the detection coordinate system based on the external dimension data of the container and denote it as the external contour model; Coincide any vertex angle of the external contour model with the coordinate origin, and make the front view, left view, and top view of the external contour model parallel to the X-Z plane, Y-Z plane, and X-Y plane respectively, where the vertex angle coinciding with the coordinate origin is denoted as the characteristic vertex angle.

4. The container detection method based on machine vision according to claim 3, characterized in that, Defect detection includes colored defect detection and colorless defect detection, and colored defect detection includes: For any colored detection part, use a camera to take a picture of the colored detection part, and denote the obtained image as the colored detection image; denote the contour of the colored detection part in the colored detection image as the colored detection contour; identify the colors in the colored detection image, and based on the identification results, obtain the dividing lines between the different colors in the colored detection image, denoted as the colored dividing lines; mark the colored dividing lines within the colored detection contour; Obtain the detection standard corresponding to the colored detection part based on the seven-item detection method, and paint the colored detection contour based on the detection standard; when there is any non-coincidence of any length between the boundary line of any two painted colors and any colored boundary line during the painting process, mark the colored detection part as a colored defective part; When the boundary lines between all distinct colors during the painting process coincide with all colored boundary lines; overlap the painted colored detection contour with the colored detection image. When the colors of the colored detection contour and the colored detection image in any overlapping area are different, mark the colored detection part as a colored defective part; when the colors of the colored detection contour and the colored detection image in all overlapping areas are the same, mark the colored detection part as a qualified part.

5. The container detection method based on machine vision according to claim 1, wherein, Colorless defect detection includes: For any colorless detection part, use a camera to take a picture of the colorless detection part, and record the obtained image as the colorless detection image; mark the area of the colorless detection part in the colorless detection image as the colorless detection area; obtain the colorless detection area corresponding to the flawless colorless detection part based on the seven-item detection method, and record it as the colorless comparison area; For any pixel point in the colorless detection area: Obtain the gray value of the pixel point and the gray values of all pixel points in the eight-neighborhood of the pixel point, respectively obtain the differences between the gray value of the pixel point and the gray values of all pixel points in the eight-neighborhood, and record the largest difference as the neighborhood difference; obtain the neighborhood differences corresponding to all pixel points; Denote the mode of all adjacent differences as the maximum adjacent difference; mark the pixel points with the maximum adjacent difference as characteristic pixel points within the colorless detection area; for any characteristic pixel point α: denote the distance between the characteristic pixel point α and the farthest characteristic pixel point from the characteristic pixel point α as the limit characteristic parameter of the characteristic pixel point α; obtain the limit characteristic parameters of all characteristic pixel points, and use a characteristic algorithm to obtain the detection parameters of the colorless detection area. The characteristic algorithm is as follows: where F is the detection parameter, G i is the limit characteristic parameter of the i-th characteristic pixel point among all characteristic pixel points, G sq is the average value of all limit characteristic parameters, c is the number of characteristic pixel points, and g is the number of adjacent characteristic pixel points. An adjacent characteristic pixel point is a characteristic pixel point with a characteristic pixel point in its eight-neighborhood; When the detection parameters of the colorless comparison area are the same as those of the colorless detection area, mark the colorless detection part as a qualified part; when the detection parameters of the colorless comparison area are different from those of the colorless detection area, mark the colorless detection part as a defective part.

6. The container detection method based on machine vision according to claim 1, wherein The first filling of the outer contour model based on the defect detection result includes: The first filling is: Obtain the positions of all parts to be detected in the container, and mark the positions of all parts to be detected in the outer contour model based on the positional relationship between all parts to be detected and the characteristic apex; based on the positional relationship between the part to be detected and the outer contour model and the actual position of each part to be detected, mark the part to be detected that is inside the outer contour model and in contact with the inner wall of the container as the inner part; Mark the outer contour model at this time as the part filling model.

7. The container detection method based on machine vision according to claim 1, characterized in that, Use lidar to scan the interior of the container, and perform secondary filling on the part filling model based on the scan data to obtain a virtual container model; Identifying the defects of the container based on the virtual container model includes: Use lidar to scan the interior of the container, and mark the space corresponding to the scan data as the actual interior space. Place the actual interior space inside the part filling model based on the positional relationship between the actual interior space and the characteristic apex; Mark the part filling model at this time as the virtual container model; When any inner part is not in contact with the actual interior space, mark the inner part as a fitting defective part, and mark the container as a space misaligned container; when all inner parts are in contact with the actual interior space, mark the container as a space intact container; Mark all defective parts in the virtual container model.

8. A container detection system based on machine vision is used to implement the container detection method based on machine vision according to any one of claims 1-7, characterized in that, It includes an external detection and analysis module, a defect filling module, and a virtual model comparison module; The external detection and analysis module is used to obtain multiple parts to be detected based on the seven-item inspection method for containers, classify the multiple parts to be detected, and obtain colored detection parts and colorless detection parts based on the classification results; build an external contour model based on the external dimension data of the container; The defect filling module is used to detect defects in the multiple classified parts to be detected based on machine vision, and perform a first filling on the external contour model based on the defect detection results; record the external contour model after the first filling as the part filling model; The virtual model comparison module is used to scan the interior of the container using lidar, perform a second filling on the part filling model based on the scan data, and obtain a virtual container model; perform uniform thickness detection and overlap detection on the virtual container simulation, and identify the defects of the container based on the detection results.

9. An electronic device, characterized in that, It includes a processor and a memory, and the memory stores computer-readable instructions. When the computer-readable instructions are executed by the processor, the steps in the method according to any one of claims 1-7 are run.

10. A storage medium, on which a computer program is stored, characterized in that, When the computer program is executed by the processor, the steps in the method according to any one of claims 1-7 are run.

Images