Geometric Calibration-Based Container Damage Quantification Method, System, Device, and Medium

Abstract

The present application provides a method, system, device and medium for quantifying container damage based on geometric calibration, which is applied to the technical field of container damage detection at docks. A geometric model of each component of the container is constructed based on the segmentation result of the image, and the relative positions and spatial layouts of each component of the container (such as parts like the container door, the front of the container, the sides and the top surface, etc.) are determined. Furthermore, the specific working conditions to which the container belongs are inferred, so as to establish a quantitative vector scale of each component of the container in the image under this working condition. Finally, the position of the container damage, the actual size of the damage box and other quantitative results are obtained by using the established geometric model and the quantitative vector scale, realizing the quantitative processing of container damage.

Description

Technical Field

[0001] This application relates to the technical field of terminal container damage detection, and specifically relates to a method, system, device, and medium for quantifying container damage based on geometric calibration. Background Art

[0002] During the long-distance transportation and frequent loading and unloading of containers, containers are often damaged to varying degrees due to external impacts, improper loading and unloading, or vibrations during long-distance transportation. These damages not only affect the service life of the containers but also pose a threat to the safety of the goods. Therefore, accurately quantifying the size of these damages is crucial for the maintenance and safe use of containers.

[0003] Some existing solutions directly predict the category and location information of container damage from images, such as CN112819793A, CN115187535A, CN115222697A, etc. Although the identification and classification of container damage can be achieved, the damage detection based on images is limited to predicting the category and approximate location of the damage, without considering the quantification of the damage size, which affects the reliability of maintenance and repair decisions. Some other existing technologies detect container damage based on images and point cloud data, such as CN110992337A and CN115965885A, etc. By using lidar scanning to obtain the three-dimensional structure information of the container, more accurate identification and quantification of the damage are achieved. However, it relies on using lidar scanning to obtain point cloud data, which increases the hardware and computational costs, maintenance difficulty, and system complexity of the container damage detection and quantification system. In some logistics and port environments, it is not feasible to deploy and use lidar. Summary of the Invention

[0004] In view of this, the present invention provides a method, system, device, and medium for quantifying container damage based on geometric calibration. By establishing a geometric model according to a two-dimensional image, it is convenient to establish a new quantization vector scale based on the known standard size of the actual container, so as to calculate quantization data such as the damage location and the size of the annotation box using the quantization scale.

[0005] The present invention provides the following technical solutions:

[0006] The present invention provides a method for quantifying container damage based on geometric calibration, including:

[0007] Based on the segmentation result of the container image, construct a geometric model of each component of the container in the container image, and the components include one or more of the following parts: container door, front of the container, side, top surface, corner fitting surface;

[0008] Determine the relative positions and spatial layouts of the respective components based on the corner fitting positions and segmentation results, and infer the specific working conditions of the container according to the relative positions and spatial layouts;

[0009] Establish a quantization vector scale for each of the respective components by combining the standard dimensions of different container types and corner fittings;

[0010] Judge the actual position of the container damage based on the geometric model and quantization vector scale established in the image; and calculate the actual size of the damaged frame based on the quantization vector scale established for the surface where the container damage is located.

[0011] Compared with the prior art, the beneficial effects that can be achieved by at least one of the above technical solutions adopted in the present invention at least include:

[0012] 1. The present invention realizes inferring the specific working conditions of the container based on a two-dimensional image through the established geometric model of each part of the container in the image, and establishes a quantization vector scale for the container door, the front of the container, the side surface and the top surface, realizing geometric calibration of the two-dimensional image;

[0013] 2. The present invention determines the position of the damage by combining the standard dimensions of the container and the segmentation result, and calculates the actual size of the damaged frame based on the established quantization vector scale;

[0014] 3. The present invention realizes the size quantization of the container damage only through two-dimensional images and image segmentation technology, avoiding expensive lidar equipment and complex data processing processes, simplifying the implementation of the damage quantization system, and reducing the hardware cost, maintenance difficulty and overall complexity of the system. Description of the Drawings

[0015] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the following will briefly introduce the drawings required for use in the embodiments. Obviously, the drawings described below are only some embodiments of the present application. For those of ordinary skill in the art, other drawings can be obtained based on these drawings without creative efforts.

[0016] Figure 1 is the overall schematic diagram of the container damage quantization processing based on geometric calibration in the present application;

[0017] Figure 2 is the flowchart of the container damage quantization method in the present application;

[0018] Figure 3 is the schematic diagram of calculating the actual size of the damaged frame based on the vector scale in the present application;

[0019] Figure 4 is the flowchart of a container damage quantization method in the present application;

[0020] Figure 5 is a schematic structural diagram of the container damage quantification system based on geometric calibration in the present application;

[0021] Figure 6 is a schematic structural diagram of the electronic device in the present application. Detailed implementation manners

[0022] The embodiments of the present application will be described in detail below with reference to the accompanying drawings.

[0023] The following uses specific specific examples to illustrate the implementation manners of the present application. Those skilled in the art can easily understand other advantages and effects of the present application from the content disclosed in this specification. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments. The present application can also be implemented or applied through other different specific implementation manners. Various details in this specification can also be modified or changed based on different viewpoints and applications without departing from the spirit of the present application. It should be noted that, without conflict, the following embodiments and the features in the embodiments can be combined with each other. All other embodiments obtained by those of ordinary skill in the art based on the embodiments in the present application without making creative efforts belong to the scope of protection of the present application.

[0024] It should be noted that the following describes various aspects of the embodiments within the scope of the appended claims. It should be obvious that the aspects described herein can be embodied in a wide variety of forms, and any specific structure and / or function described herein is illustrative only. Based on the present application, those skilled in the art should understand that one aspect described herein can be implemented independently of any other aspect, and two or more of these aspects can be combined in various ways. For example, any number and aspects described herein can be used to implement the device and / or practice the method. In addition, this device and / or this method can be implemented using other structures and / or functions in addition to one or more of the aspects described herein.

[0025] It should also be noted that the drawings provided in the following embodiments only illustrate the basic concept of the present application schematically. The drawings only show the components related to the present application and are not drawn according to the number, shape and size of the components in actual implementation. The type, quantity and proportion of each component in its actual implementation can be an arbitrary change, and the component layout type may also be more complex.

[0026] In addition, in the following description, specific details are provided to facilitate a thorough understanding of the examples. However, those skilled in the art will understand that the examples can be practiced without these specific details.

[0027] In the existing container damage detection and quantification technology, on the one hand, the solution that only predicts the category and position information of container damage based on images cannot provide the basic conditions required for quantification, resulting in the inability to carry out the quantification work of damage size. On the other hand, the solution based on image and point cloud data increases the hardware and computing costs, maintenance difficulty and system complexity of the container damage detection system, and has poor practical feasibility in the dock application scenario.

[0028] In view of this, the present invention proposes a container damage quantification method based on geometric calibration: as Figure 1 described, a geometric model of each part of the container is constructed based on the image segmentation result, and the actual specific working conditions of the container (such as the relative positions and spatial layouts of parts such as the container door, the front of the container, the sides, and the top surface) are introduced to establish a standardized quantification vector scale, so that the position and size of the container damage can be quantified by combining the geometric model and the quantification vector scale. For example, in size quantification, the size of the damage box is calculated using the base vector scale of the plane where it is located, and finally the quantification of the container damage can be realized.

[0029] The following will describe the technical solutions provided by the embodiments of the present application with reference to the accompanying drawings.

[0030] As Figure 2 shown, the present invention provides a container damage quantification method, which may include:

[0031] Step S202: Based on the segmentation result of the container image, construct a geometric model of each component part of the container in the container image, and each of the component parts includes one or more of the following parts: the container door, the front of the container, the sides, the top surface, and the corner fitting surface.

[0032] It should be noted that the imaging device used for image acquisition of the container is usually restricted by some factors such as the site and shooting requirements, and can only be fixedly deployed in certain places and shoot at a specific angle, etc. This is also the reason why the solution of using radar to form point cloud data does not have the actual deployment ability in practical applications. Therefore, if some calibration measures cannot be effectively adopted during processing, when quantifying damage based on two-dimensional images, there are no processing conditions, which is also the reason why the aforementioned prior art fails to provide quantification capabilities.

[0033] Therefore, the present invention first segments each component part that may be related to quantification, such as the container door, the front of the container, the sides, the top surface, the corner fitting surface, etc. from the collected container image data according to the relevant positions where damage may occur on the container and in combination with the quantification requirements, and constructs geometric models of these component parts. Thus, by converting the two-dimensional container image into a geometric model, it is convenient to perform geometric calibration transformation on the actually collected two-dimensional image based on the geometric model.

[0034] Step S204: Determine the relative positions and spatial layouts of the respective components according to the corner fitting positions and the segmentation results, and infer the specific working conditions of the container based on the relative positions and spatial layouts.

[0035] After establishing the geometric models of the respective components, the positional relationships, spatial layouts, etc. of the respective parts can be determined according to the corner fittings, and then the specific working conditions of the container, that is, the specific working conditions of the actual placement of the container in the two-dimensional image, can be inferred based on some prior knowledge.

[0036] In implementation, since the container corner fittings are located at the connection between the top surface and the side surface, in the present invention, the corner fittings are used as prior knowledge, which can be used to determine the relative positions, spatial layouts, etc. between these surfaces, and further infer the specific working conditions (i.e., placement postures) of the actual container in the two-dimensional image.

[0037] Step S206: Combine the standard dimensions of different container types and corner fittings to establish a quantization vector scale for each of the respective components in the image.

[0038] In view of the fact that due to limitations such as the acquisition angle of the collected two-dimensional image, each surface of the container body has different proportions, so the standard container can be combined, that is, the standard dimensions of different container types and corner fittings are used to establish a scale for the graphic container, which is convenient for subsequent processing to carry out quantization using this scale.

[0039] It should be noted that establishing a quantization vector scale can be said to be to predict a new scale (i.e., a new scale ratio) for this image for the two-dimensional image.

[0040] Step S208: Based on the geometric model and the quantization vector scale established in the image, determine the actual position of the container damage, and calculate the actual size of the damaged box based on the quantization vector scale where the container damage is located on the surface.

[0041] Based on the geometric model and the scale, the actual position, actual size, etc. of the container damage can all be quantized.

[0042] In some implementation manners, when establishing geometric models for each surface, the following provided preferred examples can be referred to, that is, further considering the real situation of these surfaces in the two-dimensional image, so that the constructed geometric model is more in line with the container situation in the actual two-dimensional image, which helps to improve the quantization accuracy.

[0043] For example, in establishing the geometric models of the container door and the front of the container, use corner point detection to find the four vertex coordinates of the container door and the front of the container that appear in the image, or the five vertex coordinates of an incomplete container door or the front of the container missing one corner, and sort their coordinates counterclockwise starting from the upper left corner, and construct the geometric models of the container door and the front of the container according to the sorting results.

[0044] For example, in constructing the geometric model of the top surface, when the top surface is an incomplete top of the container that only contains one end in the image, the four vertex coordinates of the partial top surface that only contains one end in the image are found using corner detection, and their coordinates are sorted counterclockwise starting from the upper left corner. According to the sorting result, the geometric model of the top surface is constructed.

[0045] For example, in constructing the geometric model of the side surface, when the side surface is a complete side surface or an incomplete side of the container that only contains one end in the image, the four vertex coordinates of the partial top surface that only contains one end in the image or the four vertex coordinates of the complete side surface are found using corner detection, and their coordinates are sorted counterclockwise starting from the upper left corner. According to the sorting result, the geometric model of the side surface is constructed.

[0046] For example, in constructing the geometric model of the corner fitting surface, when the corner fitting surfaces of the top surface and the side surface are the top surface or the side surface of the corner fitting on the top or side of the container in the image, image processing is used to find its minimum bounding rectangle to construct the geometric model of the corner fitting surface.

[0047] In some embodiments, in determining the working conditions, based on the examples provided below, the relative positions and spatial layouts of the components can also be made to better conform to the actual situation of the container in the two-dimensional image.

[0048] Therefore, when determining the relative positions and spatial layouts of the respective components, if the container image only contains a single container, the door, front, side, top surface, and corner fittings detected in the image are classified as the same container body; if the container image contains multiple containers, the door, front, side, top surface, and corner fittings that belong to the same container body are classified as the same container body using the center distances of the respective components.

[0049] In some embodiments, since the quantization vector scale is a new scale for quantifying the damage that appears in the two-dimensional image later, the preferred examples for establishing this scale can refer to the following respective embodiments.

[0050] For example, in establishing the quantization vector scale of the door and the front of the container, if the door and the front are complete, a pair of basis vectors are formed by the edges of the door or the front in the height and width directions, and the vector modulus lengths of the basis vectors are established according to the standard width and height of the actual container body to obtain the mapping relationship between pixel values; if the door and the front are incomplete with a corner missing, the two edges of the missing corner are extended to intersect to obtain the vertex that is occluded or not photographed in the image, and then a pair of basis vectors are formed by the edges of the door or the front in the height and width directions, and at the same time, the vector modulus lengths of the basis vectors are established according to the standard width and height of the actual container body to obtain the mapping relationship between pixel values.

[0051] By using the standard width and height of the actual standard box, a mapping relationship between the vector modulus in the height and width directions, i.e., pixel values, in the image is established, so that the new ruler is more consistent with the actual two-dimensional image reality.

[0052] For example, in establishing a quantitative vector scale for the side of a container, if the side of the container is complete, a pair of basis vectors are formed with the edges of the side of the container in the length and height directions, and the vector modulus of the basis vectors is established according to the standard length and height of the standard container, and the mapping relationship between pixel values ​​is obtained; if the side of the container only includes one end of the container, a pair of basis vectors are formed with the edges of the side of the container in the length and height directions, and then the actual length of the side of the container in the length direction is estimated using the standard length of the corner piece and the projection length of its vector in the length direction, and the height is set to the height of the actual container, and the vector modulus of the basis vector is established to obtain the mapping relationship between pixel values;

[0053] The vector modulus is set according to the actual situation of different box sides. For example, for a complete box side, the vector modulus and mapping relationship is established according to the standard box. If the box side is incomplete, projection in the length direction and the actual box height in the height direction are used to make the vector modulus more consistent with the actual situation of the box.

[0054] For example, in establishing a quantitative vector scale for the top surface of a container, a pair of basis vectors are formed with the edges in the height and width directions of the top of the box, and then the actual length of the top of the box in the height direction is estimated using the standard length of the corner piece and the projection length of its vector in the height direction. The width is set to the actual box width, and the vector modulus of the basis vector is established to obtain the mapping relationship between pixel values.

[0055] Similarly, setting the vector modulus for the top of the box can also better fit the actual situation of the box.

[0056] In some embodiments, when determining the damage location, an improved algorithm provided in the following example may be used. That is, based on the vertex coordinates of each part of the container and the center coordinates of the damage frame, a point-in-polygon algorithm is used to determine whether the damage is located on a certain container surface.

[0057] In some embodiments, in the quantization calculation, the calculation of the actual size of the damage frame includes: expressing the length and width of the damage frame with the basis vectors of the surface on which they are located to obtain the modulus of the length and width, and then calculating the actual size of the damage frame based on the vector modulus, that is, the mapping relationship between the pixel value and the actual size length of the basis vector.

[0058] refer to Figure 3 Schematically, the calculation formula can be expressed as:

[0059]

[0060]

[0061] Among them, is the damaged frame edge vector, is the base vector, , is the preset coefficient, , is the actual length of the base vector, is the included angle of the base vector, is the actual length of the calculated damaged frame edge.

[0062] In some embodiments, in establishing a geometric model using the segmentation result, the segmentation result can be obtained by segmenting the collected two-dimensional image using an existing segmentation model, or can be the segmentation model provided by the following example.

[0063] In implementation, the container image is segmented based on a preset segmentation model to form the segmentation result, and the pre-trained segmentation model is a model obtained through the following steps: After annotating the components of the container in the container image, a dataset required for instance segmentation is obtained, and the preset segmentation model is trained based on the dataset.

[0064] It should be noted that each segmented component can be a part that can reflect the posture, position, etc. of the container. For example, the container door, the front of the container, the top surface, the side surface, the top surface of the corner fitting, the side surface of the corner fitting, the spreader, etc.

[0065] It should be noted that the two-dimensional container image used in the present invention can be an image collected from the camera device of the quay crane.

[0066] Next, another example is listed. This example is a schematic example formed by combining the foregoing multiple examples.

[0067] Referring to Figure 4 for illustration, the present invention provides a method for quantifying container damage based on geometric calibration, which may include the following steps:

[0068] Step 1: Based on the result of image segmentation, construct a geometric model of the container door, the front of the container, the side surface, the top surface, and the top and side corner fitting surfaces existing in the image;

[0069] Step 2: According to the position of the corner fitting and the segmentation result, determine the relative positions and spatial layouts of the container door, the front of the container, the side surface, the top surface, and the top and side corner fitting surfaces, and infer the specific working conditions of the container;

[0070] Step 3: Combine the standard dimensions of different container types and corner fittings to establish quantization vector scales for the container door, the front of the container, the side surface, and the top surface respectively;

[0071] Step 4: Determine the position of the container damage based on the geometric model and the scale ruler established in the image;

[0072] Step 5: Calculate the actual size of the damaged box based on the quantization vector scale ruler established on the surface where the container damage is located.

[0073] Further, the method for constructing the geometric models of the container door and the front of the container in Step 1 is to use corner detection to find the four vertex coordinates of the container door and the front of the container that appear in the image, or the five vertex coordinates of an incomplete container door or the front of the container missing one corner, and sort their coordinates counterclockwise starting from the upper left corner.

[0074] Further, the top surface of the container in Step 1 is an incomplete container top that only contains one end of the container in the image. The method for constructing its geometric model is to use corner detection to find the four vertex coordinates of the partial container top that only contains one end in the image, and sort their coordinates counterclockwise starting from the upper left corner.

[0075] Further, the side surface of the container in Step 1 is a complete side surface or an incomplete container side that only contains one end of the container in the image. The method for constructing its geometric model is to use corner detection to find the four vertex coordinates of the partial container top that only contains one end or the four vertex coordinates of the complete container side in the image, and sort their coordinates counterclockwise starting from the upper left corner.

[0076] Further, the corner fitting surface of the top surface and the side surface of the container in Step 1 is the top surface or the side surface of the corner fitting on the top surface or the side surface of the container in the image. The method for constructing its geometric model is to use image processing to find its minimum circumscribed rectangle.

[0077] Further, the method for inferring the specific working condition of the container in Step 2 is to classify the container door, the front of the container, the side surface, the top surface, and the corner fittings detected in the image as one container body in the case of a single container. The relative positions of each part are known. In the case of a double-container, the container door, the front of the container, the side surface, the top surface, and the corner fittings belonging to the same container body are classified as one container body by using the detection results and the center distances of each part to determine the relative positions and spatial layouts of each part.

[0078] Further, the method for establishing the quantization vector scale of the container door and the front of the container in step 3 is as follows: When the container door and the front are complete, a pair of base vectors are formed by the edges in the height and width directions of the container door or the front. At the same time, a mapping relationship is established between the standard width and height of the actual container body and the vector modulus lengths in the height and width directions, that is, the pixel values. In the case where a corner of the container door or the front is missing and incomplete in the image, the two edges of the missing corner are extended to intersect to obtain the vertices that are blocked or not photographed in the image. Then, a pair of base vectors are formed by the edges in the height and width directions of the container door or the front. At the same time, a mapping relationship is established between the standard width and height of the actual container body and the vector modulus lengths in the height and width directions, that is, the pixel values.

[0079] Further, the method for establishing the quantization vector scale of the side of the container in step 3 is as follows: When the side of the container is complete, a pair of base vectors are formed by the edges in the length and height directions of the side of the container. At the same time, a mapping relationship is established between the standard length and height of the actual container body and the vector modulus lengths in the length and width directions, that is, the pixel values. In the case where only one end of the container is included in the image of the side of the container, a pair of base vectors are formed by the edges in the length and height directions of the side of the container. Then, the standard length of the corner fitting and its projected length in the vector in the length direction are used to estimate the actual length in the length direction of the side of the container, and the height is still the height of the actual container body. A mapping relationship is established between the vector modulus lengths in the length and width directions, that is, the pixel values.

[0080] Further, the method for establishing the quantization vector scale of the top surface of the container in step 3 is as follows: A pair of base vectors are formed by the edges in the height and width directions of the top of the container. Then, the standard length of the corner fitting and its projected length in the vector in the height direction are used to estimate the actual length in the height direction of the top of the container, and the width is still the width of the actual container body. A mapping relationship is established between the vector modulus lengths in the height and width directions, that is, the pixel values.

[0081] Further, the method for determining the damaged position of the container in step 4 is based on the vertex coordinates of each part of the container, combined with the center coordinates of the damaged box, and uses the algorithm of a point inside a polygon to determine whether the damage is located on a certain container surface.

[0082] Further, the method for calculating the actual size of the damaged box in step 5 is as follows: The length and width of the damaged box are represented by the base vectors of the surface where it is located to obtain the modulus lengths of the length and width. Then, based on the mapping relationship between the vector modulus length, that is, the pixel value, and the actual size length of the base vector, the actual size of the damaged box is calculated. Among them, the calculation formula can refer to the previous example and will not be elaborated here.

[0083] Based on the same inventive concept, the present invention also provides a container damage quantization system based on geometric calibration.

[0084] Reference Figure 5As shown, a container damage quantification system 100 based on geometric calibration may include:

[0085] Geometric model subsystem 101: Based on the segmentation result of the container image, construct a geometric model of each component of the container in the container image, and each of the components includes one or more of the following parts: container door, front of the container, side, top surface, corner fitting surface;

[0086] Inference subsystem 103: Determine the relative positions and spatial layouts of each of the components according to the corner fitting positions and the segmentation result, and infer the specific working conditions of the container according to the relative positions and spatial layouts;

[0087] Vector scale subsystem 105: Establish a quantization vector scale for each of the components by combining the standard dimensions of different container types and corner fittings respectively;

[0088] Calculation subsystem 107: Determine the position of the container damage based on the geometric model and scale established in the image; and calculate the actual size of the damaged frame based on the quantization vector scale established for the surface where the container damage is located.

[0089] It should be noted that the functions of the unit modules and the settings of the number of modules in the container damage quantification system can be set accordingly according to the foregoing method embodiments, and will not be elaborated here.

[0090] Based on the same inventive concept, the present invention also provides an electronic device, including: at least one processor; and a memory communicatively connected to the at least one processor; wherein, the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to execute: the geometric calibration-based container damage quantification method according to any one of the embodiments of the present application.

[0091] As Figure 6 shown, the present application also provides a schematic structural diagram of an electronic device. The structure of the electronic device 500 is shown in the figure. Here, the electronic device 500 is only an example and should not limit the functions and usage scope of the embodiments of the present invention.

[0092] In the electronic device 500, it may include: at least one processor 510; and a memory 520 communicatively connected to the at least one processor; wherein, the memory 520 stores instructions executable by the at least one processor 510, and the instructions are executed by the at least one processor 510 to enable the at least one processor 510 to execute: the geometric calibration-based container damage quantification method according to any one of the embodiments of the present application.

[0093] It should be noted that the electronic device 500 may be embodied in the form of a general-purpose computing device, for example, it may be a server device.

[0094] In implementation, the components of the electronic device 500 may include but are not limited to: at least one of the above-mentioned processors 510, at least one of the above-mentioned memories 520, and a bus 530 connecting different system components (including the memory 520 and the processor 510), where the bus 530 may include a data bus, an address bus, and a control bus.

[0095] In implementation, the memory 520 may include volatile memory, such as random access memory (RAM) 5201 and / or cache memory 5202, and may further include read-only memory (ROM) 5203.

[0096] The memory 520 may also include a program tool 5205 having a set (at least one) of program modules 5204. Such program modules 5204 include but are not limited to: an operating system, one or more application programs, other program modules, and program data. Each or some combination of these examples may include the implementation of a network environment.

[0097] The processor 510 executes various functional applications and data processing by running computer programs stored in the memory 520.

[0098] The electronic device 500 may also communicate with one or more external devices 540 (such as a keyboard, a pointing device, etc.). Such communication may be carried out through an input / output (I / O) interface 550. And, the electronic device 500 may also communicate with one or more networks (such as a local area network (LAN), a wide area network (WAN), and / or a public network, such as the Internet) through a network adapter 560. The network adapter 560 communicates with other modules in the electronic device 500 through the bus 530. It should be understood that although not shown in the figure, other hardware and / or software modules may be used in combination with the electronic device 500, including but not limited to: microcode, device drivers, redundant processors, external disk drive arrays, RAID (disk array) systems, tape drives, and data backup storage systems, etc.

[0099] In this specification, the same or similar parts among the various embodiments can be referred to each other, and each embodiment focuses on the differences from other embodiments. In particular, for the embodiments described later, the description is relatively simple, and the relevant parts can be referred to the partial description of the foregoing embodiments.

[0100] As described above, it is only the specific implementation manner of the present application. However, the protection scope of the present application is not limited thereto. Any changes or substitutions that can be easily conceived by those skilled in the art within the technical scope disclosed in the present application should be covered within the protection scope of the present application. Therefore, the protection scope of the present application shall be subject to the protection scope of the claims.

Claims

1. A container damage quantification method based on geometric calibration, characterized in that: include: Based on the segmentation result of the container image, a geometric model of each component of the container in the container image is constructed, each of the components including one or more of the following parts: container door, container front, side, top surface, corner surface; Determine the relative position and spatial layout of each of the components according to the position of the corner piece and the segmentation result, and infer the specific working condition of the container according to the relative position and spatial layout; The specific working conditions include: the placement posture of the container; Combined with the standard dimensions of different box types and corner pieces, quantitative vector scales are established for each of the components; The actual location of the container damage is determined based on the geometric model and quantized vector scale established in the image, and the actual size of the damage frame is calculated based on the quantized vector scale established on the surface where the container damage is located.

2. The container damage quantification method based on geometric calibration according to claim 1 is characterized in that: In building the geometric model, include: Corner point detection is used to find the coordinates of the four vertices of the box door and the box front that appear in the image, or the coordinates of the five vertices of the incomplete box door and the box front that lack a corner, and their coordinates are sorted counterclockwise from the upper left corner to construct the geometric model of the box door and the box front; And / or, when the top surface is an incomplete container top that only includes one end of the container in the image, corner point detection is used to find the coordinates of the four vertices of the partial container top that only includes one end in the image, and the coordinates are sorted counterclockwise from the upper left corner to construct a geometric model of the top surface; And / or, when the side surface is a complete side surface or an incomplete container side surface including only one end of the container in the image, corner point detection is used to find the coordinates of the four vertices of the partial container top including only one end or the coordinates of the four vertices of the complete container side surface in the image, and the coordinates are sorted counterclockwise from the upper left corner to construct a geometric model of the side surface; And / or, when the corner piece surfaces of the top and side surfaces are the corner piece top surfaces or corner piece side surfaces where the corner piece exists on the top or side of the box in the image, image processing is used to find its minimum circumscribed rectangle to construct a geometric model of the corner piece surface.

3. The container damage quantification method based on geometric calibration according to claim 1 is characterized in that: Determining the relative positions and spatial layouts of each of the components includes: if the container image contains only a single box, classifying the door, front, side, top and corner fittings detected in the image as the same box body; if the container image contains multiple boxes, classifying the door, front, side, top and corner fittings belonging to the same box body as the same box body using the center distance of each of the components.

4. The container damage quantification method based on geometric calibration according to claim 1, characterized in that: Establishing a quantitative vector scale for the door and front of a container includes: if the door and front of a container are complete, a pair of basis vectors are formed with the edges of the door or front of a container in the height and width directions, and the vector modulus of the basis vectors is established according to the standard width and height of the actual container, so as to obtain a mapping relationship between pixel values; if the door or front of a container is incomplete with a corner missing, the two edges of the missing corner are extended and intersected to obtain the vertices that are blocked or not photographed in the image, and then a pair of basis vectors are formed with the edges of the door or front of a container in the height and width directions, and the vector modulus of the basis vector is established according to the standard width and height of the actual container, so as to obtain a mapping relationship between pixel values; And / or, establishing a quantitative vector scale of the side of the container includes: if the side of the container is complete, forming a pair of basis vectors with the sides of the side of the container in the length and height directions, and establishing the vector modulus of the basis vectors according to the standard length and height of the standard container, and obtaining a mapping relationship between pixel values; if the side of the container only includes one end of the container, forming a pair of basis vectors with the sides of the side of the container in the length and height directions, and then estimating the actual length of the side of the container in the length direction by using the standard length of the corner piece and the projection length of its vector in the length direction, and setting the height to the height of the actual container, establishing the vector modulus of the basis vectors, and obtaining a mapping relationship between pixel values; And / or, establishing a quantitative vector scale for the top surface of the container includes: forming a pair of basis vectors with the edges in the height and width directions of the box top, and then using the standard length of the corner piece and the projection length of its vector in the height direction to estimate the actual length of the box top in the height direction, setting the width to the actual box width, establishing the vector modulus of the basis vector, and obtaining a mapping relationship between pixel values.

5. The container damage quantification method based on geometric calibration according to claim 1, characterized in that: Determining the damaged position of the container includes: based on the vertex coordinates of each part of the container, combined with the center coordinates of the damage frame, using a point-in-polygon algorithm to determine whether the damage is located on a certain container surface.

6. The container damage quantification method based on geometric calibration according to claim 1, characterized in that: The calculation of the actual size of the damaged frame includes: expressing the length and width of the damaged frame with the basis vector of the surface where the damaged frame is located to obtain the modulus of the length and width, and then calculating the actual size of the damaged frame based on the mapping relationship between the vector modulus, i.e., the pixel value and the actual size length of the basis vector; The calculation formula is expressed as: in, is the damaged frame edge vector, is the basis vector, , is the preset coefficient, , is the actual length of the basis vector, is the angle between the basis vectors, The actual length of the calculated damaged frame edge.

7. The container damage quantification method based on geometric calibration according to any one of claims 1 to 6, characterized in that: The container image is segmented based on a preset segmentation model to form the segmentation result; wherein the pre-trained segmentation model is trained by the following steps: the component parts of the container in the container image are marked to obtain a data set required for instance segmentation, and the preset segmentation model is trained based on the data set.

8. A container damage quantification system based on geometric calibration, characterized in that: include: Geometric model subsystem: based on the segmentation result of the container image, construct the geometric model of each component of the container in the container image, each of which includes one or more of the following parts: container door, container front, side, top, corner surface; Inference subsystem: determines the relative position and spatial layout of each component according to the position of the corner piece and the segmentation result, and infers the specific working condition of the container according to the relative position and spatial layout; The specific working conditions include: the placement posture of the container; Vector scale subsystem: combining the standard sizes of different box types and corner pieces to establish the quantitative vector scales of each component; Calculation subsystem: determines the location of the damaged container based on the geometric model and scale established in the image; and calculates the actual size of the damaged frame based on the quantitative vector scale established on the surface where the container is damaged.

9. An electronic device, characterized in that: include: at least one processor; And, a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute: the container damage quantification method based on geometric calibration as described in any one of claims 1-7.

10. A computer storage medium, characterized in that: The computer storage medium stores computer executable instructions, and when the computer executable instructions are executed by the processor, they perform: the container damage quantification method based on geometric calibration as described in any one of claims 1-7.

Images