Container damage quantification method and device based on perspective transformation, equipment and medium

Abstract

The invention provides a container damage quantification method and device based on perspective transformation, equipment and a medium, and the method comprises the steps: obtaining a target image of a container, and marking a corresponding damage frame in a damage region of the container in the target image; performing segmentation processing on each container surface of the container in the target image, and constructing a geometric model of each container surface based on a segmentation result; establishing a perspective mapping relationship between the geometric model of each box surface and the corresponding actual size coordinate; determining a target box surface where the damaged frame is located based on the geometric model of each box surface and the center point coordinate of the damaged frame; and estimating the actual size of the corresponding damaged area based on the perspective mapping relation between the geometric model of the target box surface and the corresponding actual size coordinates and the vertex coordinates of the damaged frame. According to the method, the damage of the container can be accurately quantified under the condition that laser radar equipment is not used.

Description

Technical Field

[0001] The present invention relates to the technical field of container damage detection, and in particular to a container damage quantification method, device, equipment and medium based on perspective transformation. Background Art

[0002] During long-distance transportation and frequent loading and unloading of containers, external impact, improper loading and unloading, or vibration during long-distance transportation often cause containers to suffer varying degrees of damage. These damages not only affect the service life of the container, but may also threaten the safety of the loaded goods. Therefore, accurately quantifying the size of these damages is crucial for the maintenance and safe use of containers.

[0003] Patent applications with publication numbers CN112819793A, CN115187535A, and CN115222697A all disclose container damage detection solutions based on image processing and deep learning, which are mainly based on direct prediction of damage categories and location information based on container images to achieve positioning and classification of container damage. However, current solutions based on image recognition of damage are limited to predicting the category and approximate location of damage, without considering the quantification of damage size, which affects the reliability of repair and maintenance decisions; moreover, in two-dimensional images taken at different angles, due to the perspective effect, the proportions and shapes of various parts of the container will be deformed, and it is difficult to accurately quantify the actual size of the damage based on deep learning, which limits the application value of the detection results.

[0004] Patent applications with publication numbers CN110992337A and CN115965885A disclose methods for detecting and quantifying container damage based on images and point cloud data. The three-dimensional structural information of the container is acquired through laser radar scanning to achieve more accurate identification and quantification of the damage. However, this solution relies on the use of laser radar scanning to acquire point cloud data, which increases the hardware and computing costs, maintenance difficulty, and system complexity of the container damage detection and quantification system. In addition, in some logistics and port environments, it is not practical to deploy and use laser radar. Summary of the invention

[0005] In view of the above-mentioned deficiencies of the prior art, the object of the present invention is to provide a method, device, equipment and medium for quantifying container damage based on perspective transformation, so as to accurately quantify the damage of the container without using a lidar device.

[0006] In order to achieve the above object, the present invention adopts the following technical solutions: In a first aspect, the present invention provides a container damage quantification method based on perspective transformation, the method comprising: Acquire a target image of a container, in which a damaged area of ​​the container is marked with a corresponding damaged frame; Segmenting each container surface of the container in the target image, and constructing a geometric model of each container surface based on the segmentation result; Establishing a perspective mapping relationship between the geometric model of each box surface and the corresponding actual size coordinates; Determine the target box surface where the damaged frame is located based on the geometric models of the respective box surfaces and the coordinates of the center point of the damaged frame; Based on the perspective mapping relationship between the geometric model of the target box surface and the corresponding actual size coordinates, and the vertex coordinates of the damaged frame, the actual size of the corresponding damaged area is estimated.

[0007] Furthermore, the step of constructing the geometric model of each box surface includes: The vertex coordinates of each box surface are determined from the target image using a corner detection algorithm, and the vertex coordinates of each box surface are stored in a predetermined order.

[0008] Furthermore, when a corner piece is provided on the corresponding box surface, the geometric model of the box surface also includes the corner piece size of the corner piece.

[0009] Furthermore, the establishing of the perspective mapping relationship between the geometric model of each box surface and the corresponding actual size coordinates includes executing, for each box surface, respectively: According to the geometric model of the corresponding box surface, the image coordinates of the four corner points of the corresponding box surface are obtained; The perspective mapping relationship is established by taking the image coordinates of the four corner points of the corresponding box surface as the coordinates before perspective transformation and taking the actual coordinates of the four corner points of the corresponding box surface as the coordinates after perspective transformation.

[0010] Furthermore, the obtaining of the image coordinates of the four corner points of the corresponding box surface includes: When the corresponding box surface is complete, the coordinates of the four vertices in the geometric model of the box surface are used as the image coordinates of the four corner points of the box surface; When the corresponding box surface is missing corners, two sides corresponding to the missing corners are extended and intersected based on the geometric model of the box surface to complete the corresponding corners, and the image coordinates of the four corner points of the completed box surface are obtained; When the corresponding box surface is missing one end, the image size of the box surface is estimated based on the corner piece size in the geometric model of the box surface, and the image coordinates of the four corner points of the box surface are obtained based on the estimated image size.

[0011] Furthermore, the established perspective mapping relationship is shown in the following formula:

[0012] in,( , ) is the image coordinate of the i-th corner point of the corresponding box surface, ( , ) is the actual coordinate of the i-th corner point of the box surface, H is the perspective mapping matrix corresponding to the box surface, i=1,2,3,4.

[0013] Furthermore, the estimating the actual size of the corresponding damaged area based on the perspective mapping relationship between the geometric model of the target box surface and the corresponding actual size coordinates, and the vertex coordinates of the damaged frame, includes: Based on the geometric models of the various box surfaces, it is determined to which box surface the coordinates of the center point of the damaged frame belong, and the box surface to which it belongs is used as the target box surface where the damaged frame is located.

[0014] Furthermore, the estimating the actual size of the corresponding damaged area includes: According to the perspective mapping relationship between the geometric model of the target box surface and the corresponding actual size coordinates, the vertex coordinates of the damaged frame are perspectively transformed to obtain the vertex perspective coordinates of the damaged frame; The minimum bounding rectangle corresponding to the perspective coordinates of the vertices of the damaged frame is obtained, and the size of the minimum bounding rectangle is used as the actual size of the corresponding damaged area.

[0015] Furthermore, before determining the perspective mapping relationship between the geometric model of each box surface and the corresponding actual size coordinate, the method further includes: Based on the segmentation result, determining whether the target image includes a single container or two containers; When a single container is included, each of the container faces existing in the target image is regarded as the container face of the container; When two containers are included, each of the container surfaces is respectively assigned to a corresponding container.

[0016] On the second side, the present invention provides a container damage quantification device based on perspective transformation, the device comprising: An image acquisition module, used for acquiring a target image of a container, wherein a damaged area of ​​the container is marked with a corresponding damaged frame in the target image; A model building module, used for segmenting each container surface of the container in the target image, and building a geometric model of each container surface based on the segmentation result; A mapping relationship establishment module, used to establish a perspective mapping relationship between the geometric model of each box surface and the corresponding actual size coordinates; A damage position determination module, used for determining the target box surface where the damage frame is located based on the geometric models of the respective box surfaces and the coordinates of the center point of the damage frame; The damage size estimation module is used to estimate the actual size of the corresponding damaged area based on the perspective mapping relationship between the geometric model of the target box surface and the corresponding actual size coordinates, and the vertex coordinate system of the damage frame.

[0017] In a third aspect, the present invention provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps of the container damage quantification method as described above when executing the computer program.

[0018] In a fourth aspect, the present invention provides a computer-readable storage medium having a computer program stored thereon, wherein the computer program, when executed by a processor, implements the steps of the container damage quantification method as described above.

[0019] By adopting the above technical solution, the present invention has the following beneficial effects: The present invention constructs geometric models of each container surface, establishes a perspective mapping relationship between the geometric models of each container surface and the corresponding actual size coordinates, and maps the vertex coordinates of the damaged frame in the image to the actual physical coordinate space based on the perspective mapping relationship corresponding to the target container surface where the damaged frame is located, so as to estimate the actual size of the damaged area, thereby effectively correcting the perspective distortion of the container in the image under different shooting angles, and being able to more accurately obtain the actual size of the damaged area. Compared with the detection method directly based on two-dimensional images, the present invention significantly improves the accuracy of damage quantification based on perspective transformation; and the present invention only realizes the size quantification of container damage through two-dimensional images and image segmentation technology, avoiding the use of expensive laser radar equipment and complex data processing procedures, simplifying the implementation of the damage quantification system, and reducing the hardware cost, maintenance difficulty and overall complexity of the system. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] Figure 1 This is a schematic diagram of the structure of a typical container; Figure 2 This is a flow chart of a container damage quantification method based on perspective transformation according to Embodiment 1 of the present invention; Figure 3A This is a complete schematic diagram of the box surface in Example 1 of the present invention; Figure 3B This is a schematic diagram of a box surface lacking a corner in Example 1 of the present invention; Figure 3C This is a schematic diagram of a box surface with one end missing in Example 1 of the present invention; Figure 4This is a schematic diagram of perspective transformation from the geometric model of the container surface to the actual size coordinates of the container surface in Example 1 of the present invention; Figure 5 It is a schematic diagram of the principle of estimating the actual size of the damaged area in Embodiment 1 of the present invention; Figure 6 This is a hardware architecture diagram of an electronic device according to Embodiment 3 of the present invention. DETAILED DESCRIPTION

[0021] In order to make the purpose, technical scheme and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present invention.

[0022] The terms used in the present invention are only for the purpose of describing specific embodiments and are not intended to limit the present disclosure. The singular forms "a", "the" and "the" used in the present disclosure and the appended claims are also intended to include plural forms unless the context clearly indicates other meanings. It should also be understood that the term "and / or" used herein refers to and includes any or all possible combinations of one or more associated listed items.

[0023] It should be noted that the terms “front”, “rear”, “left”, “right” and similar expressions used in this application are for illustrative purposes only and are not limiting.

[0024] As mentioned above, among the existing container damage detection technologies, one method is to predict the category and location information of container damage based only on images. This method cannot quantify the damage size and does not consider the impact of perspective effect on damage positioning. Another method is to detect and quantify container damage based on images and point cloud data. This method increases the hardware and computing costs, maintenance difficulty and system complexity of container damage detection.

[0025] In view of this, the present invention proposes a container damage quantification method, device, equipment and medium based on perspective transformation, so as to establish a perspective mapping relationship between the coordinates of the container surface in the image and the actual size coordinates based on the geometric model of each container surface constructed in the image and the corresponding actual standard size, and then calculate the actual size of the damaged area according to the position of the container damage frame using the perspective mapping relationship corresponding to the container surface where it is located, thereby realizing the quantification of container damage.

[0026] Example 1 This embodiment provides a container damage quantification method based on perspective transformation. Figure 1As shown, the container is formed by six rectangular box surfaces to form a cubic box body, and the six box surfaces specifically refer to the front box surface, the rear box surface, the side box surface (including the left box surface and the right box surface), the top box surface and the bottom box surface.

[0027] As an example and not a limitation, at least one of the front box surface, the rear box surface, the side box surface and the top box surface is provided with corner pieces 1 at four corners, respectively. Figure 1 It is exemplarily shown that the four corners of the top box surface and the side box surface are respectively provided with corner pieces 1.

[0028] In one practicable manner, the rear surface of the centralized box is the box door. In another practicable manner, the side surface of the container is the box door. This embodiment does not limit the specific position of the box door.

[0029] like Figure 2 As shown, the container damage quantification method based on perspective transformation in this embodiment specifically includes the following steps: S1, obtaining a target image of a container, in which a damaged area of ​​the container is marked with a corresponding damaged frame.

[0030] As an example but not a limitation, this embodiment first collects multiple video images of at least one of the front, rear, side and top surfaces of the container through a camera, and selects multiple candidate images from the multiple video images, in which at least one surface of each candidate image is complete, or lacks a corner (is blocked or not captured), or lacks an end; then, based on image processing and deep learning methods, the damaged area of ​​the container in the candidate image is detected, and if a damaged area is detected, the candidate image is used as the target image, and the damaged area is located in the target image and the corresponding damaged frame (i.e., the frame of the minimum circumscribed rectangle of the damaged area) is marked.

[0031] In one practicable manner, at least one camera may be used to capture images of the container. The at least one camera includes a front camera, a rear camera, a left camera, a right camera, and a top camera. The front camera is arranged at the front end of the container passage, and is mainly used to capture the video image of the front box surface of the container; the rear camera is arranged at the rear end of the container passage, and is mainly used to capture the video image of the rear box surface of the container; the left camera is arranged on the left side of the container passage, and is mainly used to capture the video image of the left box surface of the container; the right camera is arranged on the right side of the container passage, and is mainly used to capture the video image of the right box surface of the container; the top camera is arranged at the top of the container passage, and is mainly used to capture the video image of the top box surface of the container. The aforementioned container passage is, for example, located at the entrance and exit gates of a yard, a dock, a bonded park, a container manufacturing plant, etc.

[0032] In an practicable manner, the front camera, rear camera, left camera, right camera, and top camera are respectively located at the front upper part, rear upper part, left upper part, right upper part, and directly above the container passage. During the movement of the container, the camera may capture 1-3 sides of the container.

[0033] S2, segmenting each container surface in the target image, and constructing a geometric model of each container surface based on the segmentation result.

[0034] It should be understood that the box surfaces here only refer to the box surfaces existing in the target image. In this embodiment, the box surfaces can be segmented and identified by identifying the edges of the box surfaces and combining the installation positions of the corresponding cameras.

[0035] After obtaining the segmentation results of the box surface, this embodiment uses a corner detection algorithm to determine the vertex coordinates of each box surface from the target image, and stores the vertex coordinates of each box surface in a predetermined order (for example, starting from the upper left vertex and sorting counterclockwise or clockwise) to obtain a geometric model of each box surface.

[0036] It should be understood that when a box surface is complete, the box surface has four vertices T1-T4, such as Figure 3A As shown; when a box surface lacks a corner (is blocked or not photographed), the box surface has five vertices T1~T5, such as Figure 3B As shown, the dotted lines represent missing corners; when a box surface is missing one end (obstructed or not photographed), the box surface has four vertices T1~T4, as shown in Figure 3C As shown, where the dotted line indicates the missing end.

[0037] In addition, when a corner piece is provided on the corresponding box surface, the geometric model of the box surface may further include a corner piece size of the corner piece, and the corner piece size includes an image size corresponding to a minimum circumscribed rectangle of the corner piece in the target image.

[0038] S3, establishing a perspective mapping relationship between the geometric model of each box surface and the corresponding actual size coordinates. The specific establishment process includes performing the following operations for each box surface: S31, obtaining the image coordinates of the four corner points of the corresponding box surface according to the geometric model of the corresponding box surface.

[0039] In this embodiment, the corners of the box surface refer to the right angles formed by its four sides. Figure 3A As shown, when the corresponding box surface in the target image is complete, its four vertices are the corner points of its four corners, so the coordinates of the four vertices in the geometric model of the box surface are directly used as the image coordinates of the four corner points of the box surface.

[0040] like Figure 3BAs shown in , when the corresponding box surface (such as the front box surface or the back box surface) in the target image lacks a corner, based on the geometric model of the box surface, the two edges corresponding to the missing corner are extended to intersect to complete the corresponding corner (such as Figure 3B The image coordinates T1, T2, T6, and T5 of the four corner points of the box surface after completion are obtained; like Figure 3C As shown, when the corresponding box surface (such as the side surface or the top surface) in the target image is missing one end, the image size of the box surface is estimated based on the corner piece size in the geometric model of the box surface, and the image coordinates of the four corner points of the box surface are obtained based on the estimated image size.

[0041] For example, assuming that the corresponding box surface is missing one end along the first direction (such as the length direction), first obtain the ratio between the size of the corner piece along the first direction in the geometric model of the box surface and the actual projection size (known quantity) of the corner piece of the box surface along the first direction; then, by multiplying the actual size of the box surface along the first direction by the ratio, the image size L of the box surface along the first direction can be obtained (see Figure 3C ). So far, the complete image size of the box surface is obtained, and the image coordinates of its two corner points T1 and T2 are known, so the image coordinates of the four corner points T1, T2, T5, and T6 of the box surface can be determined.

[0042] S32, using the image coordinates of the four corner points of the corresponding box surface as the coordinates before perspective transformation, and using the actual coordinates of the four corner points of the corresponding box surface as the coordinates after perspective transformation, to establish a perspective mapping relationship. The specific perspective mapping relationship is expressed as follows:

[0043] Where i=1,2,3,4, ( , ) is the image coordinate of the i-th corner point of the corresponding box surface, ( , ) is the actual coordinate (known quantity) of the i-th corner point of the box surface, H is the perspective mapping matrix corresponding to the box surface, and the perspective mapping matrix is ​​used to describe the third-order perspective mapping relationship from the image box surface to the actual box surface.

[0044] For example, Figure 4 As shown, the left side is a box surface in the target image, and the right side is the box surface in the actual physical space, and the perspective mapping matrix between the two is H.

[0045] S4, based on the geometric models of each box surface and the coordinates of the center point of the damaged frame, determine the target box surface where the damaged frame is located.

[0046] Specifically, firstly, based on the geometric model of each box surface, the principle of a point within a polygon is used to determine which box surface the coordinates of the center point of the damaged frame belong to, and then the box surface to which it belongs is used as the target box surface where the damaged frame is located.

[0047] S5, based on the perspective mapping relationship between the geometric model of the target box surface and the corresponding actual size coordinates, and the vertex coordinates of the damaged box, the actual size of the damaged area is estimated. The specific estimation process is as follows: Firstly, according to the perspective mapping relationship between the geometric model of the target box surface and the corresponding actual size coordinates, the vertex coordinates of the damaged box are perspectively transformed to obtain the vertex perspective coordinates of the damaged box.

[0048] Specifically, Figure 5 As shown, the damaged frame abcd in the target image is perspective transformed according to the following formula to obtain the damaged frame ABCD that is perspectively mapped to the actual physical space.

[0049]

[0050] Where i=1,2,3,4, ( , ) is the image coordinate of the i-th vertex of the damaged frame, ( , ) is the perspective coordinate of the i-th vertex of the damaged box, and H is the perspective mapping matrix between the geometric model of the target box surface obtained in step S3 and the actual size coordinate of the target box surface.

[0051] Then, the minimum bounding rectangle corresponding to the perspective coordinates of the vertices of the damaged frame is obtained, and the size of the minimum bounding rectangle is used as the actual size of the damaged area.

[0052] Specifically, after obtaining the transformed damaged frame ABCD, the minimum circumscribed rectangle A BC The size of D estimates the actual size of the damaged area, and the calculation formula can be expressed as:

[0053] Among them, w and h are the actual sizes of the damaged area in two extension directions. Indicates the size of the minimum enclosing rectangle.

[0054] In an practicable manner, before determining the perspective mapping relationship between the geometric model of each box surface and the corresponding actual size coordinates in step S2, the method of this embodiment may further include: based on the segmentation result of step S1, determining whether the specific working condition of the container in the target image is a single-container situation (i.e., the image includes a single container) or a double-container situation (i.e., the image includes two containers). When a single container is included, each box surface existing in the target image is used as the box surface of the container, and the relative position of each part of the container is known; when two containers are included, each box surface is respectively assigned to the corresponding container, for example, the front box surface, rear box surface, side box surface, top box surface and corner fittings belonging to the same box body are assigned to one container according to the center distance of each part, so as to determine the relative position and spatial layout of each part, so as to facilitate the subsequent execution of steps S2 to S5 for each container separately.

[0055] It can be seen that this embodiment constructs the geometric model of each box surface of the container, establishes the perspective mapping relationship between the geometric model of each box surface and the corresponding actual size coordinates, and based on the perspective mapping relationship corresponding to the target box surface where the damage box is located, maps the vertex coordinates of the damage box in the image to the actual physical coordinate space to estimate the actual size of the damaged area, thereby effectively correcting the perspective distortion of the container in the image under different shooting angles, and can more accurately obtain the actual size of the damaged area. Compared with the detection method directly based on two-dimensional images, the present invention significantly improves the accuracy of damage quantification based on perspective transformation; and the present invention only realizes the size quantification of container damage through two-dimensional images and image segmentation technology, avoiding the use of expensive lidar equipment and complex data processing procedures, simplifying the implementation of the damage quantification system, and reducing the hardware cost, maintenance difficulty and overall complexity of the system.

[0056] Example 2 This embodiment provides a container damage quantification device based on perspective transformation, the device comprising: An image acquisition module, used for acquiring a target image of the container, in which a damaged area of ​​the container is marked with a corresponding damaged frame; A model building module, used for segmenting each container surface of the container in the target image, and building a geometric model of each container surface based on the segmentation result; A mapping relationship establishment module, used to establish a perspective mapping relationship between the geometric model of each box surface and the corresponding actual size coordinates; A damage position determination module, used for determining the target box surface where the damage frame is located based on the geometric models of the respective box surfaces and the coordinates of the center point of the damage frame; The damage size estimation module is used to estimate the actual size of the corresponding damaged area based on the perspective mapping relationship between the geometric model of the target box surface and the corresponding actual size coordinates, and the vertex coordinate system of the damage frame.

[0057] Furthermore, the model building module uses a corner detection algorithm to determine the vertex coordinates of each box surface from the target image, and stores the vertex coordinates of each box surface in a predetermined order.

[0058] Furthermore, when a corner piece is provided on the corresponding box surface, the geometric model of the box surface also includes the corner piece size of the corner piece.

[0059] Furthermore, the mapping relationship establishing module includes: A corner coordinate acquisition unit, used for acquiring the image coordinates of four corner points of the corresponding box surface according to the geometric model of the corresponding box surface; The transmission transformation unit is used to establish the perspective mapping relationship by taking the image coordinates of the four corner points of the corresponding box surface as the coordinates before perspective transformation and taking the actual coordinates of the four corner points of the corresponding box surface as the coordinates after perspective transformation.

[0060] Furthermore, the corner coordinate acquisition unit is specifically used for: When the corresponding box surface is complete, the coordinates of the four vertices in the geometric model of the box surface are used as the image coordinates of the four corner points of the box surface; When the corresponding box surface is missing corners, two sides corresponding to the missing corners are extended and intersected based on the geometric model of the box surface to complete the corresponding corners, and the image coordinates of the four corner points of the completed box surface are obtained; When the corresponding box surface is missing one end, the image size of the box surface is estimated based on the corner piece size in the geometric model of the box surface, and the image coordinates of the four corner points of the box surface are obtained based on the estimated image size.

[0061] Furthermore, the perspective mapping relationship is shown in the following formula:

[0062] in,( , ) is the image coordinate of the i-th corner point of the corresponding box surface, ( , ) is the actual coordinate of the i-th corner point of the box surface, H is the perspective mapping matrix corresponding to the box surface, i=1,2,3,4.

[0063] Furthermore, the damaged position determination module is specifically used to: determine to which box surface the center point coordinates of the damaged frame belong based on the geometric models of the various box surfaces, and use the box surface to which it belongs as the target box surface where the damaged frame is located.

[0064] Furthermore, the damage size estimation module is specifically used for: According to the perspective mapping relationship between the geometric model of the target box surface and the corresponding actual size coordinates, the vertex coordinates of the damaged frame are perspectively transformed to obtain the vertex perspective coordinates of the damaged frame; The minimum bounding rectangle corresponding to the perspective coordinates of the vertices of the damaged frame is obtained, and the size of the minimum bounding rectangle is used as the actual size of the corresponding damaged area.

[0065] Furthermore, the device further comprises a working condition determination module, and the working condition determination module is specifically used for: Based on the segmentation result, determining whether the target image includes a single container or two containers; When a single container is included, each of the container faces existing in the target image is regarded as the container face of the container; When two containers are included, each of the container surfaces is respectively assigned to a corresponding container.

[0066] This embodiment constructs the geometric model of each box surface of the container, establishes the perspective mapping relationship between the geometric model of each box surface and the corresponding actual size coordinates, and maps the vertex coordinates of the damaged frame in the image to the actual physical coordinate space based on the perspective mapping relationship corresponding to the target box surface where the damaged frame is located, so as to estimate the actual size of the damaged area, thereby effectively correcting the perspective distortion of the container in the image under different shooting angles, and being able to more accurately obtain the actual size of the damaged area. Compared with the detection method directly based on two-dimensional images, the present invention significantly improves the accuracy of damage quantification based on perspective transformation; and the present invention only realizes the size quantification of container damage through two-dimensional images and image segmentation technology, avoiding the use of expensive laser radar equipment and complex data processing procedures, simplifying the implementation of the damage quantification system, and reducing the hardware cost, maintenance difficulty and overall complexity of the system.

[0067] Example 3 This embodiment provides an electronic device, which can be expressed in the form of a computing device (for example, a server device), including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor can implement the steps of the container damage quantification method provided in Embodiment 1 when executing the computer program.

[0068] Figure 6 FIG. 4 shows a schematic diagram of the hardware structure of this embodiment. Figure 6 As shown, the electronic device 30 specifically includes: At least one processor 31, at least one memory 32, and a bus 33 for connecting different system components (including the processor 31 and the memory 32), wherein: The bus 33 includes a data bus, an address bus, and a control bus.

[0069] The memory 32 includes a volatile memory, such as a random access memory (RAM) 321 and / or a cache memory 322 , and may further include a read-only memory (ROM) 323 .

[0070] The memory 32 also includes a program / utility 325 having a set (at least one) of program modules 324, such program modules 324 including but not limited to: an operating system, one or more application programs, other program modules and program data, each of which or some combination may include an implementation of a network environment.

[0071] The processor 31 executes various functional applications and data processing by running the computer program stored in the memory 32, such as the steps of the container damage quantification method provided in Example 1 of the present invention.

[0072] The electronic device 30 can further communicate with one or more external devices 34 (e.g., keyboards, pointing devices, etc.). Such communication can be performed through an input / output (I / O) interface 35. In addition, the electronic device 30 can also communicate with one or more networks (e.g., local area networks (LANs), wide area networks (WANs), and / or public networks, such as the Internet) through a network adapter 36. The network adapter 36 communicates with other modules of the electronic device 30 through a bus 33. It should be understood that, although not shown in the figure, other hardware and / or software modules can be used in conjunction with the electronic device 30, including but not limited to: microcode, device drivers, redundant processors, external disk drive arrays, RAID (disk array) systems, tape drives, and data backup storage systems, etc.

[0073] It should be noted that although several units / modules or sub-units / modules of the electronic device are mentioned in the above detailed description, this division is merely exemplary and not mandatory. In fact, according to the embodiments of the present application, the features and functions of two or more units / modules described above can be embodied in one unit / module. Conversely, the features and functions of one unit / module described above can be further divided into multiple units / modules to be embodied.

[0074] Example 4 This embodiment provides a computer-readable storage medium on which a computer program is stored. When the program is executed by a processor, the steps of the container damage quantification method provided in Example 1 are implemented.

[0075] The readable storage medium may include but is not limited to: a portable disk, a hard disk, a random access memory, a read-only memory, an erasable programmable read-only memory, an optical storage device, a magnetic storage device or any suitable combination of the above.

[0076] In a possible implementation manner, the present invention may also be implemented in the form of a program product, which includes a program code. When the program product is run on a terminal device, the program code is used to enable the terminal device to execute the steps of the container damage quantification method provided in Example 1.

[0077] The program code for executing the present invention may be written in any combination of one or more programming languages, and may be executed entirely on a user device, partially on a user device, as an independent software package, partially on a user device and partially on a remote device, or entirely on a remote device.

[0078] Although the specific embodiments of the present invention are described above, it should be understood by those skilled in the art that this is only for illustration and the protection scope of the present invention is defined by the appended claims. Those skilled in the art may make various changes or modifications to these embodiments without departing from the principles and essence of the present invention, but these changes and modifications all fall within the protection scope of the present invention.

Claims

1. A container damage quantification method based on perspective transformation, characterized in that: The method comprises: Acquire a target image of a container, in which a damaged area of ​​the container is marked with a corresponding damaged frame; Segmenting each container surface of the container in the target image, and constructing a geometric model of each container surface based on the segmentation result; Establishing a perspective mapping relationship between the geometric model of each box surface and the corresponding actual size coordinates; Determine the target box surface where the damaged frame is located based on the geometric models of the respective box surfaces and the coordinates of the center point of the damaged frame; Based on the perspective mapping relationship between the geometric model of the target box surface and the corresponding actual size coordinates, and the vertex coordinates of the damaged frame, the actual size of the corresponding damaged area is estimated.

2. The container damage quantification method according to claim 1, characterized in that: The step of constructing the geometric model of each box surface comprises: The vertex coordinates of each box surface are determined from the target image using a corner detection algorithm, and the vertex coordinates of each box surface are stored in a predetermined order.

3. The container damage quantification method according to claim 2, characterized in that: When corner fittings are provided on the corresponding box surface, the geometric model of the box surface also includes the corner fitting dimensions of the corner fittings.

4. The container damage quantification method according to claim 1, characterized in that: The establishing of the perspective mapping relationship between the geometric model of each box surface and the corresponding actual size coordinates includes executing respectively for each box surface: According to the geometric model of the corresponding box surface, the image coordinates of the four corner points of the corresponding box surface are obtained; The perspective mapping relationship is established by taking the image coordinates of the four corner points of the corresponding box surface as the coordinates before perspective transformation and taking the actual coordinates of the four corner points of the corresponding box surface as the coordinates after perspective transformation.

5. The container damage quantification method according to claim 1, characterized in that: The step of obtaining the image coordinates of the four corner points of the corresponding box surface includes: When the corresponding box surface is complete, the coordinates of the four vertices in the geometric model of the box surface are used as the image coordinates of the four corner points of the box surface; When the corresponding box surface lacks corners, two sides corresponding to the missing corners are extended and intersected based on the geometric model of the box surface to complete the corresponding corners, and the image coordinates of the four corner points of the completed box surface are obtained; When the corresponding box surface is missing one end, the image size of the box surface is estimated based on the corner piece size in the geometric model of the box surface, and the image coordinates of the four corner points of the box surface are obtained based on the estimated image size.

6. The container damage quantification method according to claim 4, characterized in that: The established perspective mapping relationship is shown in the following formula: in,( , ) is the image coordinate of the i-th corner point of the corresponding box surface, ( , ) is the actual coordinate of the i-th corner point of the box surface, H is the perspective mapping matrix corresponding to the box surface, i=1,2,3,4.

7. The container damage quantification method according to claim 1, characterized in that: The estimating the actual size of the corresponding damaged area based on the perspective mapping relationship between the geometric model of the target box surface and the corresponding actual size coordinates, and the vertex coordinates of the damaged frame, comprises: Based on the geometric models of the various box surfaces, it is determined to which box surface the coordinates of the center point of the damaged frame belong, and the box surface to which it belongs is used as the target box surface where the damaged frame is located.

8. The container damage quantification method according to claim 1, characterized in that: The estimating the actual size of the damaged area includes: According to the perspective mapping relationship between the geometric model of the target box surface and the corresponding actual size coordinates, the vertex coordinates of the damaged frame are perspectively transformed to obtain the vertex perspective coordinates of the damaged frame; The minimum bounding rectangle corresponding to the perspective coordinates of the vertices of the damaged frame is obtained, and the size of the minimum bounding rectangle is used as the actual size of the corresponding damaged area.

9. The container damage quantification method according to claim 1, characterized in that: Before determining the perspective mapping relationship between the geometric model of each box surface and the corresponding actual size coordinate, the method further includes: Based on the segmentation result, determining whether the target image includes a single container or two containers; When a single container is included, each of the container faces existing in the target image is regarded as the container face of the container; When two containers are included, each of the container surfaces is respectively assigned to a corresponding container.

10. A container damage quantification device based on perspective transformation, characterized in that: The device comprises: An image acquisition module, used for acquiring a target image of a container, wherein a damaged area of ​​the container is marked with a corresponding damaged frame in the target image; A model building module, used for segmenting each container surface of the container in the target image, and building a geometric model of each container surface based on the segmentation result; A mapping relationship establishment module, used to establish a perspective mapping relationship between the geometric model of each box surface and the corresponding actual size coordinates; A damage position determination module, used for determining the target box surface where the damage frame is located based on the geometric models of the respective box surfaces and the coordinates of the center point of the damage frame; The damage size estimation module is used to estimate the actual size of the corresponding damaged area based on the perspective mapping relationship between the geometric model of the target box surface and the corresponding actual size coordinates, and the vertex coordinate system of the damage frame.

11. An electronic device comprising a memory, a processor, and a computer program stored in the memory and capable of running on the processor, characterized in that: When the processor executes the computer program, the steps of the container damage quantification method according to any one of claims 1 to 9 are implemented.

12. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the container damage quantification method according to any one of claims 1 to 9 are implemented.

Images