Container Deformation Recognition Method, Device and Electronic Equipment

Abstract

This application relates to the technical field of container transportation and management, and solves the technical problem that the prior art urgently needs a convenient and fast method for identifying container deformation to ensure the safe placement of containers in logistics transportation. By identifying the container side lines and container corner points from the container images, and judging whether the container is deformed by analyzing the deviation between the lines connecting the container corner points and the container side lines between the container corner points, this application can timely and accurately detect the deformation problem of the container during the use of the container and avoid the placement risk.

Description

Technical Field

[0001] This application relates to the technical field of container transportation and management, and particularly relates to a method, device, and electronic device for identifying container deformation. Background Art

[0002] During the logistics transportation process, containers for transporting goods are placed together to reduce the occupied space and improve space utilization. Most containers are designed as rectangular parallelepipeds to adapt to stacking and close placement.

[0003] However, if a container is deformed, it will not only affect the placement of the goods inside the container, but also affect the placement of other containers placed closely or stacked on it. For example, if a container has a bulge, during the container hoisting operation, the container will rotate in the air, and the bulge is likely to collide / abrade with adjacent containers, which may cause safety accidents.

[0004] Due to the large volume of containers, manually checking each container during the logistics operation will consume a lot of manpower and time. Obviously, there is an urgent need for a convenient and fast method for identifying container deformation to ensure the safe placement of containers during logistics transportation. Summary of the Invention

[0005] In view of this, the embodiments of this application are committed to providing a method for identifying container deformation to solve the technical problem that there is an urgent need for a convenient and fast method for identifying container deformation in the prior art to ensure the safe placement of containers during logistics transportation.

[0006] In a first aspect, the embodiments of this application provide a method for identifying container deformation, including:

[0007] Obtaining and processing image data to obtain a clear container image;

[0008] Identifying and obtaining the container side lines and container corner points in the container image, where the container corner points are the vertices of the included angles formed by each side of the container;

[0009] Calculating the deviation amount between the connection line between all adjacent corner points and the container side line between the adjacent corner points among the container corner points. When the deviation amount is greater than a preset first error range, it is determined that the container is deformed, and the adjacent corner points are the container corner points located on the same container side line.

[0010] Optionally, the calculating the deviation amount between the connection line between all adjacent corner points and the container side line between the adjacent corner points among the container corner points includes:

[0011] Obtaining all adjacent corner points among the container corner points;

[0012] Obtain the deviation parts between the container side lines of all adjacent corner points and the connecting lines between the adjacent corner points, and the length of the perpendicular line segment from any point on the deviation part to the connecting line between the adjacent corner points is greater than the first error range;

[0013] Divide the deviation part into at least one deformed side line segment, and each deformed side line segment obtained by the division includes a continuous container side line with one undulation, and the undulation refers to the change trend of the distance from the points on the container side line to the connecting line between the adjacent corner points, which first increases and then decreases;

[0014] Obtain the distance between each deformed side line segment and the connecting line between the adjacent corner points as the deviation amount.

[0015] Optionally, the obtaining the distance between each deformed side line segment and the connecting line between the adjacent corner points as the deviation amount includes:

[0016] For each deformed side line segment, determine the point with the maximum distance from the connecting line between the adjacent corner points in the deformed side line segment as the deformation vertex;

[0017] Calculate the length of the perpendicular line segment from each deformation vertex to the connecting line between the adjacent corner points as the deviation amount.

[0018] Optionally, the method further includes:

[0019] Identify the container placement direction according to the preset container features, and the container features include the shape of the container door;

[0020] Determine the deformation position of the container according to the container placement direction and the deviation amount.

[0021] Optionally, the method further includes:

[0022] For any container corner point, calculate the angle of the included angle where the container corner point is located. When the angle of the included angle where the container corner point is located is greater than or less than the preset included angle range, it is determined that the container is deformed.

[0023] Optionally, the method further includes:

[0024] Obtain the ambient light intensity, and determine the second error range according to the ambient light intensity;

[0025] Identify and obtain the concave area of the container image, and the color difference between the concave area and the area within a preset distance range around the concave area is greater than the second error range.

[0026] Optionally, the identifying and obtaining the concave area of the container image includes:

[0027] Identify and obtain the abnormal area of the container image, where the color difference between the abnormal area and other areas in the container image is greater than a preset third error range;

[0028] Identify the shape and uniformity of the abnormal area, and divide the abnormal area into at least two area sets, where the at least two area sets include a first sunken area set, and the first sunken area set does not include areas with regular shapes and / or uniform colors inside the abnormal area;

[0029] For the area sets other than the first sunken area set in the area sets, identify them to obtain a second sunken area set, where the color difference between the areas in the second sunken area set and other areas in the area sets other than the first sunken area set is greater than the second error range;

[0030] Merge the first sunken area set and the second sunken area set to obtain the sunken area.

[0031] Optionally, the method further includes:

[0032] Obtain the distance between the ranging device and the container and the distance between the ranging device and the sunken area through the ranging device, and calculate the sunken amount of the sunken area.

[0033] In a second aspect, an embodiment of the present application provides a container deformation identification device, including:

[0034] An image processing unit, configured to obtain image data, identify, segment, and process the edges of the image data to obtain a container image with clear edges;

[0035] An identification unit, configured to identify and obtain the container edges and container corner points in the container image, where the container corner points are the vertices of the angles formed by each side of the container;

[0036] A calculation unit, configured to calculate the deviation amount between the line connecting all adjacent corner points and the container edge line between the adjacent corner points among the container corner points. When the deviation amount is greater than a preset first error range, it is determined that the container is deformed, and the adjacent corner points are the container corner points located on the same container edge line.

[0037] In a third aspect, an embodiment of the present application provides an electronic device, including a memory and a processor;

[0038] The memory is connected to the processor and is used to store programs;

[0039] The electronic device runs the program in the memory through the processor to implement the container deformation recognition method described in the first aspect.

[0040] This application provides a container deformation recognition method. By recognizing the container image to obtain the container side lines and container corner points, and judging whether the container is deformed by analyzing the deviation between the connection lines between the container corner points and the container side lines between the container corner points, it can timely and accurately detect the deformation problem of the container during the use of the container and avoid the placement risk. BRIEF DESCRIPTION OF THE DRAWINGS

[0041] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the following will briefly introduce the drawings required for the description of the embodiments or the prior art. Obviously, the drawings in the following description are only the embodiments of the present application. For those of ordinary skill in the art, without creative efforts, other drawings can also be obtained according to the provided drawings.

[0042] Figure 1 It is a schematic flowchart of a container deformation recognition method provided by an embodiment of the present application.

[0043] Figure 2 It is an example of four top view images of a container provided by an embodiment of the present application.

[0044] Figure 3 It is an example of four left view images of a container provided by an embodiment of the present application.

[0045] Figure 4 It is an example of four front view images of a container provided by an embodiment of the present application.

[0046] Figure 5 It is a schematic diagram of the shape of a container provided by an embodiment of the present application.

[0047] Figure 6 It is a schematic structural diagram of a container deformation recognition device provided by an embodiment of the present application.

[0048] Figure 7 It is a schematic structural diagram of an electronic device in an embodiment of the present application. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0049] The following will clearly and completely describe the technical solutions in the embodiments of the present application with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments of the present application, all other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the protection scope of the present application.

[0050] The first embodiment of this application provides a method for identifying the deformation of a container, as Figure 1 shown. This method may include the following steps:

[0051] Step 101: Obtain and process image data to obtain a container image with clear lines.

[0052] In this embodiment, the image data can be obtained through an image acquisition device, such as pictures obtained by a camera, infrared images obtained by an infrared imaging device, etc.

[0053] The processing of the image data may include image recognition, image segmentation, and edge fitting and other processing methods. Since the image data collected by the image acquisition device has a large coverage range and includes a lot of content, in order to reduce the subsequent processing load, the image data can be recognized. For example, the image is recognized through semantic analysis, and a partial area belonging to the container is recognized.

[0054] Then, the container image can be segmented from the entire image to reduce the size of the image to be processed in subsequent processing.

[0055] The processing of the image data may include edge fitting. Since the clarity of the image data is limited by the hardware conditions of the image acquisition device, for the case of low image clarity, the lines in the image can be processed through edge fitting to obtain an image with clear lines.

[0056] The method of cutting the container image can be to cut the recognized container image in a large range, that is, most of the content in the cut container image is the container, but it also includes a part of the environment content around the container; or, it can be cut strictly according to the lines of the container, similar to the cutting method of "cutting along the lines" in the paper-cutting process. The cut container image does not include content other than the container.

[0057] This embodiment does not limit the specific quantity of the image data. For example, for the same container A, the image acquisition device acquires images of container A from multiple angles and divides them into the front view, left view, and top view of container A, all of which are used as image data. Then, the front view, left view, and top view are all processed to obtain images of container A from three angles. Collecting image data from multiple angles helps to more comprehensively identify the deformation of the container.

[0058] If the image data is relatively comprehensive, including the front view, left view, and top view of the container, then through subsequent processing, the entire situation of the container can be obtained, that is, the situation of each side of the container. Through quantitative analysis, such as establishing a spatial coordinate system and analyzing the distances of deformations of each side in the x, y, and z directions, the precise coordinates of each point of the deformation can be obtained, and the specific deformation distance can also be calculated and used as the deviation amount.

[0059] Step 102: Identify and obtain the container edges and container corner points in the container image. The container corner points are the vertices of the angles formed by each side of the container.

[0060] In a container image with clear lines, the container should be represented as a polygon. For example, for a container with a rectangular shape and no deformation, the container image obtained from the side of the container should be a regular rectangle.

[0061] Decompose the polygon in the container image to obtain the container edges and container corner points. Each side of the polygon corresponds to a side of the container, and each side of the polygon is used as the container edge. Each vertex of the polygon corresponds to a corner of the container, and each vertex of the polygon is used as the container corner point.

[0062] Step 103: Calculate the deviation amount between the line connecting all adjacent corner points and the container edge between the adjacent corner points among the container corner points. When the deviation amount is greater than the preset first error range, it is determined that the container is deformed. Adjacent corner points are the container corner points located on the same container edge.

[0063] Figure 2 、 Figure 3 and Figure 4 is an example for the deformation of a rectangular container. Figure 2 is the top view collected for the container, including four container image examples. Figure 3 is the left view collected for the container, including four container image examples. Figure 4 is the front view collected for the container, that is, the container image collected from the container door or the container tail, including four container image examples.

[0064] Take Figure 2For example, a set of adjacent corner points are called point A and point B, and there is a bulging deformation between AB, and the vertex of the bulge is called point P. Then, the line segment AB obtained by directly connecting AB does not coincide with the line segment APB passing through point P near point P, and the line segment APB deviates from the line segment AB near point P. Calculate the deviation between the line segment APB and the line segment AB. For an image composed of pixels, the deviation can be the number of pixel points between the line segment APB and the line segment AB. For a vector image, the deviation can be the distance between the line segment APB and the line segment AB. When the deviation is greater than the first error range, it is determined that the container is deformed.

[0065] Since both the image acquisition device and image processing have certain accuracy limitations, a first error range is preset to reduce misidentification situations. For example, for an image composed of pixels, the first error range is set to 5 pixel points, that is, when the connection line between adjacent corner points does not coincide with the container side line between adjacent corner points, but the deviation range is within 5 pixel points, it is still considered that the container has not deformed.

[0066] In the second embodiment of the present application, the container deformation recognition method in the first embodiment is further more detailed and specific. Some or all of the technical features in the second embodiment can be combined with, replaced with, etc. the first embodiment alone or jointly, so as to obtain more feasible container deformation recognition methods.

[0067] The container deformation recognition method in the second embodiment of the present application is elaborated in detail below:

[0068] Optionally, calculating the deviation between the connection line between all adjacent corner points among the container corner points and the container side line between adjacent corner points includes: obtaining all adjacent corner points among the container corner points; obtaining the deviation part between the container side line between all adjacent corner points and the connection line between adjacent corner points, and the length of the perpendicular line segment from any point on the deviation part to the connection line between adjacent corner points is greater than the first error range; dividing the deviation part into at least one deformed side line, and each deformed side line obtained by division includes a continuous container side line with one undulation, and the undulation refers to the change trend of the distance from the points on the container side line to the connection line between adjacent corner points being first increasing and then decreasing; obtaining the distance between each deformed side line and the connection line between adjacent corner points as the deviation.

[0069] This embodiment defines a method for obtaining the deviation when the situation of the container side line is relatively complex. First, in the container side line, select the set of all points whose perpendicular line segment length to the connection line between adjacent corner points is greater than the first error range to form the deviation part.

[0070] The container side line in the deviation part may be just a continuous curve or may be multiple continuous curves. For example, in Figure 3In the third example image counted from left to right, there are two bulges between corner point A and corner point B. Therefore, the deviated part obtained for this section of the container side line may include two continuous curves.

[0071] Analyze with one bulge or one depression. Whether it is a bulge or a depression, the distance from the points on the container side line that form a bulge or a depression to the line connecting the adjacent corner points will increase first and then decrease. And, the container side line that forms a bulge or a depression is continuous. Therefore, according to the feature of having a continuous single undulation, divide the deviated part to determine the specific number of deformations. The number of the divided deformed side lines represents the number of deformations on this section of the container side line. For example, if two deformed side lines are divided, it means there are two deformations on this section of the container side line.

[0072] Finally, obtain the distance between each section of the deformed side line and the line connecting the adjacent corner points as the deviation amount. The number of data of the deviation amount should correspond to the number of the deformed side lines, that is, for each section of the deformed side line, there should be a corresponding deviation distance in the deviation amount.

[0073] Optionally, obtaining the distance between each section of the deformed side line and the line connecting the adjacent corner points as the deviation amount includes: for each section of the deformed side line, determine the point with the maximum distance from the line connecting the adjacent corner points in the deformed side line as the deformation vertex; calculate the length of the perpendicular line segment between each deformation vertex and the line connecting the adjacent corner points as the deviation amount.

[0074] This embodiment defines the specific method for obtaining the deviation amount. Since the distance between a continuous section of the deformed side line and the line connecting the adjacent corner points increases first and then decreases. Therefore, the point where it changes from increasing to decreasing is the extreme value point with the maximum distance from the line connecting the adjacent corner points in the deformed side line, that is, Figures 2 - 4 point P in. Take this point as the deformation vertex and draw a perpendicular line segment to the line connecting the adjacent corner points, and calculate the length of the perpendicular line segment as the deviation amount. Among them, the length unit of the perpendicular line segment is selected according to the image type. For the image composed of pixel points, the length of the perpendicular line segment can be described by the number of pixel points.

[0075] Taking the distance between the extreme value point P and the line connecting the adjacent corner points as the deviation amount helps to quantitatively describe the severity of the deformation.

[0076] In addition, if data such as the size of the container or the image ratio is known, the conversion ratio between the distance in the container image and the actual distance can be obtained. Based on this, according to the deviation amount, it can be further calculated how many centimeters the container is deformed in reality.

[0077] Optionally, the container deformation recognition method further includes: identifying the placement direction of the container according to preset container features, where the container features include the shape of the container door; determining the deformation position of the container according to the container placement direction and the deviation amount.

[0078] This embodiment provides a method for determining the specific position of the container deformation. Since image recognition can only recognize deformations such as bulges or depressions, but cannot describe the specific position of the deformation. Therefore, this embodiment proposes that some shape features of the container can be obtained in advance. For example, for the container shown Figure 5 , there is a standard rectangular depression at the door of the container. According to these shape features of the container, the placement direction of the container can be determined. For example, in Figure 2 , it can be clearly found that the door is on the left side, that is, the container is placed with the left side facing forward. For other containers with shape characteristics or painting characteristics, these features can also be added to the preset container features. For example, a certain company's container has a large number of red wavy paintings on the door side, then "red wavy" can be added to the container features to determine the orientation of the door.

[0079] Then, according to the container placement direction, combined with the container side line and position where the deviation amount is located, the container deformation position can be determined. For example, in Figure 2 , in the third example image from left to right, it can be determined that the container is placed with the left side facing forward, and the deviation occurs at the bottom of the image, so it can be determined that there is a bulge deformation on the left side of the container in reality.

[0080] Optionally, the container deformation recognition method further includes: for any container corner point, calculating the angle of the included angle where the container corner point is located, and when the angle of the included angle where the container corner point is located is greater than or less than the preset included angle range, it is determined that the container is deformed.

[0081] This embodiment also proposes a method for analyzing the shape of the container according to the angle to identify whether the container is deformed. Calculate the angle of the included angle where the container corner point is located.

[0082] For example, for a rectangular container, when the image acquisition device is facing the container directly and can obtain the standard three-view drawings of the container, the captured image should have four container corner points. Each container corner point should be the vertex of a right angle and a major angle. The right angle should be 90°, and the major angle should be 270°. When the image acquisition device does not face the container directly for image acquisition, the number of container corner points in the image is determined by the shooting angle of the image acquisition device relative to the container. For example, if the image includes two faces of the container, the image should have six container corner points. Four of the container corner points are the vertices of a 90° right angle and a 270° major angle, and the other two container corner points are the vertices of two 90° right angles and a 180° straight angle. If the image includes three faces of the container, referring to Figure 5 , then the image should have seven container corner points (ignoring the structural features of the container, such as the recessed features of the container door). In the case of no deformation, the angles of the included angles where each container corner point is located are determined by the shooting angle of the image acquisition device relative to the container. As the shooting angle changes, the angles of the included angles where each container corner point is located will also change accordingly and are interrelated. For example, if Figure 5 the three included angles where container corner point A is located in the figure are all 120°, then the included angle angles where container corner point B is located should be 60°, 60°, and 240°.

[0083] Therefore, the included angle range needs to be preset according to the shooting angle of the image acquisition device relative to the container. When the shooting angle of the image acquisition device relative to the container changes, the preset included angle range needs to be adjusted accordingly. Which included angle the preset included angle range specifically targets for the container corner point can be set as needed. Generally, the acute angle, right angle, or obtuse angle where the container corner point is located is selected to set the included angle range, and the corresponding included angle range for the major angle and straight angle is rarely set.

[0084] Considering that both the image acquisition device and image processing have certain accuracy limitations, the preset included angle range is set. For example, for the standard three-view drawings of the container, the included angle range corresponding to the right angle where the container corner point is located can be [89°, 91°]. Within this range, it is determined that the container has not deformed. When it is greater than 89° or less than 91°, it is determined that the container has deformed.

[0085] This embodiment analyzes based on the angles of the container corner points and can be used to supplement the method in the foregoing embodiment to identify the small deformation of the container, that is, the situation where the deviation is within the first error range but the deformation involves a large range and causes the included angle where the container corner point is located to change.

[0086] Optionally, the container deformation recognition method further includes: obtaining the ambient light intensity, determining a second error range according to the ambient light intensity; identifying and obtaining the concave area of the container image, and the color difference between the concave area and the area within a preset distance range around the concave area is greater than the second error range.

[0087] This embodiment proposes a method for obtaining the concave area of a container image. Due to the characteristics of the concave structure, its ability to reflect ambient light is very different from that of the surface of an undeformed container, which is reflected in the image as a color difference between the concave area and the normal area around the concave area. This embodiment defines that within a preset distance range around the concave area, if the color within this range has an error greater than the second error range from the color of the concave area, it is determined that a concave area appears.

[0088] The ambient light intensity determines the color difference between the concave area and the normal area in the image. Specifically, when the ambient light is strong, the color difference between the concave area and the normal area is large, and when the ambient light is weak, the color difference between the concave area and the normal area is also large. Therefore, it is necessary to obtain the ambient light intensity in advance to set the second error range. In addition, when different light sources illuminate the surface of the container, there may also be errors between normal areas. For example, when using a light bulb to illuminate at a distance relatively close to the container. As a point light source, the light of the light bulb illuminating the surface of the container nearby and the surface of the container in the distance will also cause a small color difference between the nearby and distant surfaces of the container in the image. Therefore, the second error range also needs to ensure that it can filter out the color difference generated by the ambient light.

[0089] The specific definition of the color difference can be determined according to the type of the image. For example, for an image composed of pixels, the color difference between regions can be obtained by calculating the RGB values of pixel points.

[0090] Since the deformed areas of some concave deformations of the container do not involve the edges of the container, these concavities cannot be found by analyzing the container edges. The method for determining the concave area according to the color difference in the image in this embodiment makes up for the above defects.

[0091] Optionally, identify and obtain the concave regions of the container image, including: identifying and obtaining the abnormal regions of the container image, where the color difference between the abnormal regions and other regions in the container image is greater than a preset third error range; identifying the shape and uniformity of the abnormal regions, and dividing the abnormal regions into at least two region sets, where at least two region sets include a first concave region set, and the first concave region set does not include regions with regular shapes and / or uniform colors inside the abnormal regions; for the region sets other than the first concave region set in the region sets, identify them to obtain a second concave region set, where the color difference between the regions in the second concave region set and other regions in the region sets other than the first concave region set is greater than a second error range; merge the first concave region set and the second concave region set to obtain the concave regions.

[0092] In practical applications, there may be coatings painted on the container, or due to the occlusion of other objects such as surrounding containers, shadows may be produced on the container. The above situations will all cause color differences between normal regions in the container image.

[0093] This embodiment proposes to distinguish normal regions and concave regions by identifying the colors of the images.

[0094] First, separate the regions where the color difference is greater than a preset third error range. The specific value of the third error range and its magnitude relationship with the second error range can be set according to specific needs. For example, for container A with a coating of 70% red and 30% white, if it is necessary to separately analyze the red coating part and the white coating part, then the value of the third error range only needs to meet the requirement of distinguishing different color coatings. And within the same color coating region, the color difference between the concave and the normal surface is generally smaller than the color difference between different color coatings. Therefore, in this case, the third error range is greater than the second error range. If it is desired to separate all possible concave regions during the region separation process, then the value of the third error range can be set equal to the second error range.

[0095] For container A with a coating of 70% red and 30% white in the above example, the separation result can be that the white region is the abnormal region, and if there is a concave in the red coating region of container A and the value of the third error range is equal to the second error range, then an abnormal region will also be separated in the red coating region.

[0096] Then, shape and uniformity recognition are performed on the abnormal area. Specifically, the sunken area generally has the following characteristics: 1. The shape of the sunken area is irregular and cannot be presented as a strict circle / ellipse, etc. in the image; 2. A clear-line container image has been obtained in image data processing, but the edge of the sunken area is not obvious and irregular, and the clarity of the edge line in the image before processing is poor. Therefore, the image data processing will not recognize the edge line with poor clarity of the sunken area, and thus will not perform edge clarification processing on the edge line of the sunken area; 3. The ability to reflect light at different positions inside the sunken area is also different. Therefore, the color inside the sunken area is also uneven. Select one or several of the above features to preliminarily determine the sunken area. For example, if an abnormal area has a regular shape (having a regular shape means that the abnormal area not only has a clearly recognizable edge line, but also the shape of the area is a common geometric figure or a combination of several common geometric figures, such as a triangle, a circle, a parallelogram, etc.) and the color inside the area is uniform. Then it is considered that this area may not be a sunken area, and this area can be freely named, such as the first abnormal area set. If the edge of the abnormal area is not clear, the shape is irregular or the internal color is uneven, then this area is a sunken area, and this area is added to the first sunken area set.

[0097] Finally, in the area set obtained from the previous step of recognition, the areas that are not recognized as sunken areas are rechecked. The recheck mainly focuses on whether there are areas with a color difference greater than the second error range from other areas inside these areas. For example, if there is a depression inside 30% of the white painted part of container A, and the entire white painted part is recognized as an abnormal area. Since the entire white painted area has a regular shape and the color of most areas inside is uniform, this area passes the above steps of shape and uniformity recognition of the abnormal area. During the recheck, the inside of the white painted area will be checked again. It will be found during the recheck that there are some areas inside the white painted area with a color difference greater than the second error range from other areas. These areas are added to the second sunken area set as sunken areas.

[0098] Merge the first sunken area set and the second sunken area set to obtain all the sunken areas in the container image.

[0099] Optionally, the container deformation recognition method further includes: obtaining the distance between the ranging device and the container and the distance between the ranging device and the sunken area through the ranging device, and calculating the depression amount of the sunken area.

[0100] Since the method provided by the above embodiments does not include the relevant content of the depression distance in the analysis of the depression area. Therefore, this embodiment provides a method. By using a ranging device, such as lidar, the distance between the ranging device and the container (here, the container distance refers to the distance from the non-deformed surface of the container) and the distance between the ranging device and the depression area are obtained, and the depression amount of the depression area can be determined by subtracting the two.

[0101] Correspondingly, the third embodiment of the present application further provides a container deformation recognition device, as Figure 6 shown. The device includes:

[0102] An image processing unit 601, configured to obtain image data, perform recognition, segmentation, and edge processing on the image data, and obtain a container image with clear edges;

[0103] An identification unit 602, configured to identify and obtain the container edges and container corner points in the container image, where the container corner points are the vertices of the angles formed by each side of the container;

[0104] A calculation unit 603, configured to calculate the deviation amount between the connection line between all adjacent corner points and the container edge line between the adjacent corner points among the container corner points. When the deviation amount is greater than a preset first error range, it is determined that the container is deformed, and the adjacent corner points are the container corner points located on the same container edge line.

[0105] Optionally, calculating the deviation amount between the connection line between all adjacent corner points and the container edge line between the adjacent corner points among the container corner points includes:

[0106] Obtain all adjacent corner points among the container corner points;

[0107] Obtain the deviation part between the container edge line between all adjacent corner points and the connection line between the adjacent corner points, and the length of the perpendicular line segment from any point on the deviation part to the connection line between the adjacent corner points is greater than the first error range;

[0108] Divide the deviation part into at least one deformed edge line. Each deformed edge line obtained by the division includes a continuous container edge line with one undulation. The undulation refers to the change trend of the distance from the points on the container edge line to the connection line between the adjacent corner points, which first increases and then decreases;

[0109] Obtain the distance between each deformed edge line and the connection line between the adjacent corner points as the deviation amount.

[0110] Optionally, obtaining the distance between each deformed edge line and the connection line between the adjacent corner points as the deviation amount includes:

[0111] For each deformed edge line, determine the point with the maximum distance from the connection line between the adjacent corner points in the deformed edge line as the deformation vertex;

[0112] Calculate the length of the perpendicular line segment between each deformed vertex and the adjacent corner point as the deviation amount.

[0113] Optionally, the method further includes:

[0114] Identify the placement direction of the container according to the preset container features, where the container features include the shape of the container door;

[0115] Determine the deformed position of the container according to the container placement direction and the deviation amount.

[0116] Optionally, the method further includes:

[0117] For any container corner point, calculate the angle of the included angle where the container corner point is located. When the angle of the included angle where the container corner point is located is greater than or less than the preset included angle range, it is determined that the container is deformed.

[0118] Optionally, the method further includes:

[0119] Obtain the ambient light intensity, and determine the second error range according to the ambient light intensity;

[0120] Identify and obtain the sunken area of the container image, where the color difference between the sunken area and the area within a preset distance range around the sunken area is greater than the second error range.

[0121] Optionally, identifying and obtaining the sunken area of the container image includes:

[0122] Identify and obtain the abnormal area of the container image, where the color difference between the abnormal area and other areas in the container image is greater than the preset third error range;

[0123] Perform shape and uniformity identification on the abnormal area, and divide the abnormal area into at least two area sets. At least two area sets include the first sunken area set, and the first sunken area set does not include areas with regular shapes and / or uniform colors inside the abnormal area;

[0124] For the area sets other than the first sunken area set in the area sets, perform identification to obtain the second sunken area set, where the color difference between the areas in the second sunken area set and the other areas in the area sets other than the first sunken area set in the area sets is greater than the second error range;

[0125] Merge the first sunken area set and the second sunken area set to obtain the sunken area.

[0126] Optionally, the method further includes:

[0127] Obtain the distance between the ranging device and the container and the distance between the ranging device and the sunken area through the ranging device, and calculate the sunken amount of the sunken area.

[0128] The container deformation recognition device provided in this embodiment belongs to the same inventive concept as the container deformation recognition method provided in the above embodiments of the present application. It can execute the container deformation recognition method provided in any of the above embodiments of the present application and has the corresponding functional modules and beneficial effects for executing the method. For technical details not described in detail in this embodiment, reference may be made to the specific processing content of the container deformation recognition method embodiment provided in the above embodiments of the present application, which will not be elaborated here.

[0129] The functions implemented by the above image processing unit 601, recognition unit 602, and calculation unit 603 can be implemented by the same or different processors respectively, which is not limited in the embodiments of the present application.

[0130] It should be understood that the units in the above device can be implemented in the form of a processor calling software. For example, the device includes a processor, which is connected to a memory. Instructions are stored in the memory, and the processor calls the instructions stored in the memory to implement any of the above methods or the functions of each unit of the device. The processor can be a general-purpose processor, such as a CPU or a microprocessor, etc., and the memory can be a memory inside the device or a memory outside the device. Alternatively, the units in the device can be implemented in the form of a hardware circuit. By designing the hardware circuit, the functions of some or all of the units can be implemented. The hardware circuit can be understood as one or more processors. For example, in one implementation, the hardware circuit is an ASIC, and the functions of some or all of the above units are implemented by designing the logical relationship of the components in the circuit. Again, for example, in another implementation, the hardware circuit can be implemented by a PLD. Taking FPGA as an example, it can include a large number of logic gate circuits, and the connection relationship between the logic gate circuits is configured through a configuration file to implement the functions of some or all of the above units. All the units of the above device can be all implemented in the form of a processor calling software, or all implemented in the form of a hardware circuit, or some implemented in the form of a processor calling software, and the remaining part implemented in the form of a hardware circuit.

[0131] In the embodiments of the present application, a processor is a circuit with signal processing capabilities. In one implementation, the processor can be a circuit with instruction reading and running capabilities, such as a CPU, microprocessor, GPU, or DSP, etc.; in another implementation, the processor can implement certain functions through the logical relationship of a hardware circuit, and the logical relationship of the hardware circuit is fixed or can be reconfigured. For example, the processor is a hardware circuit implemented by an ASIC or PLD, such as an FPGA, etc. In a reconfigurable hardware circuit, the process of the processor loading a configuration document to implement the configuration of the hardware circuit can be understood as the process of the processor loading instructions to implement the functions of some or all of the above units. In addition, it can also be a hardware circuit designed for artificial intelligence, which can be understood as a type of ASIC, such as an NPU, TPU, DPU, etc.

[0132] It can be seen that each unit in the above device can be one or more processors (or processing circuits) configured to implement the above method. For example: CPU, GPU, NPU, TPU, DPU, microprocessor, DSP, ASIC, FPGA, or a combination of at least two of these processor forms.

[0133] In addition, each unit in the above device can be integrated in whole or in part, or can be independently implemented. In one implementation, these units are integrated together and implemented in the form of an SOC. The SOC can include at least one processor for implementing any of the above methods or implementing the functions of each unit of the device. The types of the at least one processor can be different. For example, it includes a CPU and an FPGA, a CPU and an artificial intelligence processor, a CPU and a GPU, etc.

[0134] The fourth embodiment of the present application also proposes an electronic device. Refer to Figure 7 As shown, the device includes:

[0135] A memory 700 and a processor 710;

[0136] Wherein, the memory 700 is connected to the processor 710 and is used to store programs;

[0137] The processor 710 is used to implement the container deformation recognition method disclosed in any of the above embodiments by running the program stored in the memory 700.

[0138] Specifically, the above electronic device may further include: a bus, a communication interface 720, an input device 730, and an output device 740.

[0139] The processor 710, the memory 700, the communication interface 720, the input device 730, and the output device 740 are interconnected through the bus. Among them:

[0140] The bus may include a path for transmitting information between various components of a computer system.

[0141] The processor 710 may be a general-purpose processor, such as a general-purpose central processing unit (CPU), a microprocessor, etc., or an application-specific integrated circuit (ASIC), or one or more integrated circuits for controlling the execution of the program of the present invention. It may also be a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components.

[0142] The processor 710 may include a main processor, and may also include a baseband chip, a modem, etc.

[0143] The memory 700 stores the program for implementing the technical solution of the present invention, and may also store an operating system and other key services. Specifically, the program may include program code, and the program code includes computer operation instructions. More specifically, the memory 700 may include a read-only memory (ROM), other types of static storage devices that can store static information and instructions, a random access memory (RAM), other types of dynamic storage devices that can store information and instructions, a disk memory, a flash memory, etc.

[0144] The input device 730 may include a device for receiving user input data and information, such as a keyboard, a mouse, a camera, a scanner, a light pen, a voice input device, a touch screen, a pedometer, or a gravity sensor, etc.

[0145] The output device 740 may include a device for allowing information to be output to the user, such as a display screen, a printer, a speaker, etc.

[0146] The communication interface 720 may include a device of any transceiver type for communicating with other devices or communication networks, such as Ethernet, a radio access network (RAN), a wireless local area network (WLAN), etc.

[0147] The processor 710 executes the program stored in the memory 700 and calls other devices, which can be used to implement each step of any one of the container deformation recognition methods provided in the above embodiments of the present application.

[0148] For the foregoing method embodiments, for the sake of simple description, they are all expressed as a series of action combinations. However, those skilled in the art should know that this application is not limited by the described action sequence, because according to this application, some steps can be performed in other sequences or simultaneously. Secondly, those skilled in the art should also know that the embodiments described in the specification are all preferred embodiments, and the actions and modules involved are not necessarily essential to this application.

[0149] It should be noted that the embodiments in this specification are all described in a progressive manner. Each embodiment focuses on the differences from other embodiments. For the same or similar parts among the embodiments, reference can be made to each other. For device embodiments, since they are basically similar to method embodiments, the description is relatively simple. For the relevant parts, reference can be made to the partial description of the method embodiments.

[0150] The steps in the methods of the embodiments of this application can be adjusted, combined, and deleted according to actual needs. The technical features recorded in each embodiment can be replaced or combined.

[0151] The modules and sub-modules in the devices and terminals in the embodiments of this application can be combined, divided, and deleted according to actual needs.

[0152] In several embodiments provided by this application, it should be understood that the disclosed terminals, devices, and methods can be implemented in other ways. For example, the terminal embodiments described above are only illustrative. For example, the division of modules or sub-modules is only a logical function division. In actual implementation, there can be other division methods. For example, multiple sub-modules or modules can be combined or integrated into another module, or some features can be ignored or not executed. Another point is that the displayed or discussed coupling, direct coupling, or communication connection to each other can be through some interfaces. The indirect coupling or communication connection of devices or modules can be in electrical, mechanical, or other forms.

[0153] The modules or sub-modules described as separate components can be or may not be physically separated. The components as modules or sub-modules can be or may not be physical modules or sub-modules, that is, they can be located in one place, or can be distributed to multiple network modules or sub-modules. Some or all of the modules or sub-modules can be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0154] In addition, in each embodiment of the present application, each functional module or sub-module can be integrated into a processing module, or each module or sub-module can exist physically alone, or two or more modules or sub-modules can be integrated into one module. The above-mentioned integrated modules or sub-modules can be implemented in the form of hardware, or in the form of software functional modules or sub-modules.

[0155] Those skilled in the art can further realize that the units and algorithm steps of each example described in combination with the embodiments disclosed in this article can be implemented by electronic hardware, computer software, or a combination of the two. To clearly illustrate the interchangeability of hardware and software, the composition and steps of each example have been generally described according to functions in the above description. Whether these functions are executed in a hardware or software manner depends on the specific application and design constraints of the technical solution. Skilled professionals can use different methods to implement the described functions for each specific application, but such implementation should not be considered to exceed the scope of this application.

[0156] The steps of the method or algorithm described in combination with the embodiments disclosed in this article can be directly implemented by hardware, software units executed by a processor, or a combination of the two. The software units can be placed in a random access memory (RAM), internal memory, read-only memory (ROM), electrically programmable ROM, electrically erasable programmable ROM, registers, hard disk, removable disk, CD-ROM, or any other form of storage medium well-known in the technical field.

[0157] Finally, it should also be noted that in this article, relational terms such as first and second are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any actual relationship or order between these entities or operations. Moreover, the term "comprising", "including" or any other variant thereof is intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements not only includes those elements, but also includes other elements not expressly listed, or also includes elements inherent to such process, method, article or device. Without further limitation, an element defined by the statement "including a..." does not exclude the existence of additional identical elements in the process, method, article or device including the said element.

[0158] The foregoing description of the disclosed embodiments enables those skilled in the art to implement or use the present application. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein can be implemented in other embodiments without departing from the spirit or scope of the present application. Thus, the present application is not intended to be limited to the embodiments shown herein but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A method for identifying container deformation, characterized in that, Including: Obtaining and processing image data to obtain a container image with clear lines; Identifying and obtaining the container side lines and container corner points in the container image, where the container corner points are the vertices of the angles formed by each side of the container; Calculating the deviation amount between the connection lines between all adjacent corner points among the container corner points and the container side lines between the adjacent corner points. When the deviation amount is greater than a preset first error range, it is determined that the container is deformed. The adjacent corner points are the container corner points located on the same container side line; The method further includes: Obtaining the ambient light intensity and determining a second error range according to the ambient light intensity; Identifying and obtaining the concave regions in the container image, where the color difference between the concave regions and the regions within a preset distance range around the concave regions is greater than the second error range; The identifying and obtaining the concave regions in the container image includes: Identifying and obtaining the abnormal regions in the container image, where the color difference between the abnormal regions and other regions in the container image is greater than a preset third error range; Identifying the shape and uniformity of the abnormal regions, dividing the abnormal regions into at least two region sets, where the at least two region sets include a first concave region set, and the first concave region set does not include the regions in the abnormal regions that have regular shapes and / or uniform colors inside the regions; Identifying the region sets other than the first concave region set in the region sets, obtaining a second concave region set, where the color difference between the regions in the second concave region set and the other regions in the region sets other than the first concave region set is greater than the second error range; Merging the first concave region set and the second concave region set to obtain the concave regions.

2. The method according to claim 1, characterized in that, The calculating the deviation amount between the connection lines between all adjacent corner points among the container corner points and the container side lines between the adjacent corner points includes: Obtaining all the adjacent corner points among the container corner points; Obtaining the deviated parts between the container side lines between all adjacent corner points and the connection lines between the adjacent corner points, where the length of the perpendicular line segment from any point on the deviated part to the connection line between the adjacent corner points is greater than the first error range; Dividing the deviated parts into at least one deformed side line segment. Each deformed side line segment obtained by the division includes a continuous container side line with one undulation. The undulation refers to the change trend of the distance from the points on the container side line to the connection line between the adjacent corner points, which first increases and then decreases; Obtaining the distance between each deformed side line segment and the connection line between the adjacent corner points as the deviation amount.

3. The method according to claim 2, characterized in that, The obtaining the distance between each deformed side line segment and the connection line between the adjacent corner points as the deviation amount includes: For each deformed side line segment, determining the point with the maximum distance from the deformed side line segment to the connection line between the adjacent corner points as the deformed vertex; Calculating the length of the perpendicular line segment between each deformed vertex and the connection line between the adjacent corner points as the deviation amount.

4. The method according to claim 1, wherein The method further includes: Identifying the placement direction of the container according to the preset container features, where the container features include the shape of the container door; Determine the deformation position of the container according to the container placement direction and the deviation amount.

5. The method according to claim 1, wherein The method further includes: For any corner point of the container, calculate the angle of the included angle where the container corner point is located. When the angle of the included angle where the container corner point is located is greater than or less than a preset included angle range, it is determined that the container is deformed.

6. The method according to claim 1, wherein The method further includes: Obtain the distance between the ranging device and the container and the distance between the ranging device and the concave area through the ranging device, and calculate the concavity of the concave area.

7. A container deformation recognition device, characterized in that, It includes: An image processing unit, configured to obtain image data, perform recognition, segmentation, and edge processing on the image data, and obtain a container image with clear edges; And configured to obtain the environmental light intensity, and determine a second error range according to the environmental light intensity; An identification unit, configured to identify and obtain the container side lines and container corner points in the container image, where the container corner points are the vertices of the included angles formed by each side of the container; And configured to identify and obtain the concave area in the container image, where the color difference between the concave area and the area within a preset distance range around the concave area is greater than the second error range; The identifying and obtaining the concave area in the container image includes: identifying and obtaining the abnormal area in the container image, where the color difference between the abnormal area and other areas in the container image is greater than a preset third error range; performing shape and uniformity recognition on the abnormal area, and dividing the abnormal area into at least two area sets, where the at least two area sets include a first concave area set, and the first concave area set does not include the areas in the abnormal area that have regular shapes and / or uniform colors inside the area; for the area sets other than the first concave area set in the area sets, perform recognition to obtain a second concave area set, where the color difference between the areas in the second concave area set and the other areas in the area sets other than the first concave area set in the area sets is greater than the second error range; merge the first concave area set and the second concave area set to obtain the concave area; A calculation unit, configured to calculate the deviation amount between the connection line between all adjacent corner points among the container corner points and the container side line between the adjacent corner points. When the deviation amount is greater than a preset first error range, it is determined that the container is deformed, and the adjacent corner points are the container corner points located on the same container side line.

8. An electronic device, characterized in that, It includes a memory and a processor; The memory is connected to the processor and is used to store programs; The electronic device runs the program in the memory through the processor to implement the container deformation recognition method according to any one of claims 1 to 6.

Images