Baggage Fast Modeling Method, Device and Storage Medium Based on Single-Frame Sampling

Abstract

The present application relates to the technical field of object detection, and discloses a method, device and storage medium for rapid modeling of luggage based on single-frame sampling. The method includes: collecting multiple depth images of the target luggage, and determining a target depth image that meets preset requirements from the multiple depth images; obtaining a first point cloud of the target depth image, and segmenting a target point cloud corresponding to the target luggage from the first point cloud; extracting edge corner points of the target luggage according to the target point cloud, and identifying a target shape of the target luggage according to the edge corner points; searching for a preset grid model matching the target shape in a preset database as an initial model of the target luggage; extracting a target pattern corresponding to the edge corner points in the target depth image, and performing texture mapping processing on the initial model by using the target pattern; calculating a target size of the target luggage according to the edge corner points, and adjusting the size of the initial model according to the target size to obtain a target three-dimensional model corresponding to the target luggage.

Description

Technical Field

[0001] This application relates to the field of object detection technology, and in particular to a fast luggage modeling method, device and storage medium based on single-frame sampling. Background Art

[0002] During the process of luggage consignment, it is usually necessary to track and detect the status of the luggage, so as to consign different luggage with different hardness and softness in different areas, and to check in real time whether the luggage is damaged, so as to ensure the safe delivery of the luggage.

[0003] Currently, the way to detect whether the luggage is damaged is usually to use three-dimensional modeling technology. By performing three-dimensional modeling on the luggage inspected in real time, the three-dimensional model after completion can be used to determine whether the luggage is damaged.

[0004] However, the existing three-dimensional modeling usually takes too long and cannot meet the real-time modeling requirements in the luggage consignment scenario. Summary of the Invention

[0005] The main purpose of this application is to provide a fast luggage modeling method, device and storage medium based on single-frame sampling, aiming to achieve fast three-dimensional modeling of luggage.

[0006] In a first aspect, this application provides a fast luggage modeling method based on single-frame sampling, including:

[0007] Collect multiple depth pictures of the target luggage, and determine the target depth picture that meets the preset requirements from the multiple depth pictures;

[0008] Obtain the first point cloud of the target depth picture, and segment the target point cloud corresponding to the target luggage from the first point cloud;

[0009] Extract the edge corner points of the target luggage according to the target point cloud, and identify the target shape of the target luggage according to the edge corner points;

[0010] Search for a preset mesh model that matches the target shape from a preset database, and use the preset mesh model as the initial model of the target luggage;

[0011] Extract the target pattern corresponding to the edge corner points from the target depth picture, and use the target pattern to perform texture mapping on the initial model;

[0012] Calculate the target size of the target luggage according to the edge corner points, and perform size adjustment on the initial model according to the target size to obtain the target three-dimensional model corresponding to the target luggage.

[0013] In a second aspect, the present application further provides a terminal device, which includes a processor, a memory, and a computer program stored on the memory and executable by the processor. When the computer program is executed by the processor, the steps of the above-mentioned rapid luggage modeling method based on single-frame sampling are implemented.

[0014] In a third aspect, the present application further provides a storage medium, on which a computer program is stored. When the computer program is executed by a processor, the steps of the above-mentioned rapid luggage modeling method based on single-frame sampling are implemented.

[0015] The present application provides a rapid luggage modeling method, device, and storage medium based on single-frame sampling. In the present application, the target shape of the target luggage is recognized through the target depth image of the target luggage, and a preset mesh model matching the target shape is searched from a preset database as the initial model of the target luggage. After texture mapping processing is performed on the initial model, the size of the initial model is adjusted using the calculated target size of the target luggage, and thus the target three-dimensional model corresponding to the target luggage is obtained. Through the technical solution provided by the present application, three-dimensional modeling can be achieved only using a single depth image meeting the preset requirements, with extremely high speed, extremely short time consumption, and small resource consumption, which can meet the computational resource requirements for motion modeling of luggage during the conveying process, and realizes rapid three-dimensional modeling of luggage. Description of the Drawings

[0016] To more clearly illustrate the technical solutions of the embodiments of the present application, the drawings required for description in the embodiments will be briefly introduced below. Obviously, the drawings in the following description are some embodiments of the present application. For those of ordinary skill in the art, other drawings can be obtained based on these drawings without creative efforts.

[0017] Figure 1 It is a schematic flowchart of the steps of a rapid luggage modeling method based on single-frame sampling provided by an embodiment of the present application;

[0018] Figure 2 It is a schematic diagram of an application scenario of the rapid luggage modeling method based on single-frame sampling provided by an embodiment of the present application;

[0019] Figure 3 It is a schematic block diagram of the structure of a terminal device provided by an embodiment of the present application.

[0020] The realization, functional characteristics, and advantages of the purpose of the present application will be further described in conjunction with the embodiments and with reference to the drawings. Detailed Embodiments

[0021] Next, the technical solutions in the embodiments of the present application will be clearly and completely described in conjunction with the accompanying drawings in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, rather than all the embodiments. All other embodiments obtained by those of ordinary skill in the art based on the embodiments in the present application without creative efforts shall fall within the protection scope of the present application.

[0022] The flowchart shown in the accompanying drawings is only an example illustration, and does not necessarily include all the content and operations / steps, nor does it necessarily need to be executed in the described order. For example, some operations / steps can also be decomposed, combined, or partially merged, so the actual execution order may change according to the actual situation. In addition, although the functional modules are divided in the device schematic diagram, in some cases, it can be different from the module division in the device schematic diagram.

[0023] The embodiments of the present application provide a method, device, and storage medium for rapid luggage modeling based on single-frame sampling.

[0024] Next, some embodiments of the present application will be described in detail in conjunction with the accompanying drawings. Without conflict, the following embodiments and the features in the embodiments can be combined with each other.

[0025] Please refer to Figure 1 , Figure 1 which is a schematic diagram of the step flow of a method for rapid luggage modeling based on single-frame sampling provided by the embodiments of the present application.

[0026] As Figure 1 shown, the method for rapid luggage modeling based on single-frame sampling includes steps S10 to S15.

[0027] Step S10: Collect multiple depth images of the target luggage, and determine the target depth image that meets the preset requirements from the multiple depth images.

[0028] It can be understood that depth images (Depth Images) are images collected by a depth camera, and are also called range images (Range Images).

[0029] As Figure 2 shown, the method for rapid luggage modeling based on single-frame sampling provided by the present application is applied to a luggage transmission system and is executed by a processor in a terminal device of the luggage transmission system.

[0030] Specifically, the luggage transmission system includes a terminal device 20, a conveying device 21, and an image acquisition device 22.

[0031] The conveying device includes a conveyor belt 210, a driving mechanism 211, a first guardrail 212, and a second guardrail 213. Among them, the first guardrail 212 and the second guardrail 213 are respectively installed on the driving mechanisms 211 on both sides of the conveyor belt 210.

[0032] The image acquisition device 22 includes a first vertical rod 220, a second vertical rod 221, a cross bar 222, and a depth camera 223. Among them, the first vertical rod 220 is installed on the driving mechanism 211 on the side close to the first guardrail 212, the second vertical rod 221 is installed on the driving mechanism 211 on the side close to the second guardrail 213, the first end of the cross bar 222 is connected to the end of the first vertical rod 220 far from the driving mechanism 211, and the second end of the cross bar 222 far from the first end is connected to the end of the second vertical rod 221 far from the driving mechanism 211.

[0033] The depth camera 223 is installed at the middle position on the side of the cross bar 222 close to the conveyor belt 210, and the image acquisition angle of the depth camera 223 is perpendicular to the plane where the conveyor belt 210 is located.

[0034] The terminal device 20 is electrically connected to the depth camera 223 and is used to receive and process the depth pictures collected by the depth camera 223.

[0035] When the target luggage 23 is placed on the conveyor belt 210, the conveyor belt 210 drives the target luggage 23 to move in a preset direction. During the movement of the target luggage 23, the depth camera 223 continuously collects depth pictures of the target luggage 23 to obtain multiple depth pictures.

[0036] In some embodiments, among the collected depth pictures, when the target luggage 23 is at the central position of the depth picture, the depth picture meets the preset requirements, and this depth picture is selected as the target depth picture. Or, when the target luggage 23 completely enters the shooting area of the depth camera 223, the depth picture collected by the depth camera 223 meets the preset requirements, and this depth picture is selected as the target depth picture. It can be understood that the target depth picture at least includes the picture information of the upper surface 231 of the target luggage 23.

[0037] Step S11: Obtain the first point cloud of the target depth picture, and segment the target point cloud corresponding to the target luggage from the first point cloud.

[0038] It can be understood that there is a corresponding mapping relationship between the target depth image and the first point cloud. According to the attribute parameters of the depth camera 223 that collects the target depth image, the target depth image can be converted into point cloud data to obtain the first point cloud. Among them, the first point cloud not only contains the point cloud of the target luggage 23, but also includes the point cloud of the background part other than the target luggage 23. The point cloud corresponding to the target luggage 23 is segmented from the first point cloud to obtain the target point cloud.

[0039] In some embodiments, segmenting the target point cloud corresponding to the target luggage from the first point cloud includes:

[0040] Extracting the point cloud whose projection area is within a preset horizontal area range from the first point cloud to obtain a second point cloud;

[0041] Extracting the point cloud whose depth value is within a preset depth range from the second point cloud to obtain the target point cloud corresponding to the target luggage.

[0042] It can be understood that in this application, the distance between the depth camera 223 responsible for collecting the target depth image and the conveyor belt 210 responsible for carrying and moving the target luggage 23 is fixed. Moreover, the depth camera 223 is fixedly installed in the air area above the conveyor belt 210. On this basis, the image acquisition area range of the depth camera 223 is also fixed.

[0043] In addition, the image acquisition area range of the depth camera 223 is also larger than that of the target luggage 23. In the depth image collected by the depth camera 223, it will not only include the image elements of the target luggage 23, but also include the image elements such as the conveyor belt 210, the first guardrail 212, and the second guardrail 213 other than the target luggage 23. And the distance between the target luggage 23 and the depth camera 223 is necessarily closer than the distance between the conveyor belt 210 carrying the target luggage 23 and the depth camera 223.

[0044] Therefore, a horizontal area range and a depth range can be preset to extract from the first point cloud the point cloud whose projection area of the target luggage 23 is within the preset horizontal area range and whose depth value is within the preset depth range, so as to obtain the target point cloud of the target luggage 23, where the depth value corresponds to the z-axis coordinate value in the three-dimensional coordinates of the point cloud.

[0045] In some embodiments, segmenting the target point cloud corresponding to the target luggage from the first point cloud includes:

[0046] Extracting the point cloud of the comparison depth image to obtain the comparison point cloud, where the comparison depth image is an empty-load depth image whose shooting angle corresponds to the acquisition angle and whose picture elements do not include the target luggage;

[0047] Remove the point cloud in the first point cloud that matches the comparison point cloud to obtain the target point cloud corresponding to the target luggage.

[0048] It can be understood that when there is no luggage on the conveyor belt 210, the conveyor belt 210 is in an empty state. At this time, the depth image collected by the depth camera 223 is an empty-load depth image, that is, the comparison depth image. The point cloud extracted from the comparison depth image is the comparison point cloud.

[0049] Remove the point cloud in the first point cloud whose three-dimensional coordinates and color values are the same as or similar to those of the comparison point cloud, and the target point cloud corresponding to the target luggage 23 can be obtained.

[0050] Step S12: Extract the edge corner points of the target luggage according to the target point cloud, and identify the target shape of the target luggage according to the edge corner points.

[0051] It can be understood that the target luggage 23 has a shape. The point cloud corresponding to the edges and corners of the target luggage 23 is obtained from the target point cloud, that is, the edge corner points of the target luggage 23 are obtained. After determining the edge corner points of the target luggage 23, the shape of the target luggage 23 can be identified through each edge corner point to obtain the target shape.

[0052] In some embodiments, the extracting the edge corner points of the target luggage according to the target point cloud includes:

[0053] Project the target point cloud onto the bearing plane bearing the target luggage to obtain a projected point cloud;

[0054] Generate a binary mask image according to the projected point cloud;

[0055] Determine the luggage image area of the target luggage according to the binary mask image, and obtain a matching rectangle that matches the luggage image area, where the matching rectangle is the smallest rectangle that can enclose the luggage image area;

[0056] When the matching value between the matching rectangle and the luggage image area reaches a preset value, perform back-projection on the matching rectangle to obtain the edge corner points corresponding to the four corner points of the matching rectangle from the target point cloud.

[0057] It can be understood that the plane where the conveyor belt 210 bearing the target luggage 23 is located is the bearing plane. Projecting the target point cloud onto the bearing plane, the obtained point cloud is the projected point cloud.

[0058] Using the projected point cloud, a binary mask image can be generated. Specifically, the points belonging to the projected point cloud are set to white, and the points not belonging to the projected point cloud are set to black. In the obtained binary mask image, the area corresponding to the white points is the image area of the target luggage 23, that is, the luggage image area; correspondingly, the area corresponding to the black points represents the other areas except the luggage image area.

[0059] Obtain the smallest rectangle that can enclose the luggage image area, that is, obtain the matching rectangle. It can be understood that the luggage image area is enclosed within the matching rectangle. When the ratio of the area of the white area part in the matching rectangle to the area of the matching rectangle reaches a preset value, it indicates that the luggage image area is generally rectangular, that is, the target luggage 23 is very likely to be a cuboid shape such as a suitcase. At this time, the matching rectangle can be back-projected to obtain the point cloud corresponding to the four corner points of the matching rectangle from the target point cloud as the top corner points of the target luggage 23, that is, obtain the four edge corner points of the target luggage 23.

[0060] In some embodiments, when the matching value between the matching rectangle and the luggage image area does not reach the preset value, it indicates that the luggage image area is not rectangular and may be other regular shapes such as trapezoids, pentagons, hexagons, etc. At this time, other regular shapes can be used one by one to calculate the matching degree with the luggage image area to determine the edge corner points of the target luggage 23 and infer the shape of the target luggage 23.

[0061] In some embodiments, before determining the luggage image area of the target object according to the binary mask image, it further includes:

[0062] Input the preprocessed binary mask image into a preset image model to obtain the processed binary mask image, where the preset image model is used to perform noise reduction and hole removal processing on the binary mask image;

[0063] Alternatively, use a morphological algorithm to process the preprocessed binary mask image to remove the holes and noise in the binary mask image to obtain the processed binary mask image.

[0064] It can be understood that after removing the holes and noise in the binary mask image, the shape of the target luggage 23 can be better determined through the processed binary mask image.

[0065] In some embodiments, identifying the target shape of the target luggage according to the edge corner points includes:

[0066] Judge whether each of the edge corner points is in the same plane according to the three-dimensional coordinates corresponding to each of the edge corner points;

[0067] When all the edge corner points are in the same plane, connect all the edge corner points to obtain an enclosed area;

[0068] Identify the target shape of the target luggage according to the enclosed area.

[0069] It can be understood that when the z-axis coordinates in the three-dimensional coordinates corresponding to all the edge corner points are the same or approximately the same, it can be determined that all the edge corner points are in the same plane. Or, even if there are differences in the z-axis coordinates in the three-dimensional coordinates corresponding to all the edge corner points, it is also possible to judge whether all the edge corner points are in the same plane according to the three-dimensional coordinates corresponding to all the edge corner points.

[0070] Exemplarily, assume that there are four edge corner points A, B, C, and D, and the x-axis coordinates, y-axis coordinates, and z-axis coordinates of the four edge corner points A, B, C, and D are {0, 0, 100}, {0, 50, 100}, {50, 0, 90}, and {50, 50, 90} respectively. It can be seen that although the z-axis coordinates of A and B are different from the z-axis coordinates of C and D, the distance between the coordinates of A and C is the same as the distance between the coordinates of B and D, and the distance between the coordinates of A and B is the same as the distance between the coordinates of C and D. For this situation, all the edge corner points are also in the same plane.

[0071] When all the edge corner points are in the same plane, the shape of the target luggage 23 can be judged according to the enclosed area obtained by connecting all the edge corner points. Exemplarily, assume that the enclosed area is a rectangle, and the z-axis coordinates corresponding to all the edge corner points forming the enclosed area are the same, then it can be judged that the target luggage 23 is a cuboid.

[0072] Step S13: Search for a preset grid model matching the target shape from a preset database, and use the preset grid model as the initial model of the target luggage.

[0073] It can be understood that various pre-constructed grid models of different shapes are stored in the database, and a grid model matching the target shape is searched from the database to be used as the initial model of the target luggage 23.

[0074] Exemplarily, assume that a grid model of a cuboid shape and a grid model of a cylinder shape are stored in the database. Assume that the target shape is a rectangle, then the grid model of the cuboid shape is the preset grid model matching the target shape, and this grid model is obtained as the initial model of the target luggage 23.

[0075] Step S14: Extract the target pattern corresponding to the edge corner point from the target depth image, and use the target pattern to perform texture mapping processing on the initial model.

[0076] It can be understood that after determining the edge corner points of the target luggage 23, the target patterns corresponding to the edge corner points can be extracted from the target depth image according to the three-dimensional coordinates of each edge corner point, and the initial model can be textured using the target patterns.

[0077] Among them, MVS (multi-view stereo) is a three-dimensional reconstruction algorithm, which includes point cloud extraction, three-dimensional reconstruction, and texture mapping algorithms. In some embodiments, after obtaining the target pattern, the texture mapping algorithm in the MVS algorithm can be used to perform texture mapping on the initial model, or other methods can be selected for texture mapping according to the situation, which is not limited herein.

[0078] In some embodiments, the extracting the target pattern corresponding to the edge corner point in the target depth image includes:

[0079] Obtaining the pattern corresponding to the enclosed area from the target depth image to obtain the target pattern.

[0080] It can be understood that the enclosed area is the area obtained by connecting each edge corner point, and the target pattern is the pattern corresponding to the enclosed area extracted from the target depth image.

[0081] Step S15: Calculate the target size of the target luggage using the edge corner points, and adjust the size of the initial model according to the target size to obtain the target three-dimensional model corresponding to the target luggage.

[0082] It can be understood that the edge corner points obtained in the above steps are the top edge corner points of the target luggage 23, and the top size of the target luggage 23 can be calculated according to each edge corner point.

[0083] In addition, the distance between the depth camera 223 and the conveyor belt 210 carrying the target luggage 23 is fixed. By subtracting the distance between the edge corner point and the conveyor belt 210 from the distance between the depth camera 223 and the conveyor belt 210, the height of the target luggage 23 can be calculated.

[0084] After calculating the top size and height of the target luggage 23, the target size of the target luggage 23 can be obtained, and the initial model can be stretched or compressed according to the target size. After completing the size adjustment, the target three-dimensional model corresponding to the target luggage 23 is obtained.

[0085] In some embodiments, the calculating the target size of the target luggage using the edge corner points includes:

[0086] When the target shape is a rectangle, selecting one of the edge corner points as a reference corner point;

[0087] Select the corner point closest to the reference corner point from each of the edge corner points as the width measurement corner point, and determine the width of the target luggage according to the distance between the reference corner point and the width measurement corner point;

[0088] Select the corner point farthest from the width measurement corner point from each of the edge corner points as the length measurement corner point, and determine the length of the target luggage according to the distance between the reference corner point and the length measurement corner point;

[0089] Obtain the plane point cloud of the bearing plane from the first point cloud, and select height measurement points from the plane point cloud;

[0090] Determine the height of the target luggage according to the depth distance between the reference corner point and the height measurement point.

[0091] It can be understood that when the target shape is a rectangle, there are four determined edge corner points, corresponding to the four corner points of the rectangle respectively. According to the distances between the four corner points, the width and length of the target luggage 23 can be determined; according to the distances between the four corner points and the bearing plane, the height of the target luggage 23 can be determined. After obtaining the length, width, and height of the target luggage 23, the target size of the target luggage 23 is obtained.

[0092] In some embodiments, the method further includes:

[0093] When the preset mesh model matching the target shape does not exist in the preset database, extract the voxels of the target luggage according to the target point cloud, and construct a three-dimensional mesh model according to the voxels;

[0094] Texture the three-dimensional mesh model with the target pattern to obtain the target three-dimensional model.

[0095] It can be understood that when the preset mesh model matching the target shape does not exist in the database, it is impossible to generate the three-dimensional mesh model of the target luggage 23 using the pre-constructed mesh model. At this time, it is necessary to extract voxels according to the target point cloud, construct a three-dimensional mesh model according to the voxels, and then perform texture mapping on the three-dimensional mesh model to obtain the target three-dimensional model corresponding to the target luggage 23.

[0096] Specifically, the TSDF (truncated signed distance function) algorithm can be used to calculate the TSDF values of each voxel, and then the MC (Marching Cube) algorithm can be used to extract the surface of the target luggage 23 based on the TSDF values of each voxel, so as to obtain the three-dimensional mesh model corresponding to the target luggage 23. And the texture mapping algorithm in the MVS algorithm is used to implement the texture mapping process of the three-dimensional mesh model, and the target three-dimensional model of the target luggage 23 is obtained.

[0097] It can be understood that in this embodiment, although a pre-constructed mesh model cannot be used, since the data to be processed for generating the three-dimensional mesh model by various algorithms is only the target point cloud, the time consumed for generating the three-dimensional mesh model is extremely short.

[0098] In this application, the target shape of the target luggage is recognized through the target depth image of the target luggage, and a preset mesh model matching the target shape is searched from a preset database as the initial model of the target luggage. After texture mapping processing on the initial model, the initial model is size-adjusted by using the calculated target size of the target luggage, and thus the target three-dimensional model corresponding to the target luggage is obtained. Through the technical solution provided by this application, three-dimensional modeling can be achieved only by using a single depth image that meets the preset requirements, with extremely high speed, extremely short time consumption, small resource consumption, and can meet the computational resource requirements for motion modeling of luggage during transmission, realizing fast three-dimensional modeling of luggage.

[0099] In addition, the luggage rapid modeling method based on single-frame sampling provided by this application is applied to a luggage transmission system as Figure 2 shown. This luggage transmission system has a simple structure and does not require controlling the depth camera 223 to move around the target luggage to collect depth images; moreover, this luggage transmission system has a low cost, and only one depth camera 223 can be used to collect the target depth map that meets the preset requirements, realizing fast three-dimensional modeling of luggage on low-cost low-end embedded hardware.

[0100] In some embodiments, this luggage transmission system further includes a model display module, and the target three-dimensional model obtained by using the luggage rapid modeling method based on single-frame sampling provided by this application can be displayed through the model display module.

[0101] As Figure 3 shown, this terminal device 301 includes a processor 3011, a memory, and a network interface connected through a system bus. Among them, the memory may include a storage medium 3012 and an internal memory 3015, and the storage medium 3012 may be non-volatile or volatile.

[0102] The storage medium 3012 can store an operating system and a computer program. The computer program includes program instructions that, when executed, can cause the processor 3011 to execute any one of the rapid luggage modeling methods based on single-frame sampling.

[0103] The processor 3011 is used to provide computing and control capabilities to support the operation of the entire terminal device.

[0104] The internal memory 3015 provides an environment for the operation of the computer program in the storage medium 3012. When the computer program is executed by the processor 3011, it can cause the processor 3011 to execute any one of the rapid luggage modeling methods based on single-frame sampling.

[0105] This network interface is used for network communication, such as sending assigned tasks, etc. Those skilled in the art can understand that Figure 3 the structure shown in Figure 3 is only a block diagram of some structures related to the solution of this application, and does not constitute a limitation on the terminal device to which the solution of this application is applied. The specific terminal device may include more or fewer components than those shown in the figure, or combine some components, or have different component arrangements.

[0106] It should be understood that the processor 3011 can be a central processing unit (CPU), and the processor 3011 can also be other general-purpose processors, digital signal processors (DSPs), application specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. Among them, the general-purpose processor can be a microprocessor or the processor can also be any conventional processor, etc.

[0107] Among them, in some embodiments, the processor 3011 is used to run the computer program stored in the memory to implement the following steps:

[0108] Collect multiple depth images of the target luggage, and determine a target depth image that meets the preset requirements from the multiple depth images;

[0109] Obtain the first point cloud of the target depth image, and segment the target point cloud corresponding to the target luggage from the first point cloud;

[0110] Extract the edge corner points of the target luggage according to the target point cloud, and identify the target shape of the target luggage according to the edge corner points;

[0111] Search for a preset mesh model that matches the target shape from a preset database, and use the preset mesh model as the initial model of the target luggage;

[0112] Extract the target pattern corresponding to the edge corner points from the target depth image, and use the target pattern to perform texture mapping on the initial model;

[0113] Calculate the target size of the target luggage using the edge corner points, and adjust the size of the initial model according to the target size to obtain the target three-dimensional model corresponding to the target luggage.

[0114] In some embodiments, the processor 3011 is further configured to implement:

[0115] When there is no preset mesh model that matches the target shape in the preset database, extract the voxels of the target luggage from the target point cloud, and construct a three-dimensional mesh model according to the voxels;

[0116] Perform texture mapping on the three-dimensional mesh model using the target pattern to obtain the target three-dimensional model.

[0117] In some embodiments, when the processor 3011 segments the target point cloud corresponding to the target luggage from the first point cloud, it is configured to implement:

[0118] Extract the point cloud whose projection area is within a preset horizontal area range from the first point cloud to obtain a second point cloud;

[0119] Extract the point cloud whose depth value is within a preset depth range from the second point cloud to obtain the target point cloud corresponding to the target luggage.

[0120] In some embodiments, when the processor 3011 segments the target point cloud corresponding to the target luggage from the first point cloud, it is configured to implement:

[0121] Extract the point cloud of the comparison depth image to obtain a comparison point cloud, where the comparison depth image is an empty-load depth image whose camera angle corresponds to the acquisition angle and whose picture elements do not include the target luggage;

[0122] Remove the point cloud in the first point cloud that matches the comparison point cloud to obtain the target point cloud corresponding to the target luggage.

[0123] In some embodiments, when the processor 3011 extracts the edge corner points of the target luggage from the target point cloud, it is configured to implement:

[0124] Project the target point cloud onto the bearing plane bearing the target luggage to obtain a projected point cloud;

[0125] Generate a binary mask image according to the projected point cloud;

[0126] Determine the luggage image area of the target luggage according to the binary mask image, and obtain a matching rectangle that matches the luggage image area, where the matching rectangle is the smallest rectangle that can enclose the luggage image area;

[0127] When the matching value between the matching rectangle and the luggage image area reaches a preset value, perform back-projection on the matching rectangle to obtain edge corner points corresponding to the four corner points of the matching rectangle from the target point cloud.

[0128] In some embodiments, when the processor 3011 identifies the target shape of the target luggage according to the edge corner points, it is used to implement:

[0129] Judge whether each edge corner point is in the same plane according to the three-dimensional coordinates corresponding to each edge corner point;

[0130] When each edge corner point is in the same plane, connect each edge corner point to obtain an enclosed area;

[0131] Identify the target shape of the target luggage according to the enclosed area.

[0132] In some embodiments, when the processor 3011 extracts the target pattern corresponding to the edge corner point in the target depth picture, it is used to implement:

[0133] Obtain the pattern corresponding to the enclosed area from the target depth picture to obtain the target pattern.

[0134] In some embodiments, when the processor 3011 calculates the target size of the target luggage by using the edge corner points, it is used to implement:

[0135] When the target shape is a rectangle, select one of the edge corner points as a reference corner point;

[0136] Screen the corner point closest to the reference corner point from each edge corner point as the width measurement corner point, and determine the width of the target luggage according to the distance between the reference corner point and the width measurement corner point;

[0137] Screen the corner point farthest from the width measurement corner point from each edge corner point as the length measurement corner point, and determine the length of the target luggage according to the distance between the reference corner point and the length measurement corner point;

[0138] Obtain the plane point cloud of the bearing plane from the first point cloud, and select height measurement points from the plane point cloud;

[0139] Determine the height of the target luggage according to the depth distance between the reference corner point and the height measurement point.

[0140] It should be noted that those skilled in the art can clearly understand that for the convenience and conciseness of description, the specific working process of the above-described terminal device can refer to the corresponding process in the foregoing embodiment of the method for rapid luggage modeling based on single-frame sampling, which will not be elaborated herein.

[0141] The embodiment of the present application also provides a storage medium, which is a computer-readable storage medium. A computer program is stored on the computer-readable storage medium. The computer program includes program instructions, and the method implemented when the program instructions are executed can refer to each embodiment of the method for rapid luggage modeling based on single-frame sampling of the present application.

[0142] Among them, the computer-readable storage medium may be an internal storage unit of the terminal device described in the foregoing embodiment, such as the hard disk or memory of the terminal device. The computer-readable storage medium may also be an external storage device of the terminal device, such as a plug-in hard disk equipped on the terminal device, a Smart Media Card (SMC), a Secure Digital (SD) card, a Flash Card, etc.

[0143] It should be understood that the terms used in the specification of the present application are only for the purpose of describing specific embodiments and are not intended to limit the present application. As used in the specification of the present application and the appended claims, unless the context clearly indicates otherwise, the singular forms "a", "an" and "the" are intended to include the plural forms.

[0144] It should also be understood that the term "and / or" used in the specification of the present application and the appended claims refers to any combination and all possible combinations of one or more of the associated listed items, and includes these combinations. It should be noted that in this article, the term "comprises", "comprising" or any other variation thereof is intended to cover a non-exclusive inclusion, such that a process, method, article or system comprising a series of elements includes not only those elements, but also other elements not expressly listed, or elements inherent to such process, method, article or system. Without further limitation, an element defined by the statement "comprising a..." does not exclude the presence of additional identical elements in the process, method, article or system comprising the element.

[0145] The serial numbers of the embodiments of the present application above are only for description and do not represent the advantages or disadvantages of the embodiments. As described above, it is only the specific implementation manner of the present application, but the protection scope of the present application is not limited thereto. Any person skilled in the art within the technical scope disclosed by the present application can easily think of various equivalent modifications or substitutions, and these modifications or substitutions should all be covered within the protection scope of the present application. Therefore, the protection scope of the present application shall be subject to the protection scope of the claims.

Claims

1. A rapid luggage modeling method based on single-frame sampling, characterized in that, The method includes: Collecting multiple depth images of the target luggage, and determining a target depth image that meets preset requirements from the multiple depth images; Obtaining a first point cloud of the target depth image, and segmenting a target point cloud corresponding to the target luggage from the first point cloud; Projecting the target point cloud onto a bearing plane that bears the target luggage to obtain a projected point cloud; generating a binary mask image according to the projected point cloud; determining a luggage image area of the target luggage according to the binary mask image, and obtaining a matching rectangle that matches the luggage image area, where the matching rectangle is the smallest rectangle that can enclose the luggage image area; when a matching value between the matching rectangle and the luggage image area reaches a preset value, performing back-projection on the matching rectangle to obtain edge corner points corresponding to four corner points of the matching rectangle from the target point cloud, and identifying a target shape of the target luggage according to the edge corner points; Searching a preset database for a preset mesh model that matches the target shape, and using the preset mesh model as an initial model of the target luggage; Extracting a target pattern corresponding to the edge corner points from the target depth image, and performing texture mapping on the initial model by using the target pattern; Calculating a target size of the target luggage by using the edge corner points, and performing size adjustment on the initial model according to the target size to obtain a target three-dimensional model corresponding to the target luggage.

2. The method according to claim 1, characterized in that, The method further includes: When there is no preset mesh model that matches the target shape in the preset database, extracting voxels of the target luggage according to the target point cloud, and constructing a three-dimensional mesh model according to the voxels; Performing texture mapping on the three-dimensional mesh model by using the target pattern to obtain the target three-dimensional model.

3. The method according to claim 2, characterized in that, The segmenting the target point cloud corresponding to the target luggage from the first point cloud includes: Extracting point clouds whose projection areas are within a preset horizontal area range from the first point cloud to obtain a second point cloud; Extracting point clouds whose depth values are within a preset depth range from the second point cloud to obtain the target point cloud corresponding to the target luggage.

4. The method according to claim 2, wherein The segmenting the target point cloud corresponding to the target luggage from the first point cloud includes: Extracting point clouds of a comparison depth image to obtain comparison point clouds, where the comparison depth image is an empty-load depth image whose camera angle corresponds to the acquisition angle and whose image elements do not include the target luggage; Removing the point clouds in the first point cloud that match the comparison point clouds to obtain the target point cloud corresponding to the target luggage.

5. The method according to claim 1, characterized in that The identifying the target shape of the target luggage according to the edge corner points includes: Judging whether each of the edge corner points is on the same plane according to three-dimensional coordinates corresponding to each of the edge corner points; When each of the edge corner points is on the same plane, connecting each of the edge corner points to obtain an enclosed area; Identifying the target shape of the target luggage according to the enclosed area.

6. The method according to claim 5, wherein The extracting the target pattern corresponding to the edge corner points from the target depth image includes: Obtaining a pattern corresponding to the enclosed area from the target depth image to obtain the target pattern.

7. The method according to claim 6, wherein Calculating the target size of the target luggage by using the edge corner points includes: When the target shape is rectangular, select one of the edge corner points as the reference corner point; Screen the corner point closest to the reference corner point from each of the edge corner points as the width measurement corner point, and determine the width of the target luggage according to the distance between the reference corner point and the width measurement corner point; Screen the corner point farthest from the width measurement corner point from each of the edge corner points as the length measurement corner point, and determine the length of the target luggage according to the distance between the reference corner point and the length measurement corner point; Obtain the plane point cloud of the bearing plane from the first point cloud, and select height measurement points from the plane point cloud; Determine the height of the target luggage according to the depth distance between the reference corner point and the height measurement point.

8. A terminal device, characterized in that, The terminal device includes a processor, a memory, a computer program stored on the memory and executable by the processor, and a data bus for realizing the connection communication between the processor and the memory. When the computer program is executed by the processor, the steps of the luggage fast modeling method based on single-frame sampling according to any one of claims 1 to 7 are realized.

9. A storage medium for computer-readable storage, characterized in that, The storage medium stores one or more programs, and the one or more programs can be executed by one or more processors to realize the steps of the luggage fast modeling method based on single-frame sampling according to any one of claims 1 to 7.

Images