Object recognition method, device, equipment and storage medium based on three-dimensional data

Abstract

The present application relates to a method, apparatus, computer equipment, and storage medium for object recognition based on three-dimensional data. The method determines a search area containing a target object within an image of objects to be depalletized and depalletized based on the unit height of the objects and the distribution of the stacking height. A target template image is determined based on a preset template image of the object to be transferred within the target object. The position of the object to be transferred is then determined based on the search area and the target template image. Compared to traditional object recognition based on two-dimensional images, this method improves object recognition efficiency by adjusting the proportions of the object's template image in combination with the stacking height information and identifying the position of the object to be transferred based on the adjusted template image.

Description

Technical Field

[0001] The present application relates to the field of industrial computer technology, and in particular to a method, apparatus, computer equipment, storage medium, and computer program product for object recognition based on three-dimensional data. Background Art

[0002] With the advancement of industrial technology, cargo handling has gradually been replaced by automation and mechanization. Depalletizing in the industrial field requires the use of industrial robots. Currently, industrial robots typically use two-dimensional images to identify the goods before depalletizing. However, depalletizing using two-dimensional image recognition can reduce recognition efficiency due to the varying postures of the goods.

[0003] Therefore, the current method for identifying objects in palletizing has the defect of low efficiency. Summary of the Invention

[0004] Based on this, it is necessary to provide a method, device, computer equipment, computer-readable storage medium and computer program product based on three-dimensional data that can improve recognition efficiency in order to address the above technical problems.

[0005] In a first aspect, the present application provides a method for object recognition based on three-dimensional data, the method comprising:

[0006] Obtaining images of pallets to be disassembled and depalletized and information on the height distribution of pallets to be disassembled and depalletized;

[0007] Obtaining the unit height corresponding to the objects in the to-be-depalletized object;

[0008] Determining a search area containing a target object in the image of the to-be-depalletized image based on the unit height and the palletizing height distribution information; the target object includes one or more objects corresponding to a target height in the to-be-depalletized image;

[0009] Acquire a preset template image of the object to be transferred in the target object;

[0010] Based on the preset template image, a target template image of the object to be transferred is obtained; the ratio of the target template image to the preset template image is determined according to the target height;

[0011] The position of the object to be transferred is determined according to the search area and the target template image.

[0012] In one embodiment, the step of obtaining the pallet height distribution information of the pallets to be depalletized includes:

[0013] Obtain the point cloud image corresponding to the objects to be disassembled and depalletized;

[0014] The height distribution information of the pallet to be depalletized is determined according to the height information of each point in the point cloud image.

[0015] In one embodiment, determining a search area containing a target object in the to-be-depalletized image based on the unit height and the palletizing height distribution information includes:

[0016] Mapping the palletizing height distribution information to the depalletizing image according to the position of the depalletizing image to be depalletized in the depalletizing image to obtain regional height distribution information of the depalletizing image to be depalletized;

[0017] According to the unit height and the regional height distribution information, an area of ​​a target height in the to-be-depalletized image is determined, and the search area is obtained according to the area of ​​the target height.

[0018] In one embodiment, determining the area of ​​target height in the to-be-depalletized image according to the unit height and the area height distribution information includes:

[0019] Obtaining the maximum height in the height distribution information of the area;

[0020] The difference between the maximum height and the unit height is obtained, and the area of ​​the target height in the to-be-depalletized image is determined according to the difference and the maximum value.

[0021] In one embodiment, obtaining a target template image of the object to be transferred based on the preset template image includes:

[0022] Determining a corresponding zoom ratio according to a target height corresponding to the search area; wherein the zoom ratio is positively correlated with the target height;

[0023] Based on the zoom ratio, the scale of the preset template image is adjusted, and based on the preset template image after the scale is adjusted, the target template image is obtained.

[0024] In one embodiment, determining the position of the object to be transferred based on the search area and the target template image includes:

[0025] Obtain the search area at multiple rotation angles;

[0026] The position of the object to be transferred is determined according to the search area and the target template image at the multiple rotation angles.

[0027] In one embodiment, obtaining a target template image of the object to be transferred based on the preset template image includes:

[0028] Determining a matching ratio range according to the target height;

[0029] Obtaining a target template image according to the ratio range and a preset template image; the target template image includes template images of different ratios within the ratio range;

[0030] Determining the position of the object to be transferred based on the search area and the target template image includes:

[0031] The position of the object to be transferred is determined by searching the search area using the template images of different proportions.

[0032] In one embodiment, searching the search area using the template images of different scales to determine the position of the object to be transferred includes:

[0033] Performing a search in the search area using the template images of different proportions;

[0034] The position of the object to be transferred is determined according to the matching position of the template image that meets the preset matching probability in the search area.

[0035] In one embodiment, the search area includes search areas at multiple rotation angles; and searching in the search area using the template images of different proportions includes:

[0036] Searching the search areas at the multiple rotation angles in sequence using the template images of different proportions;

[0037] When searching for template images one by one in the search area at the current rotation angle, if the matching probability of a template image is detected to be lower than the preset stop search probability, the search area at the next rotation angle is directly entered to search for template images one by one.

[0038] In one embodiment, determining the position of the object to be transferred based on the matching position of the template image that meets the preset matching probability in the search area includes:

[0039] If there are multiple template images that meet the preset matching probability, the position of the object to be transferred is determined according to the matching position of the template image with the highest matching probability among the multiple template images in the search area.

[0040] In a second aspect, the present application provides an object recognition device based on three-dimensional data, the device comprising:

[0041] A first acquisition module is used to acquire an image of the pallet to be disassembled and depalletized and information on the height distribution of the pallet to be disassembled and depalletized;

[0042] A second acquisition module is used to obtain the unit height corresponding to the objects in the to-be-depalletized objects;

[0043] a determination module configured to determine, based on the unit height and the palletizing height distribution information, a retrieval area in the image of the to-be-depalletized image containing a target object; the target object comprises one or more objects in the to-be-depalletized image corresponding to a target height;

[0044] A third acquisition module is used to acquire a preset template image of the object to be transferred in the target object;

[0045] an adjustment module, configured to obtain a target template image of the object to be transferred based on the preset template image; wherein the ratio of the target template image to the preset template image is determined according to the target height;

[0046] The recognition module is used to determine the position of the object to be transferred based on the search area and the target template image.

[0047] In a third aspect, the present application provides a computer device comprising a memory and a processor, wherein the memory stores a computer program, and the processor implements the steps of the above method when executing the computer program.

[0048] In a fourth aspect, the present application provides a computer-readable storage medium having a computer program stored thereon, which implements the steps of the above method when executed by a processor.

[0049] In a fifth aspect, the present application provides a computer program product, comprising a computer program, which implements the steps of the above method when executed by a processor.

[0050] The above-described three-dimensional data-based object recognition method, apparatus, computer device, storage medium, and computer program product determine a search area within an image of the objects to be depalletized and palletized based on the unit height of the objects and the distribution of the pallet heights. A target template image is determined based on a preset template image of the object to be transferred within the target object, and the position of the object to be transferred is determined based on the search area and the target template image. Compared to traditional object recognition based on two-dimensional images, this solution improves object recognition efficiency by adjusting the scale of the object template image incorporating pallet height information and identifying the position of the object to be transferred based on the adjusted template image. BRIEF DESCRIPTION OF THE DRAWINGS

[0051] Figure 1 A diagram illustrating an application environment of a method for object recognition based on three-dimensional data in one embodiment;

[0052] Figure 21 is a flow chart of a method for object recognition based on three-dimensional data in one embodiment;

[0053] Figure 3 A schematic diagram of determining a search area in one embodiment;

[0054] Figure 4 is a schematic diagram of image ratios in one embodiment;

[0055] Figure 5 is a structural block diagram of an object recognition device based on three-dimensional data in one embodiment;

[0056] Figure 6 FIG. 1 is a diagram showing the internal structure of a computer device in one embodiment. DETAILED DESCRIPTION

[0057] In order to make the purpose, technical solutions and advantages of this application more clear, the following further describes this application in detail with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain this application and are not intended to limit this application.

[0058] The object recognition method based on three-dimensional data provided in the embodiment of the present application can be applied to Figure 1 In the application environment shown, the terminal communicates with an image acquisition device via a network. The image acquisition device can capture images of the objects to be depalletized, and the terminal can acquire the stereo and planar images captured by the image acquisition device. The terminal then performs object recognition based on the received images. The terminal can be, but is not limited to, various personal computers, laptops, and tablet computers. The image acquisition device can be located at the site where the objects to be depalletized are located.

[0059] In one embodiment, Figure 2 As shown, a method for object recognition based on three-dimensional data is provided. Figure 1 The following steps are used as an example to illustrate the terminal in the figure:

[0060] Step S202: obtaining an image of the pallets to be depalletized and the pallet height distribution information of the pallets to be depalletized.

[0061] A pallet can be a pile of neatly stacked objects, i.e., a pallet containing multiple layers of stacked objects. In the industrial field, industrial robots are required to disassemble the pallet, i.e., to transfer the objects within the pallet. During the disassembly process, a terminal can obtain an image of the pallet to be disassembled, captured by an image acquisition device. The image acquisition device can be installed in an industrial robot at the industrial site where the pallet is located. The industrial robot can use the image acquisition device to capture an image of the pallet from a bird's-eye view, thereby obtaining an image of the pallet to be disassembled. Furthermore, since the objects are manually placed at different angles, when objects are stacked in multiple layers, the sizes of the images of objects in different layers as imaged by the image acquisition device will also vary due to the perspective principle that objects appear larger when they are closer and smaller when they are farther away.

[0062] The image acquisition device can also capture three-dimensional information. For example, the image acquisition device can be a 3D camera. The terminal can also obtain the height distribution information of the pallets to be depalletized. The palletizing height distribution information represents a set of heights of various locations in the pallet. The 3D camera can capture three-dimensional image information or two-dimensional images. The palletizing height distribution information can be obtained by capturing three-dimensional image information with a 3D camera. For example, in one embodiment, the terminal can obtain a point cloud image corresponding to the palletized to be depalletized captured by a 3D camera. The point cloud image can be three-dimensional image information corresponding to the palletized to be depalletized, and the point cloud image includes the spatial distribution of the palletized to be depalletized and the height of each location in the palletized to be depalletized. The terminal can determine the height distribution information of the palletized to be depalletized based on the height information of each point in the point cloud image.

[0063] Step S204: obtaining the unit height corresponding to the objects in the to-be-depalletized objects.

[0064] The objects to be depalletized include multiple stacked objects, which may be of the same type or different types. The terminal may pre-acquire the unit height of each object in the depalletized objects. The unit height represents the height of a single object. It should be noted that when the objects in the depalletized objects are of the same type, the terminal may acquire the unit height of objects of that type; when the objects in the depalletized objects are of multiple types, the terminal may acquire the unit height of each type of object.

[0065] Step S206 , determining a search area containing target objects in the image to be depalletized and depalletized based on the unit height and palletizing height distribution information; the target objects include one or more objects corresponding to the target height in the image to be depalletized and depalletized.

[0066] The terminal searches for objects to be moved from the depalletized image using the principle of sliding window matching. The larger the terminal's search area within the depalletized image, the more image matches are required, resulting in slower efficiency. Therefore, the terminal can improve efficiency by reducing the search range of the source image. Based on pre-acquired pallet height distribution information, the terminal can determine the highest point of the depalletized image. Combining the acquired unit heights of the objects, the terminal can determine the search area within the depalletized image containing the target object based on the unit heights and the pallet height distribution information. The target object includes one or more objects within the depalletized image corresponding to the target height. Since depalletizing requires disassembly from top to bottom, layer by layer, to prevent the pallet from collapsing, in some embodiments, the target height can be the height of the current top layer of the depalletized image. Furthermore, in some embodiments, if a specific object needs to be moved first, and if the object is unobstructed within the depalletized image, the target height can be the height of the object.

[0067] There can be one or more target objects at the target height, and the terminal can determine the objects at the target height as the search area. In this way, by combining the unit height of the objects and the stacking height, the terminal can reduce the search range of the objects and improve the efficiency of the matching calculation.

[0068] Step S208: obtaining a preset template image of the object to be transferred in the target object.

[0069] There may be one or more target objects at the same height. After the terminal determines the search range based on the target objects, it can identify the object to be transferred among the target objects using a preset template image. The terminal can then pre-acquire a preset template image of the object to be transferred among the target objects. The object to be transferred may be an object that needs to be transferred from a pallet to be disassembled and depalletized. For example, the terminal uses an industrial robot positioned above the object to be disassembled and depalletized to move the object to be transferred elsewhere. The preset template image may be a standard two-dimensional image corresponding to the object to be transferred. The preset template image may be generated based on the actual size and specifications of the object to be transferred. The terminal can obtain the preset template image corresponding to the object to be transferred from a preset template database. The template database may store preset template images of multiple objects.

[0070] Step S210 , based on the preset template image, a target template image of the object to be transferred is obtained; the ratio of the target template image to the preset template image is determined according to the target height.

[0071] Since the object's size ratio varies depending on its distance from the image acquisition device, after acquiring the preset template image, the terminal can adjust the preset template image to conform to the visual proportions of the object to be transferred at the target height. That is, the terminal can acquire a target template image of the object to be transferred based on the preset template image. For example, based on the target height, the terminal determines the ratio of the target template image to the preset template image, and based on this ratio, adjusts the preset template image to obtain the target template image.

[0072] Step S212: determining the position of the object to be transferred according to the search area and the target template image.

[0073] After the terminal determines the search area within the depalletized object and the target template image that matches the target height ratio, it can use the target template image to determine the location of the object to be transferred. For example, the terminal uses the target template image to search within the search area and identifies an object image whose similarity to the target template image exceeds a preset similarity threshold, using this as the image location of the object to be transferred. The terminal can then transfer the object to be transferred based on this image location. For example, the terminal transmits this image location to an industrial robot, which determines the location of the object to be transferred within the depalletized object based on this image location and then transfers the object to be transferred.

[0074] In this method of object recognition based on three-dimensional data, a search area within the depalletized image containing the target object is determined based on the unit height of the objects within the depalletized image and the distribution of the palletized heights. A target template image is then determined based on a preset template image of the object to be transferred within the target object. The position of the object to be transferred is then determined based on the search area and the target template image. Compared to traditional object recognition based on two-dimensional images, this solution improves object recognition efficiency by adjusting the scale of the object template image incorporating palletized height information and identifying the position of the object to be transferred based on the adjusted template image.

[0075] In one embodiment, determining a retrieval area containing a target object in an image to be depalletized and depalletized based on unit height and palletizing height distribution information includes: mapping the palletizing height distribution information to the image to be depalletized and depalletized based on the position of the palletized object in the image to be depalletized and depalletized to obtain regional height distribution information of the image to be depalletized and depalletized; determining an area of ​​the target height in the image to be depalletized and depalletized based on the unit height and regional height distribution information, and obtaining a retrieval area based on the area of ​​the target height.

[0076] In this embodiment, the terminal can obtain three-dimensional image information and a two-dimensional image of the pallet to be depalletized. The terminal can determine the search area in the two-dimensional image corresponding to the pallet to be depalletized through mapping. For example, the terminal can obtain the location of the pallet to be depalletized in the two-dimensional image of the pallet to be depalletized and map the pallet height distribution information in the three-dimensional image onto the pallet to be depalletized image, for example, onto the location of the pallet to be depalletized in the image of the pallet to be depalletized. This allows the terminal to obtain regional height distribution information, namely, height distribution information of the area corresponding to the pallet to be depalletized in the image of the pallet to be depalletized.

[0077] After determining the regional height distribution information, the terminal can identify the region of the target height within the image of the palletized items to be depalletized based on the unit height. The terminal can determine the region of the target height by using height differences. For example, in one embodiment, the terminal can obtain the maximum height from the regional height distribution information and the difference between the maximum height and the unit height. Based on this difference and the maximum height, the terminal can then determine the region of the target height within the image of the palletized items to be depalletized.

[0078] Specifically, if Figure 3 As shown, Figure 3 The figure is a schematic diagram of determining the search area in one embodiment. The terminal can use the above difference as the minimum height value in the area and the above maximum value as the maximum height value in the area to obtain the target height range. Thus, the terminal uses the corresponding area within the target height range in the above image of the pallet to be disassembled as the target height area. For example, Figure 3 The middle area 300 belongs to the area of ​​0.3 height, while the height of the unit object is 0.3 and the highest height of the pallet is 0.6. Then the terminal can determine the area of ​​the target height as (0.3, 0.6) through the above steps. After the terminal determines the area of ​​the target height, it can obtain the search area based on the area of ​​the target height. For example, the image of the target height area is used as the search area. Specifically, it can be Figure 3 The terminal can thus determine the search area by height restriction, shielding the image information of the bottom objects in the to-be-depalletized stack, thereby reducing the search range.

[0079] Through the above embodiment, the terminal can determine the height information of each pixel in the two-dimensional image through mapping and determine the search area based on the height information, thereby reducing the search range and improving the search efficiency, thereby improving the efficiency of palletizing and disassembling. In addition, by shielding the image of the underlying object based on the height information, interference is reduced, thereby improving the accuracy of searching for the object to be transferred.

[0080] In one embodiment, a target template image of the object to be transferred is obtained based on a preset template image, including: determining a corresponding zoom ratio according to a target height corresponding to a retrieval area; the zoom ratio is positively correlated with the target height; adjusting the scale of the preset template image based on the zoom ratio, and obtaining the target template image based on the preset template image after the scale is adjusted.

[0081] In this embodiment, the image acquisition device can be set above the pallet to be disassembled. Due to the principle that objects appear larger when they are closer and smaller when they are farther away, the sizes of objects at different heights in the stacked pallet may differ from the sizes of the preset template images. Therefore, the terminal can adjust the size of the preset template image by zooming in or out. The terminal can pre-set the relationship between the zoom ratio and the target height. For example, the zoom ratio is positively correlated with the target height, that is, the higher the target height, the greater the zoom ratio, and vice versa. The terminal can determine the corresponding zoom ratio based on the target height corresponding to the search area. After determining the zoom ratio, the terminal can adjust the scale of the preset template image based on the zoom ratio, so that the terminal can obtain the target template image based on the adjusted preset template image.

[0082] Specifically, if Figure 4 As shown, Figure 4 Figure 2 is a schematic diagram of image scale in one embodiment. Based on the distance of the image acquisition device from the object, the terminal can pre-distribute and determine the scaling ratio of the object's preset template image. For example, the scale from the top to the bottom of the pallet can be from 1.5x to 0.5x. Because the terminal pre-maps the height information of each point in the image of the pallet to be disassembled, the terminal can obtain the relationship between height and scale. After determining the target height corresponding to the target object, the terminal can scale the preset template image corresponding to the object to be transferred based on the scale corresponding to the target height. For example, if the object to be transferred is at a scale of approximately 0.8, the terminal can scale the preset template image to 0.8 times the original scale to obtain the target template image.

[0083] Through this embodiment, the terminal can adjust the proportion of the template image of the object based on the height of the object to adapt to the visual perception of the object in the two-dimensional image, thereby improving the accuracy of object retrieval and the efficiency of disassembly and depalletizing.

[0084] In one embodiment, determining the position of the object to be transferred based on the search area and the target template image includes: acquiring the search area at multiple rotation angles; and determining the position of the object to be transferred based on the search area and the target template image at multiple rotation angles.

[0085] In this embodiment, since the objects are manually placed, the placement angles of the objects in the unpacked and depalletized items may not be consistent. To improve the terminal's retrieval accuracy for the objects to be transferred, the terminal can match a preset template image in the retrieval area by changing the angle of the retrieval area. The terminal can obtain retrieval areas at multiple rotation angles based on multiple retrieval angles and the above-mentioned retrieval area. For example, the terminal can rotate the above-mentioned retrieval area according to multiple rotation angles to obtain retrieval areas at multiple angles. The terminal can determine the position of the object to be transferred based on the retrieval areas at multiple rotation angles and the target template image. For example, the terminal can match the target template image with the retrieval areas at each rotation angle to determine the position of the object to be transferred based on the matching results.

[0086] Through this embodiment, the terminal can achieve the effect of improving the depalletizing efficiency by matching the template image with multiple search areas at different rotation angles.

[0087] In one embodiment, based on a preset template image, a target template image of the object to be transferred is obtained, including: determining a matching ratio range according to the target height; obtaining a target template image according to the ratio range and the preset template image; the target template image includes template images of different ratios within the ratio range.

[0088] In this embodiment, when the target template image is determined based on the preset template image, the terminal can improve the quality of the retrieval by setting redundant images. The terminal can determine the matching ratio range based on the above target height. For example, if the target height is 0.3, based on the relationship between the height and the matching ratio, the terminal can determine that the ratio corresponding to 0.3 is 0.7, and the corresponding ratio range can be set by the terminal to 0.8~0.6. The terminal can then obtain the target template image based on the above ratio range and the preset template image. The above target template image includes template images of different ratios within the above ratio range. For example, the terminal obtains multiple template images of different ratios as the target template image within the ratio range of 0.8~0.6 according to a preset step size, such as 0.05.

[0089] The terminal can perform matching within the search area based on multiple template images of varying scales and determine the location of the object to be transferred. For example, in one embodiment, the terminal can perform a search within the search area using template images of varying scales to determine the location of the object to be transferred. The terminal can perform a search within the search area using these template images of varying scales. Furthermore, during this search, the terminal can obtain a matching location within the search area that satisfies a preset matching probability, and then determine the location of the object to be transferred based on this matching location.

[0090] Specifically, for each template image ratio, the terminal can search the template image of that ratio in the aforementioned search area, thereby obtaining a matching probability between each object identified by the template image of that ratio in the search area and the object to be transferred. The terminal can then obtain the matching position of the template image in the aforementioned search area when it meets the preset matching probability, and thus the terminal can determine the position of the object to be transferred based on the matching position. The aforementioned template images of different ratios can determine multiple matching positions that meet the conditions. If there are multiple matching positions, the terminal may use the matching position with the highest matching probability as the position of the object to be transferred.

[0091] Through this embodiment, the terminal can search in the search area based on template images of different proportions within the proportion range, thereby improving the efficiency of determining the position of the object to be transferred, and further improving the efficiency of disassembly and depalletizing.

[0092] In one embodiment, searching is performed in a search area using template images of different proportions, including: searching in the search area at multiple rotation angles in sequence using template images of different proportions; when searching for template images one by one in the search area at the current rotation angle, if it is detected that the matching probability of a template image is lower than a preset stop search probability, directly entering the search area at the next rotation angle to search for template images one by one.

[0093] In this embodiment, the above-mentioned search area may include search areas at multiple rotation angles. That is, the search area includes areas at multiple rotation angles, and the template image includes multiple template images at different ratios. When the terminal searches and matches the object to be transferred, it can use template images of different ratios to search in the search areas at multiple rotation angles in sequence. Moreover, when the terminal searches for template images one by one in the search area at the current rotation angle, if it detects that the matching probability of a template image is lower than the preset stop search probability, the terminal can stop searching the search area at the current rotation angle and directly enter the search area at the next rotation angle to search for template images one by one.

[0094] Through the above matching process, the terminal can obtain multiple matching probabilities after matching the object to be transferred using template images of different proportions in the search area at multiple rotation angles. In one embodiment, the terminal can determine the number of template images that meet the preset matching probability from the above matching. If the number is multiple, the terminal can determine the position of the object to be transferred based on the matching position of the template image with the highest flow probability among the multiple template images in the above search area. For example, in the search area at one rotation angle, after matching using template images of different proportions, the terminal obtains a matching probability of 0.8 corresponding to the template image that meets the preset matching probability; in the search area at another rotation angle, after matching using template images of different proportions, the terminal obtains a matching probability of 0.9 corresponding to the template image that meets the preset matching probability. The terminal can then use the matching position corresponding to the matching probability of 0.9 as the position of the object to be transferred.

[0095] Specifically, the terminal determines the ratio range based on the target height and obtains multiple template images within the ratio range. The terminal rotates the search area to obtain the search area at multiple different rotation angles. Taking 10-degree intervals, a matching ratio of 0.6 to 0.8, and a matching ratio change step of 0.05 as an example, the terminal can obtain the matching probability through multiple matching steps as shown in the following table:

[0096]

[0097] Among them, each matching probability in the table can be the maximum matching value obtained in each search. As can be seen from the above table, each column represents the search of template images of different proportions in the search area at the same angle, and the search results of each proportion are very close. Therefore, for the search area at the same angle, if the terminal detects that the matching probability of a template of a certain proportion is less than the preset stop search probability, for example, less than 0.6, the terminal can stop searching for template images of other proportions and search for template images one by one in the search area of ​​the next rotation angle, reducing the number of searches.

[0098] Taking a preset matching probability of 0.86 as an example, the table above shows that, in a search area at a 20-degree angle and a template image ratio of 0.7, the terminal detected an object with a matching probability of 0.88; and in a search area at a 330-degree angle and a template image ratio of 0.65, the terminal detected an object with a matching probability of 0.91. If multiple template images meet the preset matching probability, the terminal can determine the matching position of the template image with the highest matching probability in the search area as the location of the object to be transferred. For example, the terminal can determine the matching position corresponding to 0.91 as the location of the object to be transferred.

[0099] Through the above embodiment, the terminal can perform multiple matches using template images of different proportions in the search area at different rotation angles, thereby determining the position of the object to be transferred from the results of multiple matches, thereby improving the efficiency of depalletizing.

[0100] It should be understood that, although the steps in the flowcharts of the above embodiments are shown in sequence as indicated by the arrows, these steps are not necessarily performed in the order indicated by the arrows. Unless otherwise specified herein, there is no strict order restriction on the execution of these steps, and these steps can be performed in other orders. Moreover, at least a portion of the steps in the flowcharts of the above embodiments may include multiple steps or multiple stages, and these steps or stages are not necessarily performed at the same time, but can be performed at different times. The execution order of these steps or stages is not necessarily to be performed in sequence, but can be performed in turn or alternately with other steps or at least a portion of steps or stages in other steps.

[0101] Based on the same inventive concept, embodiments of the present application also provide a three-dimensional data-based object recognition device for implementing the aforementioned three-dimensional data-based object recognition method. The solution provided by this device is similar to the solution described in the aforementioned method. Therefore, the specific limitations of one or more of the following embodiments of the three-dimensional data-based object recognition device can be found in the above-mentioned limitations of the three-dimensional data-based object recognition method and will not be further elaborated here.

[0102] In one embodiment, Figure 5 As shown, a device for object recognition based on three-dimensional data is provided, comprising: a first acquisition module 500, a second acquisition module 502, a determination module 504, a third acquisition module 506, an adjustment module 508 and a recognition module 510, wherein:

[0103] The first acquisition module 500 is used to acquire the image of the pallet to be depalletized and the pallet height distribution information of the pallet to be depalletized.

[0104] The second obtaining module 502 is used to obtain the unit height corresponding to the objects in the to-be-depalletized objects.

[0105] The determination module 504 is configured to determine a search area containing target objects in the image to be depalletized based on the unit height and palletizing height distribution information; the target objects include one or more objects corresponding to the target height in the image to be depalletized.

[0106] The third acquisition module 506 is used to acquire a preset template image of the object to be transferred in the target object.

[0107] The adjustment module 508 is configured to obtain a target template image of the object to be transferred based on the preset template image; the ratio of the target template image to the preset template image is determined according to the target height.

[0108] The recognition module 510 is used to determine the position of the object to be transferred based on the search area and the target template image.

[0109] In one embodiment, the first acquisition module 500 is configured to acquire a point cloud image corresponding to the pallet to be depalletized; and determine the pallet height distribution information of the pallet to be depalletized based on the height information of each point in the point cloud image.

[0110] In one embodiment, the determination module 504 is configured to map the palletizing height distribution information to the image of the depalletizing image based on the position of the depalletizing object in the image of the depalletizing image, thereby obtaining regional height distribution information of the image of the depalletizing image; determine an area of ​​a target height in the image of the depalletizing image based on the unit height and regional height distribution information, and obtain a search area based on the area of ​​the target height.

[0111] In one embodiment, the determination module 504 is configured to obtain the maximum height in the regional height distribution information; obtain the difference between the maximum height and the unit height, and determine the region of the target height in the to-be-depalletized image based on the difference and the maximum height.

[0112] In one embodiment, the adjustment module 508 is used to determine the corresponding zoom ratio according to the target height corresponding to the search area; the zoom ratio is positively correlated with the target height; based on the zoom ratio, the proportion of the preset template image is adjusted, and the target template image is obtained based on the preset template image after the proportion is adjusted.

[0113] In one embodiment, the recognition module 510 is configured to obtain a search area at multiple rotation angles; and determine the position of the object to be transferred based on the search area at multiple rotation angles and the target template image.

[0114] In one embodiment, the adjustment module 508 is configured to determine a matching ratio range according to the target height; obtain a target template image according to the ratio range and a preset template image; and the target template image includes template images of different ratios within the ratio range.

[0115] In one embodiment, the recognition module 510 is configured to search in a search area using template images of different scales to determine the location of the object to be transferred.

[0116] In one embodiment, the recognition module 510 is configured to search in a search area using template images of different proportions; and determine the position of the object to be transferred based on the matching position of the template image in the search area that meets a preset matching probability.

[0117] In one embodiment, the above-mentioned recognition module 510 is used to search in the search area at multiple rotation angles in sequence using template images of different proportions; when searching for template images one by one in the search area at the current rotation angle, if it is detected that the matching probability of a template image is lower than the preset stop search probability, it directly enters the search area at the next rotation angle to search for template images one by one.

[0118] In one embodiment, the recognition module 510 is configured to determine the position of the object to be transferred based on the matching position of the template image with the highest matching probability among the multiple template images in the search area if there are multiple template images that meet the preset matching probability.

[0119] Each module in the above-mentioned three-dimensional data-based object recognition device can be implemented in whole or in part through software, hardware, or a combination thereof. Each module can be embedded in or independent of a processor in a computer device in hardware form, or can be stored in a computer device memory in software form, so that the processor can call and execute the corresponding operations of each module.

[0120] In one embodiment, a computer device is provided, which may be a server, and its internal structure diagram may be as shown in Figure Y. The computer device includes a processor, a memory, and a network interface connected via a system bus. The processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program, and a database. The internal memory provides an environment for the operation of the operating system and computer program in the non-volatile storage medium. The database of the computer device is used to store XX data. The network interface of the computer device is used to communicate with an external terminal via a network connection. When the computer program is executed by the processor, a method for object recognition based on three-dimensional data is implemented.

[0121] In one embodiment, a computer device is provided. The computer device may be a terminal, and its internal structure diagram may be as follows: Figure 6As shown. The computer device includes a processor, memory, communication interface, display screen and input device connected via a system bus. The processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and computer program in the non-volatile storage medium. The communication interface of the computer device is used to communicate with an external terminal in a wired or wireless manner. The wireless manner can be achieved through WIFI, mobile cellular network, NFC (near field communication) or other technologies. When the computer program is executed by the processor, a method for object recognition based on three-dimensional data is implemented. The display unit of the computer device is used to form a visually visible image, and can be a display screen, a projection device or a virtual reality imaging device. The display screen can be a liquid crystal display screen or an electronic ink display screen. The input device of the computer device can be a touch layer covering the display screen, or a key, trackball or touchpad provided on the computer device housing, or an external keyboard, touchpad or mouse.

[0122] Those skilled in the art will understand that Figure 6 The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than shown in the figure, or combine certain components, or have a different component arrangement.

[0123] In one embodiment, a computer device is provided, including a memory and a processor. The memory stores a computer program, and the processor implements the above-mentioned object recognition method based on three-dimensional data when executing the computer program.

[0124] In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the above-mentioned object recognition method based on three-dimensional data is implemented.

[0125] In one embodiment, a computer program product is provided, comprising a computer program, which implements the above-mentioned object recognition method based on three-dimensional data when executed by a processor.

[0126] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties.

[0127] Those skilled in the art will appreciate that all or part of the processes in the above-mentioned embodiment methods can be implemented by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, database or other media used in the embodiments provided in this application may include at least one of non-volatile and volatile memory. Non-volatile memory may include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory may include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM). The database involved in the various embodiments provided herein may include at least one of a relational database and a non-relational database. Non-relational databases may include, but are not limited to, distributed databases based on blockchains. The processor involved in the various embodiments provided herein may be, but are not limited to, a general-purpose processor, a central processing unit, a graphics processing unit, a digital signal processor, a programmable logic unit, a data processing logic unit based on quantum computing, and the like.

[0128] The technical features of the above embodiments can be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0129] The above-described embodiments merely represent several implementation methods of the present application. While the descriptions are relatively specific and detailed, they should not be construed as limiting the scope of the present application. It should be noted that a person of ordinary skill in the art may make various modifications and improvements without departing from the spirit of the present application, and these modifications and improvements fall within the scope of protection of the present application. Therefore, the scope of protection of the present application shall be determined by the appended claims.

Claims

1. A method for object recognition based on three-dimensional data, characterized in that: The method comprises: Obtaining images of pallets to be disassembled and depalletized and information on the height distribution of pallets to be disassembled and depalletized; Obtaining the unit height corresponding to the objects in the to-be-depalletized object; According to the position of the to-be-depalletized pallet in the to-be-depalletized image, the palletizing height distribution information is mapped to the to-be-depalletized image to obtain regional height distribution information of the to-be-depalletized image; a maximum height in the regional height distribution information is obtained; a difference between the maximum height and the unit height is obtained, and an area of ​​a target height in the to-be-depalletized image is determined according to the difference and the maximum height; according to the area of ​​the target height, a retrieval area containing a target object in the to-be-depalletized image is obtained; the target object includes one or more objects corresponding to the target height in the to-be-depalletized image; the target height is the height of the current top layer of the to-be-depalletized, or, when the to-be-transferred object among the target objects is in an unobstructed state in the to-be-depalletized image, the target height is the height of the to-be-transferred object; Acquire a preset template image of the object to be transferred in the target object; Based on the preset template image, a target template image of the object to be transferred is obtained; the ratio of the target template image to the preset template image is determined according to the target height; The position of the object to be transferred is determined according to the search area and the target template image.

2. The method according to claim 1, characterized in that The steps of obtaining the pallet height distribution information of the pallets to be depalletized include: Obtain the point cloud image corresponding to the objects to be disassembled and depalletized; The height distribution information of the pallet to be depalletized is determined according to the height information of each point in the point cloud image.

3. The method according to claim 1, characterized in that The step of acquiring a target template image of the object to be transferred based on the preset template image includes: Determining a corresponding zoom ratio according to a target height corresponding to the search area; wherein the zoom ratio is positively correlated with the target height; Based on the zoom ratio, the scale of the preset template image is adjusted, and based on the preset template image after the scale is adjusted, the target template image is obtained.

4. The method according to claim 1, wherein Determining the position of the object to be transferred based on the search area and the target template image includes: Obtain the search area at multiple rotation angles; The position of the object to be transferred is determined according to the search area and the target template image at the multiple rotation angles.

5. The method according to claim 1, wherein The step of acquiring a target template image of the object to be transferred based on the preset template image includes: Determining a matching ratio range according to the target height; Obtaining a target template image according to the ratio range and a preset template image; the target template image includes template images of different ratios within the ratio range; Determining the position of the object to be transferred based on the search area and the target template image includes: The position of the object to be transferred is determined by searching the search area using the template images of different proportions.

6. The method according to claim 5, characterized in that The searching in the search area by using the template images of different proportions to determine the position of the object to be transferred includes: Performing a search in the search area using the template images of different proportions; The position of the object to be transferred is determined according to the matching position of the template image that meets the preset matching probability in the search area.

7. The method according to claim 6, characterized in that The search area includes search areas at multiple rotation angles; and searching in the search area using the template images of different proportions includes: Searching the search areas at the multiple rotation angles in sequence using the template images of different proportions; When searching for template images one by one in the search area at the current rotation angle, if the matching probability of a template image is detected to be lower than the preset stop search probability, the search area at the next rotation angle is directly entered to search for template images one by one.

8. The method according to claim 6 or 7, characterized in that The determining the position of the object to be transferred according to the matching position of the template image that meets the preset matching probability in the search area includes: If there are multiple template images that meet the preset matching probability, the position of the object to be transferred is determined according to the matching position of the template image with the highest matching probability among the multiple template images in the search area.

9. An object recognition device based on three-dimensional data, characterized in that: The device comprises: A first acquisition module is used to acquire an image of the pallet to be disassembled and depalletized and information on the height distribution of the pallet to be disassembled and depalletized; A second acquisition module is used to obtain the unit height corresponding to the objects in the to-be-depalletized objects; a determination module configured to map the palletizing height distribution information to the depalletizing image to be depalletized based on the position of the depalletizing object in the depalletizing image to obtain regional height distribution information of the depalletizing image to be depalletized; obtain a maximum height in the regional height distribution information; obtain a difference between the maximum height and the unit height, and determine a region of a target height in the depalletizing image to be depalletized based on the difference and the maximum height; obtain a retrieval region containing a target object in the depalletizing image to be depalletized based on the region of the target height; the target object includes one or more objects corresponding to the target height in the depalletizing image to be depalletized; the target height is the height of the current top layer of the depalletizing object, or, when an object to be transferred among the target objects is in an unobstructed state in the depalletizing image to be depalletized, the target height is the height of the object to be transferred; A third acquisition module is used to acquire a preset template image of the object to be transferred in the target object; an adjustment module, configured to obtain a target template image of the object to be transferred based on the preset template image; wherein the ratio of the target template image to the preset template image is determined according to the target height; The recognition module is used to determine the position of the object to be transferred based on the search area and the target template image.

10. A computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 8 are implemented.

11. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 8 are implemented.

12. A computer program product comprising a computer program, characterized in that When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 8 are implemented.

Images