Image recognition method and device, and computer-readable storage medium

Abstract

The embodiment of the present application discloses an image recognition method and device, and a computer-readable storage medium, which can improve the accuracy of image recognition. The method includes: collecting a target image and obtaining point cloud data of the target image; performing normal calculation on the point cloud data to obtain the normal of each point in the point cloud data; calculating the i-th cylinder candidate parameter based on the point cloud data and the normal of each point; wherein i is a positive integer greater than or equal to 1; performing inside and outside detection on the i-th cylinder candidate parameter to obtain the i-th detection result; performing iterative search processing based on the i-th detection result, the point cloud data and the normal of each point to determine the target cylinder parameter; based on the target cylinder parameter, performing a planar unfolding of the surface of the target image to obtain the recognition result of the cylinder surface.

Description

Technical Field

[0001] The present application relates to the field of image recognition, and in particular to an image recognition method and device, and a computer-readable storage medium. Background Art

[0002] At present, the traditional random sampling consensus algorithm (RANSAC) is usually used to detect and recognize the surface image of cylindrical objects. When there are many objects other than the target cylindrical object in the image, such as Figure 1 As shown in FIG. 1 , the positions and surface information of other cylindrical objects adjacent to the target cylinder together form a cylindrical surface in the shape of a misdetected area, which affects the surface recognition of the target cylinder; or, as shown in FIG. Figure 2 As shown, when the target cylinder is in the background of another cylindrical surface shape, it is easy to interfere with the target cylinder in image detection, causing false detection, thereby resulting in low image recognition accuracy. Summary of the invention

[0003] The embodiments of the present application are intended to provide an image recognition method and device, and a computer-readable storage medium, which can improve the accuracy of image recognition.

[0004] The technical solution of this application is implemented as follows:

[0005] The present application provides an image recognition method, including:

[0006] Collecting a target image and obtaining point cloud data of the target image;

[0007] Calculating the normal of the point cloud data to obtain the normal of each point in the point cloud data;

[0008] Based on the point cloud data and the normal of each point, calculate the i-th cylinder candidate parameter; wherein i is a positive integer greater than or equal to 1;

[0009] Performing inner and outer side detection on the i-th cylinder candidate parameter to obtain an i-th detection result;

[0010] Performing an iterative search process based on the i-th detection result, the point cloud data and the normal of each point to determine the target cylinder parameters;

[0011] Based on the target cylinder parameters, the surface of the target image is planarly unfolded to obtain a recognition result of the cylindrical surface.

[0012] The embodiment of the present application provides an image recognition device, including: a collection unit, a calculation unit, a detection unit, a determination unit and a recognition unit, wherein:

[0013] The acquisition unit is used to acquire a target image and obtain point cloud data of the target image;

[0014] The calculation unit is used to calculate the normal of the point cloud data to obtain the normal of each point in the point cloud data;

[0015] The calculation unit is further used to calculate the i-th cylinder candidate parameter based on the point cloud data and the normal of each point; wherein i is a positive integer greater than or equal to 1;

[0016] The detection unit is used to perform inner and outer side detection on the i-th cylinder candidate parameter to obtain an i-th detection result;

[0017] The determination unit is used to perform iterative search processing based on the i-th detection result, the point cloud data and the normal of each point to determine the target cylinder parameters;

[0018] The recognition unit is used to planarly unfold the surface of the target image based on the target cylinder parameters to obtain a recognition result of the cylindrical surface.

[0019] An embodiment of the present application provides an image recognition device, including: a processor, a memory, and a communication bus, wherein the memory communicates with the processor via the communication bus, and the memory stores one or more programs executable by the processor. When the one or more programs are executed, the processor executes the image recognition method as described in any one of the above items.

[0020] An embodiment of the present application provides a computer-readable storage medium, which stores one or more programs. The one or more programs can be executed by one or more processors to implement the image recognition method as described in any one of the above items.

[0021] The embodiment of the present application provides an image recognition method and device, and a computer-readable storage medium, the method comprising: collecting a target image and obtaining point cloud data of the target image; performing normal calculation on the point cloud data to obtain the normal of each point in the point cloud data; calculating the i-th cylinder candidate parameter based on the point cloud data and the normal of each point; wherein i is a positive integer greater than or equal to 1; performing inner and outer side detection on the i-th cylinder candidate parameter to obtain the i-th detection result; performing iterative search processing based on the i-th detection result, the point cloud data and the normal of each point to determine the target cylinder parameter; based on the target cylinder parameter, performing a planar unfolding of the surface of the target image to obtain the recognition result of the cylinder surface. Using the above method, the image recognition device adds detection of the inner and outer sides of the candidate cylinder parameters when using an iterative algorithm to search for the best target cylinder parameters, thereby achieving further screening of the image recognition results and improving the accuracy of image recognition. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] Figure 1 This is an illustration of the situation of misdetection in the prior art Figure 1 ;

[0023] Figure 2 This is an illustration of the situation of misdetection in the prior art Figure 2 ;

[0024] Figure 3 A schematic diagram of an image recognition system provided in an embodiment of the present application;

[0025] Figure 4 An optional flowchart of the image recognition method provided in the embodiment of the present application;

[0026] Figure 5 An optional flowchart of the image recognition method provided in the embodiment of the present application;

[0027] Figure 6-1 A schematic diagram of the camera optical axis direction vector provided in an embodiment of the present application;

[0028] Figure 6-2 Schematic diagram of normal constraints provided in the embodiment of the present application Figure 1 ;

[0029] Figure 6-3 Schematic diagram of normal constraints provided in the embodiment of the present application Figure 2 ;

[0030] Figure 7 A schematic diagram of a point cloud normal vector provided in an embodiment of the present application;

[0031] Figure 8 An optional flowchart of the image recognition method provided in the embodiment of the present application;

[0032] Fig. 9 An optional flowchart of the image recognition method provided in the embodiment of the present application;

[0033] Fig.10 An optional flowchart of the image recognition method provided in the embodiment of the present application;

[0034] Fig.11 An optional flowchart of the image recognition method provided in the embodiment of the present application;

[0035] Fig.12 A schematic diagram of an exemplary centroid vector from the center point of the i-th cylinder to the centroid provided in an embodiment of the present application;

[0036] Fig.13 An optional flowchart of the image recognition method provided in the embodiment of the present application;

[0037] Fig.14 A schematic diagram of an exemplary first direction value and a second direction value provided in an embodiment of the present application;

[0038] Fig.15 An optional flowchart of the image recognition method provided in the embodiment of the present application;

[0039] Figure 16-1 Schematic diagram of a cylindrical object for surface image and text restoration and recognition provided by an embodiment of the present application Figure 1 ;

[0040] Figure 16-2 Schematic diagram of a cylindrical object for surface image and text restoration and recognition provided by an embodiment of the present application Figure 2 ;

[0041] Fig.17 A schematic diagram of an image recognition device provided in an embodiment of the present application Figure 1 ;

[0042] Fig.18 A schematic diagram of an image recognition device provided in an embodiment of the present application Figure 2 . DETAILED DESCRIPTION

[0043] The technical solutions in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application.

[0044] An image recognition system provided in an embodiment of the present application is as follows: Figure 3As shown, it includes an image acquisition module 10, a cylinder detection module 20, a validity judgment module 30, a graphic transformation module 40 and a display module 50. Among them, the image acquisition module 10 is used to collect point cloud data of the target image, including an image acquisition device that can collect three-dimensional data information. Exemplarily, the image acquisition device can be a camera with a time of flight sensor (TOF, Time Of Fly), a structured light camera, a binocular camera, etc., and an RGB image acquisition device. The cylinder detection module 20 is used to use an image detection algorithm to detect the parameters of the cylinder graphics contained in the point cloud data of the target image as cylinder candidate parameters. The validity judgment module 30 is used to perform validity detection on the cylinder candidate parameters detected by the cylinder detection module 20. The validity detection can include detecting whether the volume of the cylinder shape corresponding to the cylinder candidate parameters exceeds a threshold, whether the point cloud density covered by the cylinder shape formed by the cylinder candidate parameters is higher than a threshold, whether the cylinder shape formed by the cylinder candidate parameters is a convex cylinder, etc. The cylinder detection module 20 is also used to iterate the cylinder candidate parameters that have passed the validity test. For example, a random sampling consensus algorithm is used to optimize and iterate the cylinder candidate parameters that have passed the validity test, and finally obtain the target cylinder parameters. The graphic transformation module 40 is used to fit the cylinder shape corresponding to the target cylinder parameters according to the target cylinder parameters, and flatten the side surface of the cylinder shape corresponding to the target cylinder parameters from the curved surface to a plane to obtain the recognition result of the target object. The display module 50 is used to display the recognition result on the display interface.

[0045] based on Figure 3 The image recognition system provided by Figure 4 An optional flow chart of the image recognition method provided in the embodiment of the present application is combined with Figure 4 The steps shown are explained.

[0046] S101, collecting a target image and obtaining point cloud data of the target image.

[0047] The image recognition method provided in the embodiment of the present application is applicable to the scene of surface image recognition and extraction of target geometric shapes in an image, for example, Figure 3 As shown, the notices or instructions posted on the surface of buildings or commodities belong to images and text information on three-dimensional surfaces. The image recognition method provided in the embodiment of the present application can recognize images and text information posted on cylindrical surfaces. The image recognition method provided in the embodiment of the present application can also be used in the scene of detecting target obstacles.

[0048] In the embodiment of the present application, the image recognition device first captures the target image through the image acquisition device and obtains the point cloud data of the target image.

[0049] In the embodiment of the present application, the point cloud data is a set of three-dimensional coordinate data of a large number of pixel points constituting the target image. The point cloud data can be obtained by using an image acquisition device capable of acquiring three-dimensional information. For example, the image recognition device acquires or scans the target image through a camera integrated with a TOF sensor, thereby acquiring the point cloud data of the target image. The point cloud data can also be acquired through a structured light device or a depth camera based on triangulation ranging technology, which is not limited in the embodiment of the present application.

[0050] In an embodiment of the present application, the point cloud data for acquiring the target image may be point cloud data stored in advance, or may be point cloud data acquired in real time from a distance sensor such as a TOF sensor or a lidar sensor.

[0051] In the embodiment of the present application, the point cloud data contains the three-dimensional spatial position information of each point in the target image, and the image recognition device can use the point cloud data to detect the three-dimensional object in the target image.

[0052] In an embodiment of the present application, the image recognition device can perform preliminary filtering on the acquired point cloud data to filter out unnecessary information. Specifically, the image recognition device can use a background plane filtering method to filter the point cloud data to filter out background planes such as the ground to reduce interference items and facilitate subsequent processing.

[0053] In the embodiment of the present application, background plane filtering may include RANSAC plane detection, background plane determination, background plane removal, etc., which is not limited in the embodiment of the present application.

[0054] It should be noted that in the embodiment of the present application, when the TOF sensor integrated in the camera is used to obtain point cloud data, it has the characteristics of not being affected by lighting changes and object texture. While meeting the accuracy requirements, it can also reduce the cost of collecting point cloud data.

[0055] S102: Calculate the normal of the point cloud data to obtain the normal of each point in the point cloud data.

[0056] In an embodiment of the present application, after obtaining the point cloud data of the target image, the image recognition device calculates the normal corresponding to each point in the point cloud data, thereby obtaining the normal of each point in the point cloud data.

[0057] In an embodiment of the present application, the surface of a three-dimensional object in a three-dimensional space can be regarded as being composed of many tiny planes. For a point in the point cloud data of the target image, the normal of the point is a vector that is always perpendicular to the plane where the point is located.

[0058] In the embodiment of the present application, for each point in the point cloud data, the normal of each point jointly determines the shape of the three-dimensional image corresponding to the point cloud data. Therefore, in order to obtain the curvature and direction of the cylindrical surface, the image recognition device needs to calculate the normal of each point in the point cloud data to obtain the normal of each point in the point cloud data.

[0059] S103. Calculate the candidate parameters of the i-th cylinder based on the point cloud data and the normal of each point; wherein i is a positive integer greater than or equal to 1.

[0060] In the embodiment of the present application, after the image recognition device acquires the point cloud data and calculates the normal of each point in the point cloud data, the image recognition device calculates the candidate parameters of the i-th cylinder based on the point cloud data and the normal of each point.

[0061] In an embodiment of the present application, the target image contains multiple images of different shapes. The image recognition device detects cylindrical images that may be contained in the target image by calculating the i-th cylinder candidate parameters based on the point cloud data and the normal of each point.

[0062] In the embodiment of the present application, the cylinder candidate parameters may include the cylinder axis vector d, the cylinder center point c and the cylinder radius r. The image recognition device may use the cylinder candidate parameters to fit the corresponding candidate cylinder shape.

[0063] In an embodiment of the present application, the three-dimensional coordinate information of each point in the point cloud data represents the position information of the point in space, and the normal of each point represents the angle information of the plane where the point is located. Therefore, based on the above position information and angle information, the image recognition device can calculate the axis vector d, the center point c and the radius r of the cylinder according to the point cloud data on the surface of the cylinder as candidate parameters of the cylinder.

[0064] In the embodiment of the present application, i is a positive integer greater than or equal to 1. In order to obtain more accurate cylinder candidate parameters, the image recognition device can use a random sampling consensus algorithm to randomly select a preset number of points from the point cloud data for iterative calculation each time, and output different cylinder candidate parameters in each iterative calculation. For the i-th iteration, the image recognition device can calculate the i-th cylinder candidate parameters.

[0065] S104, performing inner and outer side detection on the i-th cylinder candidate parameter to obtain the i-th detection result.

[0066] In the embodiment of the present application, for the i-th iteration, after the image recognition device obtains the i-th cylinder candidate parameter, it can perform inside and outside detection on the cylinder candidate parameter to obtain the i-th detection result.

[0067] In the embodiment of the present application, in the actual scene of photographing the surface of a cylinder, the target of restoring the surface information of the cylinder is mostly a convex cylindrical surface relative to the photographer, such as the surface trademark of a bottle or can, a promotional poster on a cylinder, etc. Therefore, the image recognition device can perform inside and outside detection on the i-th cylinder candidate parameter calculated in the i-th iteration, and determine whether the side surface of the cylinder fitted according to the i-th cylinder candidate parameter is convex or concave.

[0068] In the embodiment of the present application, when the image recognition device determines that the side surface of the cylinder fitted by the i-th cylinder candidate parameters is a convex surface, the image recognition device determines that the i-th cylinder candidate parameters constitute an outer cylinder as the i-th detection result.

[0069] In the embodiment of the present application, when the image recognition device determines that the side surface of the cylinder fitted by the i-th cylinder candidate parameters is a concave surface, the image recognition device determines that the i-th cylinder candidate parameters constitute the inner cylinder as the i-th detection result.

[0070] In an embodiment of the present application, the image recognition device may also perform other methods of validity judgment on the obtained i-th cylinder candidate parameters. For example, it may be judged whether the cylinder volume corresponding to the i-th cylinder candidate parameters exceeds the volume threshold. If so, the validity judgment fails. It may also be judged whether the number of points in the point cloud covered by the cylinder fitted according to the i-th cylinder candidate parameters is equal to the number threshold. If so, the validity judgment fails.

[0071] In the embodiment of the present application, for each iteration process, the image recognition device uses the same method to perform inner and outer side detection on each cylinder candidate parameter to obtain the detection result of each iteration until the iteration condition is met and the iteration ends.

[0072] S105 , performing iterative search processing based on the i-th detection result, the point cloud data and the normal of each point to determine the target cylinder parameters.

[0073] In an embodiment of the present application, for the i-th iteration, the image recognition device can perform iterative search processing based on the i-th detection result, point cloud data and the normal of each point to determine the target cylinder parameters.

[0074] In the embodiment of the present application, the process of iterative search processing by the image recognition device can be specifically as follows: when the i-th detection result is an outer cylinder, it means that the i-th cylinder candidate parameter is suitable for the actual scenario of restoring the cylinder surface information mentioned above, and the image recognition device can fit the cylindrical shape as the target object according to the i-th cylinder candidate parameter, and take the points contained in the three-dimensional space range occupied by the target object as the inner points. For the first iteration, the image recognition device can use the first cylinder candidate parameter as the best candidate parameter and continue to iterate. If the second cylinder candidate parameter corresponds to more inner points in the target object than the best object, the second cylinder candidate parameter is used to replace the first cylinder candidate parameter as the best candidate parameter; otherwise, the next cylinder candidate parameter is calculated based on the point cloud data and the normal of each point; and so on. When the iteration condition is met, the iterative search process ends, and the cylinder candidate parameter corresponding to the best candidate parameter finally obtained is used as the target cylinder parameter.

[0075] In an embodiment of the present application, when the i-th detection result is an inner cylinder, since the target for recovering the cylindrical surface information is mostly the convex cylindrical surface relative to the photographer, the concave cylindrical surface corresponding to the i-th cylinder candidate parameter may be a wrong target for surface information recovery. Therefore, the image recognition device may deem the i-th cylinder candidate parameter invalid, and does not use the i-th cylinder candidate parameter to compare with the best candidate parameter, but instead recalculates the next cylinder candidate parameter based on the point cloud data and the normal of each point.

[0076] In the embodiment of the present application, the iteration condition may be that the number of iterations reaches a preset threshold, or that the growth of the best candidate parameter replacement slows down, etc., which is not limited in the embodiment of the present application.

[0077] In the embodiment of the present application, after the i-th iteration ends, the image recognition device determines whether the i-th iteration meets the iteration conditions. If the iteration conditions are met, the image recognition device ends the iteration and uses the best candidate parameters obtained after the i-th iteration as the target cylinder parameters; if the iteration conditions are not met, the image recognition device returns to S103 to start the next iteration.

[0078] S106. Based on the target cylinder parameters, the surface of the target image is planarly unfolded to obtain a recognition result of the cylindrical surface.

[0079] In the embodiment of the present application, after the image recognition device obtains the target cylinder parameters, the surface of the target image is planar-unfolded based on the target cylinder parameters to obtain a recognition result of the cylindrical surface.

[0080] In an embodiment of the present application, the target cylinder parameters are the curved surface parameters of the target geometric shape corresponding to the target image. The image recognition device can unfold the curved side surface represented by the target cylinder parameters into a plane based on the target cylinder parameters, and use the image after the plane is unfolded as the recognition result.

[0081] In the embodiment of the present application, the image recognition device can use existing methods to unfold the curved surface in the space into a flat surface, which will not be described in detail in the embodiment of the present application.

[0082] It can be understood that in the embodiment of the present application, when the image recognition device uses an iterative algorithm to search for the optimal target cylinder parameters, it adds detection of the inner and outer sides of the candidate cylinder parameters, thereby achieving further screening of the image recognition results and improving the accuracy of image recognition.

[0083] In some embodiments of the present application, after the target cylinder parameters are determined in S105, the target cylinder parameters may be optimized to obtain optimized target cylinder parameters. The image recognition device may design an optimization objective function according to a specific optimization objective.

[0084] In some embodiments of the present application, when the optimization goal is to make the cylindrical surface fitted according to the optimized target cylindrical parameters contain more point cloud data, the image recognition device can use formula (1) as the optimization objective function to optimize the target cylindrical parameters. Formula (1) is as follows:

[0085] θ * =argmin θ (Σ p∈P d 2 (C(θ),p)) (1)

[0086] Among them, θ * is the optimized target cylinder parameter, θ is the target cylinder parameter, p is any coordinate point within the range of the cylindrical shape fitted by the target cylinder parameter in the three-dimensional space of the point cloud data, and d 2 (C(θ),p) is the Euclidean distance from point p to the cylindrical surface, and argmin is used to calculate the distance that can make d 2 The value of θ that makes (C(θ),p) reach its minimum value is the value that makes d 2 The value of the target cylindrical parameter θ that reaches the minimum value of (C(θ),p) is taken as the optimized target cylindrical parameter θ * .

[0087] The image recognition device optimizes and adjusts the target cylinder parameters through formula (1), calculates the target cylinder parameters that can minimize the Euclidean distance from point p to the cylinder surface, and completes the optimization of the target cylinder parameters; thereby, the cylindrical surface fitted according to the optimized target cylinder parameters can contain more points, increasing the amount of point cloud data on the cylinder surface, and can perform subsequent image recognition more accurately.

[0088] In the embodiment of the present application, the image recognition device can use algorithms such as the trust region method, the Levenberg-Marquardt method, etc. to optimize the target cylinder parameters, and can also use other similar optimization algorithms, which are not limited in the embodiment of the present application.

[0089] In the embodiment of the present application, based on the optimized target cylindrical parameters, S106 may be: based on the optimized target cylindrical parameters, the surface of the target image is planar-unfolded to obtain a recognition result of the cylindrical surface.

[0090] It can be understood that in the embodiment of the present application, the image recognition device can design an optimization objective function to further optimize the obtained target cylinder parameters, and obtain more accurate optimized target cylinder parameters for surface recognition, thereby improving the accuracy of image recognition.

[0091] In the present application embodiment, based on Figure 4 In S102, the image recognition device calculates the normal of the point cloud data, and the method for obtaining the normal of each point in the point cloud data can be as follows: Figure 5 As shown, including S1021-S1025, as follows:

[0092] S1021. For each point in the point cloud data, determine a point cloud subset that includes at least one neighboring point within a preset neighboring range of each point.

[0093] In an embodiment of the present application, for each point in the point cloud data, the image recognition device can select at least one neighboring point within a preset proximity range near the point, and a set of at least one neighboring point forms a point cloud subset within the preset proximity range of the point.

[0094] For example, for each point in the point cloud data, the image recognition device selects 50 points in a preset neighborhood to form a point cloud P. K , where K=50.

[0095] S1022. Perform principal component analysis on the point cloud subset of each point to obtain the first principal component vector of each point.

[0096] In the embodiment of the present application, after the image recognition device obtains the point cloud subset within the preset neighborhood range of each point, it performs principal component analysis (PCA) on the point cloud subset of each point, and obtains the first principal component vector of each point from the PCA analysis result.

[0097] In the embodiment of the present application, for the i-th point in the point cloud data, the image recognition device may first perform principal component analysis on the point cloud subset of the i-th point by formula (2) to obtain the first principal component vector of the point, as follows:

[0098] v=PVCA(P K )(0) (2)

[0099] Among them, v is the first principal component vector of the i-th point, P K is the point cloud subset of the i-th point, and (0) represents the first principal component.

[0100] The image recognition device recognizes the point cloud subset P of the i-th point K Perform PCA principal component analysis and select the vector corresponding to the first principal component (0) from the PCA analysis results as the first principal component vector v of the i-th point.

[0101] In the embodiment of the present application, the image recognition device processes each point in the point cloud data in the same way to obtain the first principal component vector of each point.

[0102] S1023. Obtain coordinate information of each point in the point cloud data.

[0103] In the embodiment of the present application, in addition to obtaining the first principal component vector of each point, the image recognition device also needs to obtain the three-dimensional coordinate information of each point in the point cloud data.

[0104] S1024. Determine the normal vector of each point based on the coordinate information of each point and the first principal component vector of each point.

[0105] In the embodiment of the present application, for the i-th point in the point cloud data, the image recognition device can determine the normal vector of the i-th point based on the coordinate information of the i-th point and the first principal component vector of the i-th point by formula (3), as follows:

[0106] v * =argmin(Var(v T p i )) (3)

[0107] Among them, v * is the normal vector of the i-th point, p i represents the three-dimensional coordinates of the i-th point, v Trepresents the transposition of the first principal component vector v obtained in formula (1), and Var represents the variance operation.

[0108] The image recognition device performs the transposition of the first principal component vector v T and the three-dimensional coordinates p of the i-th point i The variance function of the product of is used to find the minimum parameter, and the minimum parameter is used as the normal vector v of the i-th point * .

[0109] In the embodiment of the present application, the image recognition device processes each point in the point cloud data in the same way to obtain the normal vector of each point.

[0110] S1025. Constrain the normal vector of each point to obtain the normal line of each point.

[0111] In the embodiment of the present application, after the image recognition device obtains the normal vector of each point, since the calculation of the normal vector is ambiguous, it is necessary to constrain the normal vector calculated in S1024.

[0112] In the embodiment of the present application, the image recognition device may constrain the normal vector of each point by calculating the angle between the normal vector of each point and the direction vector of the camera optical axis, which may specifically include S10251-S10252 as follows:

[0113] S10251. Determine the angle between the normal vector of each point and the optical axis direction vector; wherein the optical axis direction vector represents the optical axis direction of the acquisition device for acquiring the target image.

[0114] In the embodiment of the present application, the optical axis direction vector represents the optical axis direction of the acquisition device for acquiring the target image. For the i-th point in the point cloud data, the image recognition device can determine the angle between the normal vector of each point and the optical axis direction vector by formula (4), as follows:

[0115] θ=Z·v * (4)

[0116] Among them, θ is the angle between the normal vector of the i-th point and the optical axis direction vector, Z is the camera optical axis direction vector, and v * is the normal vector of the ith point.

[0117] The image recognition device combines Z and v * The angle θ between the normal vector of the i-th point and the optical axis direction vector is obtained by multiplying the normal vector of the i-th point and the optical axis direction vector.

[0118] In the embodiment of the present application, the image recognition device processes each point in the point cloud data in the same way, thereby obtaining the angle between the normal vector of each point and the optical axis direction vector.

[0119] S10252. Based on the angle and the preset normal constraint value, constrain the normal vector of each point corresponding to the angle to obtain the normal of each point; wherein the normal vector of each point corresponds to an angle.

[0120] In the embodiment of the present application, after the image recognition device obtains the angle between the normal vector of each point and the optical axis direction vector, each angle is compared with a preset normal constraint value, and the normal vector of each point corresponding to each angle is constrained according to the comparison result to obtain the normal of each point;

[0121] In the embodiment of the present application, the normal vector of each point corresponds to an angle.

[0122] In the embodiment of the present application, the camera optical axis direction vector Z is as follows: Figure 6-1 As shown in , for a point in the point cloud data, when the angle between the normal vector of the point and the optical axis direction vector is less than 0, the normal vector representing the point satisfies the constraint, and the normal vector of the point can be used as the normal line of the point, as shown in Figure 6-2 As shown; when the angle between the normal vector of the point and the optical axis direction vector is greater than 0, the normal vector of the point does not satisfy the normal constraint, and the normal vector of the point needs to be flipped, that is, the normal vector of the point is negated, and the flipped normal vector is used as the normal of the point, as shown Figure 6-3 shown.

[0123] In the embodiment of the present application, for each point in the point cloud data, after the image recognition device calculates the normal of each point, the schematic diagram of the point cloud normal vector can be as follows: Figure 7 As shown, Figure 7 In the figure, the left side is an image composed of point cloud data, and the right side is an image composed of the normal vector of each point in the point cloud data.

[0124] It can be understood that in the embodiment of the present application, the image recognition device obtains the normal of each point in the point cloud data, thereby obtaining the surface shape of the image corresponding to the point cloud data.

[0125] In the present application embodiment, based on Figure 4 In S103, the image recognition device calculates the candidate parameters of the i-th cylinder based on the point cloud data and the normal of each point as follows: Figure 8 As shown, including S1031-S1033:

[0126] S1031. Select the i-th sampling point data from the point cloud data; wherein the i-th sampling point data includes the first sampling point data and the second sampling point data.

[0127] In the embodiment of the present application, in order to calculate the i-th cylinder candidate parameters corresponding to the i-th iteration, the image recognition device first selects the i-th sampling point data required for the i-th iteration from the point cloud data.

[0128] In the embodiment of the present application, the sampling point data is the three-dimensional coordinate data of the sampling points randomly selected in the point cloud data. For each iteration, the image recognition device can randomly select two sampling points and treat them as two points on the side surface of a cylinder to calculate the candidate parameters of the cylinder corresponding to the two sampling points. Therefore, the i-th sampling point data may include the first sampling point data and the second sampling point data.

[0129] In some embodiments, the image recognition device may select two sampling points from a plurality of three-dimensional coordinate points of the point cloud data, and use the positions of the two sampling points under homogeneous coordinates as the first sampling point data p1 and the second sampling point data p2 respectively.

[0130] S1032. Obtain, from the normal of each point, a first normal corresponding to the first sampling point data and a second normal corresponding to the second sampling point data.

[0131] In the embodiment of the present application, after the image recognition device obtains the first sampling point data and the second sampling point data, the image recognition device can obtain the first normal corresponding to the first sampling point data and the second normal corresponding to the second sampling point data from the normal of each point calculated in S102.

[0132] In some embodiments, for the first sampling point data p1 selected by the image recognition device, the image recognition device can obtain the corresponding first normal n1; for the second sampling point data p2 selected by the image recognition device, the image recognition device can obtain the corresponding second normal n2.

[0133] S1033. Determine the i-th cylinder axis vector, the i-th cylinder center point, and the i-th cylinder radius based on the first sampling point data, the first normal line, the second sampling point data, and the second normal line.

[0134] In the embodiment of the present application, the image recognition device can calculate the i-th candidate cylinder parameter corresponding to the i-th iteration based on the first sampling point data, the first normal, the second sampling point data and the second normal obtained in the i-th iteration.

[0135] In the embodiment of the present application, the i-th candidate cylinder parameter includes the i-th candidate cylinder parameter.

[0136] In the present application embodiment, based on Figure 8 The image recognition device determines the i-th cylinder axis vector, the i-th cylinder center point and the i-th cylinder radius based on the first sampling point data, the first normal line, the second sampling point data and the second normal line. Specifically, the i-th cylinder axis vector, the i-th cylinder center point and the i-th cylinder radius can be as follows: Fig. 9As shown, including S10331-S10335, as follows:

[0137] S10331. Calculate the i-th cylinder axis vector based on the first normal and the second normal.

[0138] In the embodiment of the present application, the image recognition device can calculate the i-th cylinder axis vector based on the first normal and the second normal.

[0139] In the embodiment of the present application, since the first normal line represents the normal vector of the plane where the first sampling point data is located, and the second normal line represents the normal vector of the plane where the second sampling point data is located, the image recognition device can calculate the normal vector that is perpendicular to both the plane where the first sampling point data is located and the plane where the second sampling point data is located based on the first normal line and the second normal line, that is, the axis vector of the cylinder.

[0140] In the embodiment of the present application, the image recognition device can calculate the i-th cylinder axis vector by formula (5), and formula (5) is as follows:

[0141] d=n1×n2 (5)

[0142] Wherein, n1 is the first normal corresponding to the first sampling point data obtained in the i-th iteration, n2 is the second normal corresponding to the second sampling point data obtained in the i-th iteration, and d is the cylinder axis vector.

[0143] The image recognition device performs a cross product on the first normal n1 and the second normal n2 to obtain the i-th cylinder axis vector d.

[0144] S10332. Determine, according to the first sampling point data and the first normal, the three-dimensional projection space matrix parameters of the straight line where the first normal is located.

[0145] In an embodiment of the present application, the straight line where the first normal is located can be determined by two different points contained in the first normal or two different planes containing it. The image recognition device determines the expression of the intersection of the two different points contained in the first normal or the two different planes containing the first normal based on the first sampling point data and the first normal, as the three-dimensional projection space matrix parameter of the straight line where the first normal is located.

[0146] In the embodiment of the present application, the image recognition device can determine the Plücker matrix representation of the two different points included in the first normal by using formula (6), and formula (6) is as follows:

[0147]

[0148] Among them, p1 is the first sampling point data, L1 is the Plücker matrix of the straight line where the first normal n1 corresponding to p1 is located, and L1 is a 2*2 matrix containing 4 degrees of freedom parameters.

[0149] The image recognition device calculates the Plücker matrix representation of the straight line where the first normal line is located in the three-dimensional Euclidean space according to the first sampling point data and the first normal line.

[0150] S10333. Determine the plane where the second sampling point data is located according to the second sampling point data and the second normal.

[0151] In the embodiment of the present application, since a plane in a three-dimensional space can be uniquely determined by a point and a line on the plane, the image recognition device can determine the plane where the second sampling point data is located based on the second sampling point data and the second normal.

[0152] In the embodiment of the present application, the image recognition device can determine the plane where the second sampling point data is located by formula (7), and formula (7) is as follows:

[0153] π2=(n2×d;-p2·(n2×d)) (7)

[0154] Among them, p2 is the second sampling point data, π2 is the plane where the sampling point data p2 is located, and π2 is a 4*1 matrix containing 3 coordinate components and 1 normal vector component.

[0155] The image recognition device calculates the three coordinate components of the plane π2 where the second sampling point data is located based on the cylinder axis vector d and the second normal n2, and calculates the direction vector of the plane where the second sampling point data is located based on the second sampling point data, the cylinder axis vector d, and the second normal n2, and finally obtains the plane π2 where the second sampling point data is located, which includes 3 coordinate components and 1 normal vector component.

[0156] S10334. Determine the i-th cylinder center point where the straight line and the plane intersect according to the three-dimensional projection space matrix parameters of the straight line where the first normal line is located and the plane where the second sampling point data is located.

[0157] In the embodiment of the present application, the image recognition device can determine the i-th cylinder center point where the straight line intersects the plane through the three-dimensional projection space matrix parameters of the straight line where the first normal line is located and the plane where the second sampling point data is located.

[0158] In the embodiment of the present application, after obtaining L1 and π2, the image recognition device can calculate the center point of the i-th cylinder by formula (8):

[0159] c=L1π2 (8)

[0160] Among them, c is the center point of the cylinder where the straight line L1 intersects the plane π2.

[0161] S10335. Determine the radius of the i-th cylinder according to the first sampling point data and the i-th cylinder center point; or determine the radius of the i-th cylinder according to the second sampling point data and the i-th cylinder center point.

[0162] In the embodiment of the present application, after the image recognition device obtains the i-th cylinder center point where the straight line where the first normal line is located intersects with the plane where the second sampling point data is located through the above process, since for a point on the side surface of the cylinder, the distance from the point to the center of the cylinder is the radius of the cylinder. The image recognition device can determine the i-th cylinder radius based on the first sampling point data and the i-th cylinder center point; or it can also determine the i-th cylinder radius based on the second sampling point data and the i-th cylinder center point.

[0163] In the embodiment of the present application, the image recognition device can calculate the radius of the i-th cylinder by formula (9), and the formula (9) is as follows:

[0164] r=|p1-c| (9)

[0165] Wherein, r is the radius of the cylinder, and the image recognition device determines the radius of the i-th cylinder by calculating the distance from the first sampling point data to the center point of the i-th cylinder.

[0166] In the embodiment of the present application, the image recognition device can also use the same calculation principle to determine the three-dimensional projection space matrix parameters of the line where the second normal line is located based on the second sampling point data and the second normal line; then determine the plane where the first sampling point data is located based on the first sampling point data and the first normal line; then determine the i-th cylinder center point where the line and the plane intersect based on the three-dimensional projection space matrix parameters of the line where the second normal line is located and the plane where the first sampling point data is located. No further details will be given here.

[0167] S1034. The i-th cylinder axis vector, the i-th cylinder center point and the i-th cylinder radius constitute the i-th cylinder candidate parameters.

[0168] In an embodiment of the present application, the image recognition device can fit a uniquely corresponding cylindrical shape through the i-th cylinder axis vector, the i-th cylinder center point and the i-th cylinder radius, and the image recognition device uses the i-th cylinder axis vector, the i-th cylinder center point and the i-th cylinder radius as candidate parameters of the i-th cylinder.

[0169] It can be understood that in the embodiment of the present application, for the i-th iteration, the image recognition device randomly selects two sampling points from the point cloud data, and calculates the corresponding i-th cylinder candidate parameters on the premise that the two sampling points are assumed to be two points on the side surface of the cylinder. Therefore, the calculated i-th cylinder candidate parameters need to be further tested and verified for validity.

[0170] In the present application embodiment, based on Figure 8 In S104, the inside and outside of the i-th cylinder candidate parameters are detected to obtain the i-th detection result. Fig.10 As shown, including S201-S204, as follows:

[0171] S201. Determine a target object composed of i-th cylinder candidate parameters from a target image; wherein the target object is an image having a cylindrical shape.

[0172] In the embodiment of the present application, after the image recognition device calculates the i-th cylinder candidate parameters, the image recognition device determines the target object constituted by the i-th cylinder candidate parameters from the target image.

[0173] In the embodiment of the present application, the target object is an image having a cylindrical shape.

[0174] In the embodiment of the present application, the image recognition device fits the unique corresponding cylindrical shape image in the three-dimensional space of the point cloud data according to the i-th cylinder axis vector, the i-th cylinder center point and the i-th cylinder radius in the i-th cylinder candidate parameters as the target object constituted by the i-th cylinder candidate parameters.

[0175] S202: Acquire the internal point cloud data belonging to the target object from the point cloud data.

[0176] In the embodiment of the present application, after the image recognition device acquires the target object, the point cloud data within the spatial range occupied by the target object in the point cloud data is used as the inlier point cloud data.

[0177] S203. Calculate the centroid of the corresponding inlier point cloud according to the inlier point cloud data.

[0178] In the embodiment of the present application, the inlier point cloud data is also formed by the aggregation of multiple points. The image recognition device calculates the center of gravity of the corresponding inlier point cloud based on the inlier point cloud data.

[0179] In the embodiment of the present application, the center of gravity of the inner point cloud is a three-dimensional coordinate value, which is the average value of all coordinates in the inner point cloud data.

[0180] In the embodiment of the present application, the image recognition device can calculate the center of gravity of the inner point cloud by calculating the average value of the inner point cloud coordinates, or other methods can be used, which is not limited in the embodiment of the present application.

[0181] S204. Based on the center of gravity, the i-th cylinder candidate parameter and the optical axis direction vector, perform inside and outside detection on the i-th cylinder candidate parameter to obtain the i-th detection result; wherein the optical axis direction vector represents the optical axis direction of the acquisition device for acquiring the target image.

[0182] In the embodiment of the present application, after obtaining the center of gravity, the image recognition device can perform inside and outside detection on the i-th cylinder candidate parameter based on the center of gravity, the i-th cylinder candidate parameter and the optical axis direction vector to obtain the i-th detection result.

[0183] In the present application embodiment, based on Fig.10 The image recognition device performs inside and outside detection on the i-th cylinder candidate parameter based on the center of gravity, the i-th cylinder candidate parameter and the optical axis direction vector. The method for obtaining the i-th detection result can be as follows: Fig.11 As shown, including S2041-S2043, as follows:

[0184] S2041. Construct a centroid vector based on the center point and centroid of the i-th cylinder.

[0185] In the embodiment of the present application, the camera optical axis direction vector is as follows: Figure 6-1 As shown, the image recognition device can use the vector from the center point of the i-th cylinder to the center of gravity as the center of gravity vector, as shown in Fig.12 shown.

[0186] In some embodiments of the present application, when the center of gravity is G and the center point of the i-th cylinder is c, the image recognition device may calculate the value of (Gc) as the center of gravity vector.

[0187] S2042. When the centroid vector is consistent with the optical axis direction vector, the i-th detection result that the i-th cylinder candidate parameter constitutes the outer cylinder is obtained, and the inner and outer side detection of the i-th cylinder candidate parameter is completed.

[0188] In the embodiment of the present application, when the center of gravity vector is consistent with the optical axis direction vector, that is, (GC)·Z>0, it means that the center of gravity vector is consistent with the optical axis direction vector, and the image recognition device obtains the i-th detection result that the i-th cylinder candidate parameter constitutes the outer cylinder, and completes the inner and outer side detection of the i-th cylinder candidate parameter.

[0189] S2043. When the centroid vector is inconsistent with the optical axis direction vector, the i-th detection result of the i-th cylinder candidate parameter constituting the inner cylinder is obtained, and the inner and outer side detection of the i-th cylinder candidate parameter is completed.

[0190] In the embodiment of the present application, when the centroid vector is inconsistent with the optical axis direction vector, that is, (GC)·Z<0, it means that the centroid vector is inconsistent with the optical axis direction vector, and the image recognition device obtains the i-th detection result that the i-th cylinder candidate parameter constitutes the inner cylinder, and completes the inner and outer side detection of the i-th cylinder candidate parameter.

[0191] In the present application embodiment, based on Figure 8Another method for performing inner and outer detection on the i-th cylinder candidate parameter in S104 to obtain the i-th detection result is as follows: Fig.13 As shown, including S301-S304, as follows:

[0192] S301. Obtain a first direction value according to first sampling point data, a first normal line and an i-th cylinder center point.

[0193] In the present application embodiment, Fig.14 As shown, the image recognition device calculates the direction vector from the center point of the i-th cylinder to the first sampling point data as the first direction value based on the first sampling point data, i.e., the three-dimensional coordinates of the first sampling point, the first normal, and the center point of the i-th cylinder.

[0194] In some embodiments of the present application, the image recognition device may calculate the value of (p1-c)·n1 as the first direction value.

[0195] S302: Obtain a second direction value according to the second sampling point data, the second normal line and the i-th cylinder center point.

[0196] In the present application embodiment, Fig.14 As shown, the image recognition device calculates the direction vector from the i-th cylinder center point to the second sampling point data as the second direction value based on the second sampling point data, i.e., the three-dimensional coordinates of the second sampling point, the second normal, and the i-th cylinder center point.

[0197] In some embodiments of the present application, the image recognition device may calculate the value of (p2-c)·n2 as the second direction value.

[0198] S303: When both the first direction value and the second direction value are greater than 0, the i-th detection result that the i-th cylinder candidate parameter constitutes the outer cylinder is obtained, and the inner and outer side detection of the i-th cylinder candidate parameter is completed.

[0199] In an embodiment of the present application, when the first direction value and the second direction value are both greater than 0, it means that the cylindrical shape fitted by the i-th cylinder candidate parameter is a convex cylindrical side surface, and the image recognition device obtains the i-th detection result that the i-th cylinder candidate parameter constitutes an outer cylinder, and completes the inner and outer side detection of the i-th cylinder candidate parameter.

[0200] S304. When any one of the first direction value and the second direction value is less than 0, the i-th detection result that the i-th cylinder candidate parameter constitutes the inner cylinder is obtained, and the inner and outer side detection of the i-th cylinder candidate parameter is completed.

[0201] In an embodiment of the present application, when at least one of the first direction value and the second direction value is less than 0, it indicates that the cylindrical shape fitted by the i-th cylinder candidate parameter is a concave cylindrical side surface, and the image recognition device obtains the i-th detection result that the i-th cylinder candidate parameter constitutes the inner cylinder, thereby completing the inner and outer side detection of the i-th cylinder candidate parameter.

[0202] In the present application embodiment, based on Fig.11 or Fig.13 In S105, an iterative search process is performed based on the i-th detection result, the point cloud data and the normal of each point to determine the target cylinder parameters as follows: Fig.15 As shown, including S1051-S1056, as follows:

[0203] S1051. When the i-th detection result is that the i-th cylinder candidate parameters constitute the inner cylinder, the i+1-th cylinder candidate parameters are calculated based on the point cloud data and the normal of each point until the target cylinder parameters that meet the iteration conditions are obtained.

[0204] In the embodiment of the present application, when the i-th detection result is that the i-th cylinder candidate parameters constitute the inner cylinder, since the objects for surface image and text restoration and recognition are usually convex surfaces for the photographer, such as Figure 16-1 The surface of a cylindrical object or Figure 16-2 Therefore, the i-th cylinder candidate parameter is an invalid parameter for image recognition. The image recognition device abandons the i-th cylinder candidate parameter obtained in the i-th iteration and restarts the next iteration, that is, randomly selecting sampling points and their corresponding normals from the point cloud data and the normal of each point, and calculating the i+1-th cylinder candidate parameter until the target cylinder parameter that meets the iteration conditions is obtained.

[0205] S1052. When the i-th detection result is that the i-th cylinder candidate parameters constitute an outer cylinder, determine a target object constituted by the i-th cylinder candidate parameters from the target image; wherein the target object is an image having a cylindrical shape.

[0206] In an embodiment of the present application, when the i-th detection result is that the i-th cylinder candidate parameters constitute an outer cylinder, it means that the i-th cylinder candidate parameters can be used for image recognition, and the image recognition device determines from the target image an image of a cylindrical shape fitted by the i-th cylinder candidate parameters as a target object.

[0207] S1053. Count the number of i-th inliers in the inlier point cloud data belonging to the target object from the point cloud data.

[0208] In the embodiment of the present application, the image recognition device counts the number of inliers within the space occupied by the target object from the point cloud data as the i-th inlier number of the inlier point cloud data.

[0209] S1054. When the number of the i-th inner points is greater than or equal to the corresponding maximum number of inner points of the cylinder candidate parameter before the i-th cylinder candidate parameter and the iteration condition is met, the i-th cylinder candidate parameter is used as the target cylinder parameter.

[0210] In an embodiment of the present application, when the number of the i-th inner points is greater than or equal to the corresponding maximum number of inner points of the cylinder candidate parameter before the i-th cylinder candidate parameter and the iteration condition is met, the image recognition device takes the i-th cylinder candidate parameter as the target cylinder parameter.

[0211] In an embodiment of the present application, for the first iteration, the image recognition device takes the first cylinder candidate parameter as the best cylinder candidate parameter; for the second iteration, when the second number of inner points corresponding to the second cylinder candidate parameter is greater than the first number of inner points corresponding to the first cylinder candidate parameter, it means that the cylindrical shape fitted by the second cylinder candidate parameter has a higher degree of fit than the cylindrical shape fitted by the first cylinder candidate parameter, and the image recognition device takes the second cylinder candidate parameter as the best cylinder candidate parameter; for the i-th iteration, when the i-th number of inner points is greater than or equal to the maximum number of inner points corresponding to the cylinder candidate parameter before the i-th cylinder candidate parameter, the image recognition device takes the i-th cylinder candidate parameter as the best cylinder candidate parameter, and so on, until the iteration condition is met, the image recognition device takes the best cylinder candidate parameter obtained in the last iteration as the target cylinder parameter.

[0212] S1055. When the number of the i-th inner points is less than the maximum number of inner points of the cylinder candidate parameter before the i-th cylinder candidate parameter and the iteration condition is met, the cylinder candidate parameter corresponding to the maximum number of inner points is used as the target cylinder parameter.

[0213] In an embodiment of the present application, when the number of the i-th inner points is less than the maximum number of inner points corresponding to the cylinder candidate parameter before the i-th cylinder candidate parameter and the iteration condition is met, the image recognition device takes the cylinder candidate parameter corresponding to the maximum number of inner points as the target cylinder parameter.

[0214] In the embodiment of the present application, for the first iteration, the image recognition device uses the first cylinder candidate parameter as the best cylinder candidate parameter. For the second iteration, when the second number of inner points corresponding to the second cylinder candidate parameter is less than the first number of inner points corresponding to the first cylinder candidate parameter, it means that the cylinder shape fitted by the first cylinder candidate parameter is more well-fitted than the cylinder shape fitted by the second cylinder candidate parameter, and the image recognition device continues to use the first cylinder candidate parameter as the best cylinder candidate parameter. For the ith iteration, when the ith number of inner points is less than the maximum number of inner points corresponding to the cylinder candidate parameter before the ith cylinder candidate parameter, the image recognition device uses the cylinder candidate parameter corresponding to the maximum number of inner points as the best cylinder candidate parameter. And so on, until the iteration condition is met, the image recognition device uses the best cylinder candidate parameter obtained in the last iteration as the target cylinder parameter.

[0215] S1056. When the iteration condition is not met, the i+1th cylinder candidate parameters are calculated based on the point cloud data and the normal of each point until the target cylinder parameters that meet the iteration condition are obtained.

[0216] In an embodiment of the present application, when the iteration condition is not met, exemplarily, when the number of iterations does not reach a preset threshold, the image recognition device performs the next iteration, randomly re-selects sampling points and their corresponding normals from the point cloud data and the normal of each point, and calculates the i+1th cylinder candidate parameters until the target cylinder parameters that meet the iteration condition are obtained.

[0217] It can be understood that in the embodiment of the present application, the image recognition device can make the cylindrical shape fitted by the target cylinder candidate parameters closer to the target object in the target image by continuously optimizing the cylinder candidate parameters in the iterative process, thereby improving the accuracy of image recognition.

[0218] The present application embodiment provides an image recognition device 5, such as Fig.17 As shown, the image recognition device 5 includes a collection unit 100, a calculation unit 200, a detection unit 300, a determination unit 400, and a recognition unit 500, wherein:

[0219] The acquisition unit 100 is used to acquire a target image and obtain point cloud data of the target image;

[0220] The calculation unit 200 is used to calculate the normal of the point cloud data to obtain the normal of each point in the point cloud data;

[0221] The calculation unit 200 is further used to calculate the i-th cylinder candidate parameter based on the point cloud data and the normal of each point; wherein i is a positive integer greater than or equal to 1;

[0222] The detection unit 300 is used to perform inner and outer detection on the i-th cylinder candidate parameter to obtain an i-th detection result;

[0223] The determination calculation unit 400 is used to perform iterative search processing based on the i-th detection result, the point cloud data and the normal of each point to determine the target cylinder parameters;

[0224] The recognition unit 500 is used to planarly unfold the surface of the target image based on the target cylinder parameters to obtain a recognition result of the cylindrical surface.

[0225] In some embodiments of the present application, the computing unit 200 is further used to determine, for each point in the point cloud data, a point cloud subset that includes at least one neighboring point within a preset neighborhood range of each point; and to perform principal component analysis on the point cloud subset of each point to obtain a first principal component vector of each point; and to obtain coordinate information of each point in the point cloud data; and to determine a normal vector of each point based on the coordinate information of each point and the first principal component vector of each point; and to constrain the normal vector of each point to obtain a normal line of each point.

[0226] In some embodiments of the present application, the calculation unit 200 is also used to determine the angle between the normal vector of each point and the optical axis direction vector; wherein the optical axis direction vector represents the optical axis direction of the acquisition device that acquires the target image; and based on the angle and a preset normal constraint value, the normal vector of each point corresponding to the angle is constrained respectively to obtain the normal of each point; wherein the normal vector of each point corresponds to an angle.

[0227] In some embodiments of the present application, the detection unit 300 is also used to determine, from the target image, a target object constituted by the i-th cylinder candidate parameters; wherein the target object is an image having a cylindrical shape; and to obtain, from the point cloud data, inner point cloud data belonging to the target object; and, based on the inner point cloud data, to calculate the center of gravity of the corresponding inner point cloud; and, based on the center of gravity, the i-th cylinder candidate parameters and the optical axis direction vector, to perform inner and outer side detection on the i-th cylinder candidate parameters to obtain the i-th detection result; wherein the optical axis direction vector represents the optical axis direction of the acquisition device that acquires the target image.

[0228] In some embodiments of the present application, the detection unit 300 is also used to construct a center of gravity vector based on the i-th cylinder center point and the center of gravity; and when the center of gravity vector is consistent with the optical axis direction vector, the i-th detection result that the i-th cylinder candidate parameters constitute the outer cylinder is obtained, and the inside and outside detection of the i-th cylinder candidate parameters is completed; and when the center of gravity vector is inconsistent with the optical axis direction vector, the i-th detection result that the i-th cylinder candidate parameters constitute the inner cylinder is obtained, and the inside and outside detection of the i-th cylinder candidate parameters is completed.

[0229] In some embodiments of the present application, the computing unit 200 is further used to select the i-th sampling point data from the point cloud data; wherein the i-th sampling point data includes first sampling point data and second sampling point data; and from the normal of each point, obtain the first normal corresponding to the first sampling point data and the second normal determined by the first sampling point data; and based on the first number of sampling points, the first normal, the first number of sampling points and the second normal, determine the i-th cylinder axis vector, the i-th cylinder center point and the i-th cylinder radius; and the i-th cylinder axis vector, the i-th cylinder center point and the i-th cylinder radius constitute the i-th cylinder candidate parameter.

[0230] In some embodiments of the present application, the calculation unit 200 is further used to calculate the i-th cylinder axis vector based on the first normal and the second normal; determine the three-dimensional projection space matrix parameters of the straight line where the first normal is located according to the first sampling point data and the first normal; and determine the plane where the second sampling point data is located according to the second sampling point data and the second normal; and determine the i-th cylinder center point where the straight line and the plane intersect according to the three-dimensional projection space matrix parameters of the straight line where the first normal is located and the plane where the second sampling point data is located; and determine the i-th cylinder radius according to the first sampling point data and the i-th cylinder center point; or determine the i-th cylinder radius according to the second sampling point data and the i-th cylinder center point.

[0231] In some embodiments of the present application, the calculation unit 200 is further used to determine the three-dimensional projection space matrix parameters of the straight line where the second normal is located according to the second sampling point data and the second normal; and determine the plane where the first sampling point data is located according to the first sampling point data and the first normal; and determine the i-th cylinder center point where the straight line intersects the plane according to the three-dimensional projection space matrix parameters of the straight line where the second normal is located and the plane where the first sampling point data is located.

[0232] In some embodiments of the present application, the detection unit 300 is further used to obtain a first direction value based on the first sampling point data, the first normal and the i-th cylinder center point; and to obtain a second direction value based on the second sampling point data, the second normal and the i-th cylinder center point; and when the first direction value and the second direction value are both greater than 0, the i-th detection result that the i-th cylinder candidate parameter constitutes an outer cylinder is obtained, and the inside and outside detection of the i-th cylinder candidate parameter is completed; and when any one of the first direction value and the second direction value is less than 0, the i-th detection result that the i-th cylinder candidate parameter constitutes an inner cylinder is obtained, and the inside and outside detection of the i-th cylinder candidate parameter is completed.

[0233] In some embodiments of the present application, the determination unit 400 is further used to, when the i-th detection result is that the i-th cylinder candidate parameters constitute an inner cylinder, calculate the i+1-th cylinder candidate parameters based on the point cloud data and the normal of each point until the target cylinder parameters that meet the iteration condition are obtained; and when the i-th detection result is that the i-th cylinder candidate parameters constitute an outer cylinder, determine from the target image a target object constituted by the i-th cylinder candidate parameters; wherein the target object is an image with a cylindrical shape; and count the i-th number of inner points of the inner point cloud data belonging to the target object from the point cloud data. ; and when the i-th number of inner points is greater than or equal to the maximum number of inner points corresponding to the cylinder candidate parameter before the i-th cylinder candidate parameter, and the iteration condition is met, the i-th cylinder candidate parameter is used as the target cylinder parameter; and when the i-th number of inner points is less than the maximum number of inner points corresponding to the cylinder candidate parameter before the i-th cylinder candidate parameter, and the iteration condition is met, the cylinder candidate parameter corresponding to the maximum number of inner points is used as the target cylinder parameter; and when the iteration condition is not met, the i+1-th cylinder candidate parameter is calculated based on the point cloud data and the normal of each point, until the target cylinder parameter that meets the iteration condition is obtained.

[0234] The present application embodiment provides an image recognition device 6, such as Fig.18 As shown, the image recognition device 6 includes: a processor 125, a memory 126 and a communication bus 127. The memory 126 communicates with the processor 125 via the communication bus 127. The memory 126 stores one or more programs executable by the processor 125. When the one or more programs are executed, the processor 125 executes the image recognition method as described in any one of the above items.

[0235] An embodiment of the present application provides a computer-readable storage medium, which stores one or more programs. The one or more programs can be executed by one or more processors to implement the image recognition method as described in any one of the above items.

[0236] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems, or computer program products. Therefore, the present application may adopt the form of hardware embodiments, software embodiments, or embodiments in combination with software and hardware. Moreover, the present application may adopt the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage and optical storage, etc.) that contain computer-usable program code.

[0237] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0238] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0239] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0240] The above description is only a preferred embodiment of the present application and is not intended to limit the protection scope of the present application.

Claims

1. An image recognition method, characterized in that: include: Collecting a target image and obtaining point cloud data of the target image; Calculating the normal of the point cloud data to obtain the normal of each point in the point cloud data; Based on the point cloud data and the normal of each point, calculate the i-th cylinder candidate parameter; wherein i is a positive integer greater than or equal to 1; Performing inner and outer side detection on the i-th cylinder candidate parameter to obtain an i-th detection result; Performing an iterative search process based on the i-th detection result, the point cloud data and the normal of each point to determine the target cylinder parameters; Based on the target cylinder parameters, the surface of the target image is plane-unfolded to obtain a recognition result of the cylindrical surface; The step of calculating the candidate parameters of the i-th cylinder based on the point cloud data and the normal of each point includes: Selecting the i-th sampling point data from the point cloud data; wherein the i-th sampling point data includes the first sampling point data and the second sampling point data; Obtaining, from the normals of each point, a first normal corresponding to the first sampling point data and a second normal corresponding to the second sampling point data; Determine the i-th cylinder axis vector, the i-th cylinder center point and the i-th cylinder radius based on the first sampling point data, the first normal line, the second sampling point data and the second normal line; The i-th cylinder axis vector, the i-th cylinder center point and the i-th cylinder radius constitute the i-th cylinder candidate parameters; The step of performing inner and outer detection on the i-th cylinder candidate parameter to obtain the i-th detection result includes: Determine a target object consisting of the i-th cylinder candidate parameters from the target image; wherein the target object is an image having a cylindrical shape; From the point cloud data, acquiring inlier point cloud data belonging to the target object; According to the inlier point cloud data, calculating the center of gravity of the corresponding inlier point cloud; Based on the center of gravity, the i-th cylinder candidate parameter and the optical axis direction vector, the i-th cylinder candidate parameter is detected inside and outside to obtain the i-th detection result; wherein the optical axis direction vector represents the optical axis direction of the acquisition device that acquires the target image.

2. The method according to claim 1, characterized in that The step of calculating the normal of the point cloud data to obtain the normal of each point in the point cloud data includes: For each point in the point cloud data, determining a point cloud subset including at least one neighboring point within a preset neighboring range of each point; Performing principal component analysis on the point cloud subset of each point to obtain a first principal component vector of each point; Obtaining coordinate information of each point in the point cloud data; Determine a normal vector of each point based on the coordinate information of each point and the first principal component vector of each point; Constrain the normal vector of each point to obtain the normal line of each point.

3. The method according to claim 2, characterized in that The constraining the normal vector of each point to obtain the normal line of each point includes: Determine the angle between the normal vector of each point and the optical axis direction vector; wherein the optical axis direction vector represents the optical axis direction of the acquisition device that acquires the target image; Based on the angle and the preset normal constraint value, the normal vector of each point corresponding to the angle is constrained respectively to obtain the normal of each point; wherein the normal vector of each point corresponds to an angle.

4. The method according to claim 1, characterized in that: The step of performing inner and outer side detection on the i-th cylinder candidate parameter based on the center of gravity, the i-th cylinder candidate parameter and the optical axis direction vector to obtain the i-th detection result includes: Based on the i-th cylinder center point and the center of gravity, a center of gravity vector is constructed; When the centroid vector is consistent with the optical axis direction vector, the i-th detection result that the i-th cylinder candidate parameter constitutes the outer cylinder is obtained, and the inner and outer side detection of the i-th cylinder candidate parameter is completed; When the centroid vector is inconsistent with the optical axis direction vector, the i-th detection result that the i-th cylinder candidate parameter constitutes the inner cylinder is obtained, and the inner and outer side detection of the i-th cylinder candidate parameter is completed.

5. The method according to claim 1, characterized in that The determining of the i-th cylinder axis vector, the i-th cylinder center point and the i-th cylinder radius based on the first sampling point data, the first normal line, the second sampling point data and the second normal line comprises: Calculating the i-th cylinder axis vector based on the first normal and the second normal; Determine, according to the first sampling point data and the first normal line, a three-dimensional projection space matrix parameter of the straight line where the first normal line is located; Determine the plane where the second sampling point data is located according to the second sampling point data and the second normal; Determine the center point of the i-th cylinder where the straight line intersects the plane according to the three-dimensional projection space matrix parameters of the straight line where the first normal line is located and the plane where the second sampling point data is located; The radius of the i-th cylinder is determined according to the first sampling point data and the i-th cylinder center point; or the radius of the i-th cylinder is determined according to the second sampling point data and the i-th cylinder center point.

6. The method according to claim 1, characterized in that After calculating the i-th cylinder axis vector based on the first normal and the second normal, determining the i-th cylinder radius according to the first sampling point data and the i-th cylinder center point; Or before determining the radius of the i-th cylinder according to the second sampling point data and the i-th cylinder center point, the method further includes: Determine, according to the second sampling point data and the second normal, the matrix parameters of the straight line where the second normal is located; Determine the plane where the first sampling point data is located according to the first sampling point data and the first normal; The i-th cylinder center point where the straight line intersects the plane is determined according to the three-dimensional projection space matrix parameters of the straight line where the second normal line is located and the plane where the first sampling point data is located.

7. The method according to any one of claims 1, 5 and 6, characterized in that: The i-th cylinder candidate parameter includes: the i-th cylinder center point; the inside and outside detection of the i-th cylinder candidate parameter is performed to obtain the i-th detection result, including: Obtaining a first direction value according to the first sampling point data, the first normal line and the i-th cylinder center point; Obtaining a second direction value according to the second sampling point data, the second normal line and the i-th cylinder center point; When both the first direction value and the second direction value are greater than 0, the i-th detection result that the i-th cylinder candidate parameter constitutes an outer cylinder is obtained, and the inner and outer side detection of the i-th cylinder candidate parameter is completed; When any one of the first direction value and the second direction value is less than 0, the i-th detection result that the i-th cylinder candidate parameter constitutes the inner cylinder is obtained, and the inner and outer side detection of the i-th cylinder candidate parameter is completed.

8. The method according to claim 1, characterized in that The iterative search process based on the i-th detection result, the point cloud data and the normal of each point to determine the target cylinder parameters includes: When the i-th detection result is that the i-th cylinder candidate parameters constitute an inner cylinder, based on the point cloud data and the normal of each point, the i+1-th cylinder candidate parameters are calculated until the target cylinder parameters that meet the iteration conditions are obtained; When the i-th detection result is that the i-th cylinder candidate parameters constitute an outer cylinder, determining a target object constituted by the i-th cylinder candidate parameters from the target image; wherein the target object is an image having a cylindrical shape; From the point cloud data, counting the number of i-th inliers of the inlier point cloud data belonging to the target object; When the number of the i-th inner points is greater than or equal to the maximum number of inner points corresponding to the cylinder candidate parameter before the i-th cylinder candidate parameter and the iteration condition is satisfied, the i-th cylinder candidate parameter is used as the target cylinder parameter; When the number of the i-th inner points is less than the maximum number of inner points corresponding to the cylinder candidate parameter before the i-th cylinder candidate parameter and the iteration condition is satisfied, the cylinder candidate parameter corresponding to the maximum number of inner points is used as the target cylinder parameter; When the iteration condition is not met, the i+1th cylinder candidate parameters are calculated based on the point cloud data and the normal of each point until the target cylinder parameters that meet the iteration condition are obtained.

9. An image recognition device, characterized in that: The image recognition device includes a collection unit, a calculation unit, a detection unit, a determination unit and a recognition unit, wherein: The acquisition unit is used to acquire a target image and obtain point cloud data of the target image; The calculation unit is used to calculate the normal of the point cloud data to obtain the normal of each point in the point cloud data; The calculation unit is further used to calculate the i-th cylinder candidate parameter based on the point cloud data and the normal of each point; wherein i is a positive integer greater than or equal to 1; The detection unit is used to perform inner and outer side detection on the i-th cylinder candidate parameter to obtain an i-th detection result; The determination unit is used to perform iterative search processing based on the i-th detection result, the point cloud data and the normal of each point to determine the target cylinder parameters; The recognition unit is used to planarly unfold the surface of the target image based on the target cylinder parameters to obtain a recognition result of the cylindrical surface; The calculation unit is further used to select the i-th sampling point data from the point cloud data; wherein the i-th sampling point data includes first sampling point data and second sampling point data; obtain a first normal corresponding to the first sampling point data and a second normal corresponding to the second sampling point data from the normal of each point; determine the i-th cylinder axis vector, the i-th cylinder center point and the i-th cylinder radius based on the first sampling point data, the first normal, the second sampling point data and the second normal; the i-th cylinder axis vector, the i-th cylinder center point and the i-th cylinder radius constitute the i-th cylinder candidate parameter; The detection unit is also used to determine the target object composed of the i-th cylinder candidate parameters from the target image; wherein the target object is an image having a cylindrical shape; obtain the inner point cloud data belonging to the target object from the point cloud data; calculate the center of gravity of the corresponding inner point cloud based on the inner point cloud data; based on the center of gravity, the i-th cylinder candidate parameters and the optical axis direction vector, perform inside and outside detection on the i-th cylinder candidate parameters to obtain the i-th detection result; wherein the optical axis direction vector represents the optical axis direction of the acquisition device that acquires the target image.

10. An image recognition device, characterized in that: include: A processor, a memory and a communication bus, wherein the memory communicates with the processor via the communication bus, the memory stores one or more programs executable by the processor, and when the one or more programs are executed, the processor executes the method according to any one of claims 1 to 8.

11. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores one or more programs, and the one or more programs can be executed by one or more processors to implement the method according to any one of claims 1 to 8.

Images