An object detection method and apparatus

Abstract

This invention provides an object detection method and apparatus, relating to the field of data processing technology. The method includes: obtaining point cloud data to be detected; performing point cloud segmentation on the point cloud data to obtain non-ground points in the point cloud data; performing point cloud clustering on the non-ground points to obtain multiple point cloud clusters including laser points among the non-ground points; for each point cloud cluster, extracting global features of the laser points in the point cloud cluster; and detecting candidate objects corresponding to the point cloud cluster based on the extracted global features. Applying the object detection method provided by this invention, objects corresponding to point cloud data can be detected.

Description

Technical Field

[0001] This invention relates to the field of data processing technology, and in particular to an object detection method and apparatus. Background Technology

[0002] Today, LiDAR is widely used in the field of autonomous driving. LiDAR can collect point cloud data of the actual environment. When processing the point cloud data collected by LiDAR, autonomous vehicles expect to know the vehicles, obstacles, and other objects existing in the actual environment based on the point cloud data, and thus assist the autonomous vehicle in driving based on the information of the objects obtained.

[0003] Therefore, a solution is needed to perform object detection on point cloud data in order to detect the objects in the actual environment corresponding to the point cloud data. Summary of the Invention

[0004] The purpose of this invention is to provide an object detection method and apparatus for detecting objects in the real environment corresponding to point cloud data. The specific technical solution is as follows:

[0005] In a first aspect, embodiments of the present invention provide an object detection method, the method comprising:

[0006] Obtain the point cloud data to be detected;

[0007] The point cloud data to be detected is segmented to obtain non-ground points in the point cloud data to be detected.

[0008] Perform point cloud clustering processing on the non-ground points to obtain multiple point cloud clusters including laser points in the non-ground points;

[0009] For each point cloud cluster, extract the global features of the laser points in the point cloud cluster, and detect the candidate objects corresponding to the point cloud cluster based on the extracted global features.

[0010] In one embodiment of the present invention, extracting the global features of laser points in the point cloud cluster includes:

[0011] The laser points in the point cloud cluster are rotated and upgraded in dimensionality to obtain the first feature after processing.

[0012] The laser points in the point cloud cluster are subjected to dimensionality-upgrading processing to obtain the processed second feature;

[0013] The third feature is obtained by superimposing the features corresponding to the first and second features of the same laser point in the point cloud cluster;

[0014] The third feature is subjected to pooling to obtain the global features of the laser points in the point cloud cluster.

[0015] In one embodiment of the present invention, the step of rotating and dimensionally increasing the laser points in the point cloud cluster to obtain the processed first feature includes:

[0016] Based on the preset first rotation parameters, the laser points in the point cloud cluster are rotated.

[0017] The position information of the laser points in the point cloud cluster after rotation is subjected to dimensionality-up processing to obtain the intermediate features of the laser points in the point cloud cluster.

[0018] The intermediate feature is rotated according to the preset second rotation parameters;

[0019] The intermediate features after rotation are upgraded to obtain upgraded intermediate features, which are used as the first features.

[0020] In one embodiment of the present invention, before superimposing the features corresponding to the first and second features of the same laser point in the point cloud cluster to obtain the third feature, the method further includes:

[0021] Based on the first feature, determine whether the type of the candidate object corresponding to the point cloud cluster belongs to a valid object type;

[0022] If so, then the step of superimposing the features corresponding to the first and second features of the same laser point in the point cloud cluster to obtain the third feature is performed.

[0023] In one embodiment of the present invention, the method further includes:

[0024] Based on the extracted global features, predict the orientation of the candidate objects corresponding to the point cloud cluster.

[0025] In one embodiment of the present invention, detecting candidate objects corresponding to the point cloud cluster based on the extracted global features includes:

[0026] Based on the position information of the laser points in the point cloud cluster and the orientation predicted based on the global features, the size of the candidate object corresponding to the point cloud cluster is determined;

[0027] Based on the size of the candidate object corresponding to the point cloud cluster, determine the candidate object corresponding to the point cloud cluster.

[0028] In one embodiment of the present invention, the step of performing point cloud clustering processing on the non-ground points to obtain multiple point cloud clusters including laser points among the non-ground points includes:

[0029] According to the preset region growth rule, point cloud clustering is performed on all non-ground points to obtain multiple point cloud clusters including laser points among the non-ground points. The preset region growth rule is a region growth rule for the angle between laser point normals, and the laser point normal is the normal of the line segment determined by adjacent laser points.

[0030] Secondly, embodiments of the present invention also provide an object detection device, the device comprising:

[0031] The point cloud acquisition module is used to acquire the point cloud data to be detected;

[0032] The point cloud segmentation module is used to perform point cloud segmentation processing on the point cloud data to be detected, and to obtain non-ground points in the point cloud data to be detected.

[0033] The point cloud clustering module is used to perform point cloud clustering processing on the non-ground points to obtain multiple point cloud clusters including laser points in the non-ground points;

[0034] The object detection module is used to extract the global features of the laser points in each point cloud cluster, and detect the candidate objects corresponding to the point cloud cluster based on the extracted global features.

[0035] In one embodiment of the present invention, the object detection module includes:

[0036] The first processing submodule is used to rotate and upgrade the laser points in each point cloud cluster to obtain the processed first feature.

[0037] The second processing submodule is used to perform dimensionality-upgrading processing on the laser points in each point cloud cluster to obtain the processed second feature.

[0038] The feature overlay submodule is used to overlay the features corresponding to the first and second features of the same laser point in each point cloud cluster to obtain a third feature.

[0039] The pooling processing submodule is used to perform pooling processing on the third feature for each point cloud cluster to obtain the global feature of the laser point in the point cloud cluster.

[0040] The object detection submodule is used to detect candidate objects corresponding to each point cloud cluster based on the extracted global features.

[0041] In one embodiment of the present invention, the first processing submodule is specifically used for:

[0042] Based on the preset first rotation parameters, the laser points in the point cloud cluster are rotated.

[0043] The position information of the laser points in the point cloud cluster after rotation is subjected to dimensionality-up processing to obtain the intermediate features of the laser points in the point cloud cluster.

[0044] The intermediate feature is rotated according to the preset second rotation parameters;

[0045] The intermediate features after rotation are upgraded to obtain upgraded intermediate features, which are used as the first features.

[0046] In one embodiment of the present invention, the apparatus further includes:

[0047] The type determination submodule is used to determine, for each point cloud cluster, whether the type of the candidate object corresponding to the point cloud cluster belongs to a valid object type based on the first feature before the third feature is obtained by superimposing the features corresponding to the same laser point in the point cloud cluster in the first feature and the second feature. If yes, the feature superposition submodule is triggered.

[0048] In one embodiment of the present invention, the apparatus further includes:

[0049] The orientation prediction module is used to predict the orientation of the candidate objects corresponding to the point cloud cluster based on the extracted global features.

[0050] In one embodiment of the present invention, the object detection module is specifically used for:

[0051] For each point cloud cluster, global features of the laser points in the point cloud cluster are extracted. Based on the position information of the laser points in the point cloud cluster and the orientation predicted based on the global features, the size of the candidate object corresponding to the point cloud cluster is determined. Based on the size of the candidate object corresponding to the point cloud cluster, the candidate object corresponding to the point cloud cluster is determined.

[0052] In one embodiment of the present invention, the point cloud clustering module is specifically used for:

[0053] According to the preset region growth rule, point cloud clustering is performed on all non-ground points to obtain multiple point cloud clusters including laser points among the non-ground points. The preset region growth rule is a region growth rule for the angle between laser point normals, and the laser point normal is the normal of the line segment determined by adjacent laser points.

[0054] Thirdly, embodiments of the present invention also provide an electronic device, including a processor, a communication interface, a memory, and a communication bus, wherein the processor, the communication interface, and the memory communicate with each other through the communication bus;

[0055] Memory, used to store computer programs;

[0056] When a processor executes a program stored in memory, it implements any of the steps described in the first aspect above.

[0057] Fourthly, embodiments of the present invention also provide a computer-readable storage medium storing a computer program, which, when executed by a processor, implements the steps of any of the methods described in the first aspect above.

[0058] Beneficial effects of the embodiments of the present invention:

[0059] As can be seen from the above, when performing object detection on point cloud data using the scheme provided in this embodiment of the invention, point cloud clustering is performed on the non-ground points in the point cloud data to be detected. The laser points in each point cloud cluster can be considered as laser points corresponding to the same candidate object. Thus, for each point cloud cluster, after extracting the global features of the candidate object corresponding to that point cloud cluster, the candidate object corresponding to the point cloud cluster can be accurately detected based on the global features of each laser point in that point cloud cluster. Therefore, applying the object detection scheme provided in this embodiment of the invention can improve the accuracy of object detection. Attached Figure Description

[0060] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other embodiments can be obtained based on these drawings.

[0061] Figure 1 A flowchart illustrating the first object detection method provided in an embodiment of the present invention;

[0062] Figure 2 This is a flowchart illustrating the second object detection method provided in an embodiment of the present invention;

[0063] Figure 3 This is a flowchart illustrating the third object detection method provided in an embodiment of the present invention.

[0064] Figure 4 This is a flowchart illustrating the fourth object detection method provided in an embodiment of the present invention;

[0065] Figure 5 This is a flowchart illustrating the fifth object detection method provided in an embodiment of the present invention.

[0066] Figure 6a This is a flowchart illustrating the sixth object detection method provided in an embodiment of the present invention.

[0067] Figure 6bThis is a schematic diagram of the structure of a direction prediction network provided in an embodiment of the present invention;

[0068] Figure 7 This is a flowchart illustrating the seventh object detection method provided in an embodiment of the present invention.

[0069] Figure 8 This is a schematic diagram of the structure of the first object detection device provided in an embodiment of the present invention;

[0070] Figure 9 This is a schematic diagram of the structure of the second object detection device provided in an embodiment of the present invention;

[0071] Figure 10 This is a schematic diagram of the structure of the third object detection device provided in an embodiment of the present invention;

[0072] Figure 11 This is a schematic diagram of the structure of the fourth object detection device provided in an embodiment of the present invention;

[0073] Figure 12 This is a schematic diagram of the structure of an electronic device provided in an embodiment of the present invention. Detailed Implementation

[0074] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art based on the present invention are within the scope of protection of the present invention.

[0075] First, the subject responsible for implementing the solution provided in the embodiments of the present invention will be described.

[0076] The execution subject of the solution provided in the embodiments of the present invention can be an electronic device such as an industrial control computer, a desktop computer, or a laptop computer.

[0077] The object detection method and apparatus provided by the present invention will be described in detail below through specific embodiments.

[0078] See Figure 1 , Figure 1 This is a flowchart illustrating the first object detection method provided in an embodiment of the present invention. In this embodiment, the method includes the following steps S101-S104.

[0079] Step S101: Obtain the point cloud data to be detected.

[0080] Specifically, point cloud data of the actual environment collected by lidar can be obtained and used as the point cloud data to be detected.

[0081] LiDAR can be installed on vehicles or along roadsides. If installed on a vehicle, it can obtain point cloud data of the vehicle's driving environment, which can then be used as the point cloud data to be detected. If installed along a roadside, it can obtain point cloud data of a pre-defined area on the road, which can also be used as the point cloud data to be detected.

[0082] Step S102: Perform point cloud segmentation processing on the point cloud data to be detected to obtain non-ground points in the point cloud data to be detected.

[0083] Laser points in point cloud data can be divided into ground points and non-ground points. Ground points refer to laser points in point cloud data that correspond to the ground, while non-ground points refer to laser points in point cloud data that are not ground points.

[0084] Specifically, point cloud segmentation can be performed on the point cloud data to be detected using either of the following two methods.

[0085] In the first implementation, the distance between the lidar collecting the point cloud data to be detected and the ground can be known in advance. Thus, after obtaining the point cloud data to be detected, the ground points in the point cloud data to be detected can be determined based on the distance between the lidar and the ground, thereby determining that the lidar points in the point cloud data other than the ground points are non-ground points.

[0086] In the second implementation, existing point cloud segmentation techniques can be used to segment the point cloud data to be detected, such as ground segmentation techniques based on rule-based algorithms.

[0087] Step S103: Perform point cloud clustering on the non-ground points to obtain multiple point cloud clusters including laser points among the non-ground points.

[0088] Specifically, point cloud clustering of non-ground points can yield multiple point cloud clusters. Each point cloud cluster contains laser points that can be considered as laser points corresponding to an object, and different point cloud clusters correspond to different objects.

[0089] In one embodiment of the present invention, point cloud clustering processing of non-ground points can be performed through the following two implementation methods.

[0090] In the first implementation method, it can be achieved through the following... Figure 2 In the embodiment shown, step S103A performs point cloud clustering processing on all non-ground points, which will not be described in detail here.

[0091] In the second implementation method, existing clustering techniques can be used to perform point cloud clustering on non-ground points, which will not be described in detail here.

[0092] For example, the clustering techniques mentioned above can be grid clustering, Euclidean clustering, etc.

[0093] Step S104: For each point cloud cluster, extract the global features of the laser points in the point cloud cluster, and detect the candidate objects corresponding to the point cloud cluster based on the extracted global features.

[0094] The aforementioned candidate objects can be vehicles, traffic signs, obstacles, etc.

[0095] Specifically, when extracting global features of laser points in a point cloud cluster, it is possible to use subsequent... Figure 3 The method provided in steps S104A-S104C of the illustrated embodiment involves first extracting local features of laser points in the point cloud cluster, and then obtaining global features based on the extracted local features. For a detailed explanation of how global features are extracted using steps S104A-S104C, please refer to the subsequent sections. Figure 3 The illustrated embodiment will not be described in detail here.

[0096] In addition, existing point cloud feature extraction techniques can be used to extract global features of laser points in point cloud clusters, which will not be detailed here.

[0097] In one embodiment of the present invention, based on the global features of laser points in a point cloud cluster, candidate objects corresponding to the point cloud cluster can be detected by any of the following two implementation methods.

[0098] In the first implementation, multiple candidate object types can be pre-set, and the probability of the object corresponding to the point cloud cluster being an object of each candidate object type can be predicted based on global features. Thus, the candidate object corresponding to the point cloud cluster can be determined based on the prediction results corresponding to each candidate object type.

[0099] In one embodiment of the present invention, the probability that the object corresponding to the predicted point cloud cluster is an object of a preset type can be achieved using a pre-trained prediction network.

[0100] In the second implementation method, it can also be achieved through subsequent... Figure 3 The candidate objects corresponding to the detection point cloud clusters in steps S104D-S104E of the embodiment shown are not described in detail here.

[0101] In one embodiment of the present invention, when detecting candidate objects based on global features, the above two implementation methods can be applied to perform object detection, thereby obtaining two detection results. These two detection results can be mutually verified to ensure the accuracy of the final object detection result and improve the detection redundancy.

[0102] As can be seen from the above, when performing object detection on point cloud data using the scheme provided in this embodiment of the invention, point cloud clustering is performed on the non-ground points in the point cloud data to be detected. The laser points in each point cloud cluster can be considered as laser points corresponding to the same candidate object. Thus, for each point cloud cluster, after extracting the global features of the candidate object corresponding to that point cloud cluster, the candidate object corresponding to the point cloud cluster can be accurately detected based on the global features of each laser point in that point cloud cluster. Therefore, applying the object detection scheme provided in this embodiment of the invention can improve the accuracy of object detection.

[0103] When performing point cloud clustering on non-ground points, in addition to using existing clustering techniques, the following methods can also be used: Figure 2 In the embodiment shown, step S103A performs point cloud clustering processing.

[0104] In one embodiment of the present invention, see Figure 2 The flowchart of the second object detection method is provided. In this embodiment, the above step S103 can be implemented by the following step S103A.

[0105] Step S103A: According to the preset region growth rules, perform point cloud clustering on all non-ground points to obtain multiple point cloud clusters including laser points among the non-ground points.

[0106] The preset region growth rule is: a region growth rule based on the angle between the laser point normals.

[0107] The laser point normal is the normal to the line segment defined by adjacent laser points.

[0108] Specifically, for any laser point among non-ground points (for ease of description, this laser point is referred to as the first laser point), multiple second laser points adjacent to the first laser point can be identified. Based on the position information of the first laser point and each second laser point, the normal of the line segment formed by the first laser point and each second laser point can be determined, thereby calculating the angle between each pair of normals. If there exists an angle between two normals that is less than a preset angle threshold, then it can be determined that the second laser point corresponding to these two normals belongs to the same point cloud cluster as the first laser point.

[0109] For example, for laser point a, the laser points adjacent to laser point a include laser points b, c, and d. Based on the position information of laser points a, b, c, and d, the normal l1 of the line segment formed by laser points a and b, the normal l2 of the line segment formed by laser points a and c, and the normal l3 of the line segment formed by laser points a and d can be determined. If the angle between normals l1 and l2 is less than a preset angle threshold, then laser points a, b, and c belong to the same point cloud cluster. If the angle between normals l1 and l3 is greater than the preset angle threshold, then laser points a, b, and d belong to different point cloud clusters.

[0110] In addition, the above step S103A can also be implemented using existing region growing algorithms, which will not be described in detail here.

[0111] As can be seen from the above, when using the scheme provided by the embodiments of the present invention to detect objects, according to the preset region growth rules, all non-ground points can be divided into different point cloud clusters. In this way, when the candidate objects corresponding to each point cloud cluster are detected, the candidate objects corresponding to each non-ground point can be known. Therefore, the object detection scheme provided by the embodiments of the present invention can reduce the missed identification rate of non-ground points in point cloud data.

[0112] The following describes the specific implementation method for extracting global features of laser points in a point cloud cluster.

[0113] In one embodiment of the present invention, see Figure 3 The flowchart of the third object detection method is provided. In this embodiment, the above step S104 can be implemented by the following steps S104A-S104E.

[0114] Step S104A: For each point cloud cluster, rotate and upgrade the laser points in the point cloud cluster to obtain the processed first feature.

[0115] The features of laser points in point cloud data can be divided into rotation-dependent features and rotation-independent features. Rotation-dependent features include those representing the object's orientation and direction of motion, while rotation-independent features include shape and color features. Because point cloud data is rotation-invariant—meaning the relative positions of the laser points remain unchanged before and after rotation—the shape and color (and other orientation-independent features) of the object represented by the laser points remain unchanged after rotation. However, the orientation and direction of motion (and other orientation-dependent features) of the object represented by the laser points will change after rotation.

[0116] The first feature mentioned above can be considered as a rotation-independent feature of the laser points in the point cloud cluster.

[0117] Specifically, when processing laser points in a point cloud cluster, there can be multiple processing sequences for rotation and dimensionality upgrading. Furthermore, the number of times the rotation and dimensionality upgrading processes are performed on the laser points in the point cloud cluster can be one or more. For example, the laser points in the point cloud cluster can be rotated once first, followed by one or more dimensionality upgrading processes. Alternatively, rotation and dimensionality upgrading processes can be performed alternately multiple times.

[0118] In one embodiment of the present invention, the information of the laser points in the point cloud cluster includes their position coordinates in the point cloud data to be detected, so that the entire point cloud cluster can be represented by a position matrix composed of the position coordinates of all the laser points in the point cloud cluster. For example, the position coordinates of the laser points can be three-dimensional coordinates, and the point cloud cluster can be represented as an N*3 position matrix, where N represents the number of laser points included in the point cloud cluster.

[0119] In this case, when rotating the laser points in the point cloud cluster, the position matrix can be transformed according to the preset rotation transformation matrix.

[0120] For example, the above N*3 position matrix can be multiplied by a preset 3*3 rotation transformation matrix to obtain a new N*3 position matrix.

[0121] The rotation transformation matrix mentioned above can be set by the user based on past experience.

[0122] When performing dimensionality upscaling on laser points in a point cloud cluster, a pre-trained neural network for dimensionality upscaling can be used to perform dimensionality upscaling on the aforementioned position matrix.

[0123] For example, the neural network mentioned above could be an MLP (Multilayer Perceptron) network.

[0124] In one embodiment of the present invention, a pre-trained feature extraction network can also be used to rotate and upscale the laser points in the point cloud cluster. The feature extraction network can include a rotation layer and a feature extraction layer. The rotation layer is used to rotate the laser points in the point cloud cluster, and the feature extraction layer is used to upscale the laser points in the point cloud cluster.

[0125] Step S104B: For each point cloud cluster, perform dimensionality upscaling on the laser points in the point cloud cluster to obtain the processed second feature.

[0126] Among them, the second feature mentioned above can be considered as the rotation-related feature of the laser points in the point cloud cluster.

[0127] For details on how to perform dimensionality upscaling on laser points in a point cloud cluster, please refer to step S104A above. These details will not be repeated here.

[0128] Step S104C: For each point cloud cluster, superimpose the features corresponding to the first and second features of the same laser point in the point cloud cluster to obtain the third feature.

[0129] Specifically, when obtaining the first feature, each laser point in the point cloud cluster can be rotated and its dimensionality increased to obtain the features corresponding to each laser point in the point cloud cluster, thus obtaining the first feature including the features corresponding to each laser point. Similarly, the second feature also includes the features of each laser point in the point cloud cluster. Thus, when obtaining the third feature, the features of the same laser point can be determined from the first and second features respectively, and the determined features are then superimposed. After superimposing the features of each laser point in the point cloud cluster, the superimposed features of each laser point in the point cloud cluster are obtained, thus obtaining the third feature including the superimposed features of each laser point in the point cloud cluster.

[0130] For example, by superimposing the first feature of N*1024 and the second feature of N*1024, we can obtain the third feature of N*2048.

[0131] Step S104D: For each point cloud cluster, perform pooling on the third feature to obtain the global feature of the laser point in that point cloud cluster.

[0132] The pooling process described above can be either max pooling or average pooling.

[0133] Specifically, when performing pooling on the third feature, multiple feature units to be pooled can be identified in the third feature, and then pooling can be performed on each feature unit separately.

[0134] For example, for a third feature of N*2048, 2048 feature units can be determined, and the size of each feature unit is 1*N. By performing pooling on each feature unit, a global feature of 1*2048 can be obtained.

[0135] In one embodiment of the present invention, if the pooling process is max pooling, then when performing max pooling on a feature unit, the feature value with the largest value among the feature values ​​included in the feature unit can be selected as the pooling result of the feature unit; if the pooling process is average pooling, then when performing average pooling on a feature unit, the average value of the feature values ​​included in the feature unit can be calculated as the pooling result of the feature unit.

[0136] Step S104E: For each point cloud cluster, detect the candidate object corresponding to the point cloud cluster based on the extracted global features.

[0137] For details on the specific implementation of candidate objects corresponding to global feature detection point cloud clusters, please refer to the aforementioned [link / reference]. Figure 1 Step S104 in the illustrated embodiment will not be repeated here.

[0138] As can be seen from the above, when detecting objects using the scheme provided in the embodiments of the present invention, a first feature is obtained when the laser points in the point cloud cluster are rotated. The first feature can be considered as a rotation-independent feature of the laser points in the point cloud cluster. A second feature is obtained when the laser points in the point cloud cluster are not rotated. The second feature can be considered as a rotation-dependent feature of the laser points in the point cloud cluster. The global feature is obtained based on the first and second features. Therefore, the global feature considers both rotation-independent and rotation-dependent features, thus the global feature has a stronger representation of the laser points in the point cloud cluster. In this way, object detection based on the global feature can improve the accuracy of object detection.

[0139] When rotating and scaling up laser points in a point cloud cluster, the rotation and scaling processes can be performed alternately on the laser points. The following is an example... Figure 4 The illustrated embodiment illustrates this method.

[0140] In one embodiment of this invention, see [link to embodiment]. Figure 4 The flowchart of the fourth object detection method is provided. In this embodiment, the above step S104A can be implemented by the following steps S104A1-S104A4.

[0141] Step S104A1: Rotate the laser points in the point cloud cluster according to the preset first rotation parameters.

[0142] The first rotation parameter mentioned above can be set manually.

[0143] The specific implementation method for rotating the laser points in the point cloud cluster according to the first rotation parameter can be found in step S104A above, and will not be repeated here.

[0144] Step S104A2: Perform dimensionality-upgrading processing on the position information of the laser points in the rotated point cloud cluster to obtain the intermediate features of the laser points in the point cloud cluster.

[0145] For details on how to perform dimensionality-upgrading on the position information of laser points in a point cloud cluster, please refer to step S104A above, which will not be repeated here.

[0146] Step S104A3: Rotate the intermediate feature according to the preset second rotation parameters.

[0147] The second rotation parameter can be the same as or different from the first rotation parameter.

[0148] This step is similar to step S104A1 above, and will not be repeated here.

[0149] Step S104A4: Perform dimensionality upscaling on the rotated intermediate features to obtain the dimensionality upscaling intermediate features, which are used as the first features.

[0150] This step is similar to step S104A2 above, and will not be repeated here.

[0151] As can be seen from the above, when using the solution provided in the embodiments of the present invention to detect objects, the laser points in the point cloud cluster are rotated, upgraded in dimensionality, rotated again, and upgraded in dimensionality again in sequence. By alternating multiple rotation and dimensionality upgrade processes, the aforementioned first feature can be accurately obtained, thereby improving the accuracy of global features. Furthermore, by detecting objects based on global features, the accuracy of object detection can be improved.

[0152] In addition, the laser points in the point cloud cluster were subjected to two rotations and two dimensionality upgrades in the above embodiment. In addition, other rotations and dimensionality upgrades can be performed on the laser points in the point cloud cluster according to actual needs. This embodiment does not limit this.

[0153] In autonomous driving scenarios, the acquisition area of ​​a LiDAR radar may contain multiple objects. Therefore, the point cloud data acquired by the LiDAR radar typically includes laser points corresponding to various objects. For the objects within this acquisition area, users usually only focus on a subset of the objects in the point cloud data, such as vehicles and road obstacles. Other objects, such as buildings and bushes, do not require attention. The types of objects that the user focuses on can be called effective object types.

[0154] In one embodiment of the present invention, see Figure 5 The flowchart of the fifth object detection method is provided. In this embodiment, before obtaining the global features of the laser points in the point cloud cluster, the above method further includes the following step S105.

[0155] Step S105: Based on the first feature, determine whether the type of the candidate object corresponding to the point cloud cluster belongs to a valid object type. If yes, then execute the above step S104C.

[0156] Specifically, if the type of the candidate object corresponding to the point cloud cluster is a valid object type, it means that the candidate object corresponding to the point cloud cluster is the object of interest to the user. In this case, it is necessary to detect the candidate object corresponding to the point cloud cluster so that further processing can be carried out based on the laser points corresponding to the known candidate objects. If the type of the candidate object corresponding to the point cloud cluster is not a valid object type, it means that the candidate object corresponding to the point cloud cluster is not the object of interest to the user. In this case, it is not necessary to detect the candidate object corresponding to the point cloud cluster.

[0157] In one embodiment of the present invention, the candidate object type corresponding to the point cloud cluster can be determined by any of the following two implementation methods.

[0158] In the first implementation, the first feature can be input into a pre-trained binary classification network to obtain the judgment result output by the binary classification network based on the first feature.

[0159] The aforementioned binary classification network can be trained based on the features of the sample point cloud data of valid objects and the features of the sample point cloud data of invalid objects.

[0160] In the second implementation, the features of the effective object type can be obtained in advance, and the similarity between the first feature and the features of the effective object type can be calculated. If the feature similarity is greater than or equal to the threshold, it is determined that the type of the candidate object corresponding to the point cloud cluster belongs to the effective object type. If the feature similarity is less than the threshold, it is determined that the type of the candidate object corresponding to the point cloud cluster does not belong to the effective object type.

[0161] As can be seen from the above, when using the scheme provided by the embodiments of the present invention to detect objects, the feature overlay step and subsequent steps are only executed when the type of the candidate object corresponding to the point cloud cluster is a valid object type. This eliminates the need to process each point cloud cluster, thereby reducing the amount of data for object detection and improving the efficiency of object detection.

[0162] After extracting the global features of the point cloud cluster, based on these global features, not only can candidate objects corresponding to the point cloud cluster be detected, but the orientation of the candidate objects corresponding to the point cloud cluster can also be predicted.

[0163] In one embodiment of the present invention, see Figure 6a The present invention provides a flowchart of a sixth object detection method. In this embodiment, the above method further includes the following step S106.

[0164] Step S106: For each point cloud cluster, predict the orientation of the candidate object corresponding to the point cloud cluster based on the global features of the laser points in the point cloud cluster.

[0165] The orientation of the candidate object can be a dimension direction of the candidate object, such as the length direction, width direction, or height direction.

[0166] The orientation of the aforementioned candidate objects may also be at an angle to the various dimensional directions of the candidate objects.

[0167] Specifically, the global features of laser points in a point cloud cluster can be input into a pre-trained orientation prediction network to obtain the orientation predicted by the network.

[0168] In addition, the aforementioned direction prediction network can also predict the probability that the orientation of the candidate object is the direction corresponding to a preset angle. This allows us to obtain the probability corresponding to each preset angle predicted by the direction prediction network, thereby determining that the direction pointed to by the angle with the highest probability is the orientation of the candidate object corresponding to the point cloud cluster.

[0169] In addition to using the aforementioned orientation prediction network to predict the orientation of candidate objects, existing orientation prediction techniques can also be used to predict orientation based on the global characteristics of laser points in the point cloud cluster, which will not be detailed here.

[0170] For example, the aforementioned direction prediction technique can be a deep learning-based clustering feature direction prediction technique.

[0171] As can be seen from the above, when using the solution provided in the embodiments of the present invention to detect objects, based on the global features of the laser points in the point cloud cluster, it is possible not only to predict the candidate objects corresponding to the point cloud cluster, but also to predict the orientation of the candidate objects corresponding to the point cloud cluster. Therefore, the object detection solution provided in the embodiments of the present invention can expand the application scenarios of object detection.

[0172] In one embodiment of the present invention, see Figure 6b , Figure 6b This is a schematic diagram of the structure of a direction prediction network. Figure 6b First, the global features of the laser points in the point cloud cluster are extracted. The size of the global feature is 1*2048. Then, the orientation of the candidate object corresponding to the point cloud cluster is predicted based on the extracted global features.

[0173] When extracting global features of laser points in point cloud clusters, N*1024 local features of laser points in point cloud clusters are extracted in two ways. The local features extracted in these two ways are then superimposed to obtain N*2048 features. Max pooling is then performed on the N*2048 features to obtain 1*2048 global features.

[0174] When extracting local features of laser points from a point cloud cluster, one approach inputs an N*3 point cloud cluster into a T-Net layer. This T-Net layer includes a 3*3 coordinate transformation matrix. After rotating the laser points in the point cloud cluster using the T-Net layer, a new N*3 point cloud cluster is obtained. The rotated point cloud cluster is then input into an MLP layer. After the MLP layer performs dimensionality upscaling on the laser points in the point cloud cluster, an N*64 dimensionality feature is obtained. The N*64 feature is then input into another T-Net layer. This T-Net layer includes a 64*64 coordinate transformation matrix. After rotating the N*64 feature using the T-Net layer, a rotated N*64 feature is obtained. Finally, the rotated feature is input into another MLP layer. After the MLP layer performs dimensionality upscaling on the N*64 feature, a N*1024 dimensionality feature is obtained. This N*1024 feature can be considered the first feature mentioned above. Another path inputs an N*3 point cloud cluster into the MLP layer. After the MLP layer performs dimensionality upscaling on the laser points in the point cloud cluster, it obtains an N*64 dimensionality-upgraded feature. Then, it inputs the N*64 feature into the MLP layer. After the MLP layer performs dimensionality upscaling on the N*64 feature, it obtains an N*1024 dimensionality-upgraded feature. This N*1024 feature can be considered as the second feature mentioned above.

[0175] When predicting the orientation of candidate objects corresponding to point cloud clusters based on 1*2048 global features, the 1*2048 global features are input into an MLP layer. This MLP layer is used to predict the probability that the orientation of the candidate objects corresponding to point cloud clusters belongs to a range of 180 directions. This MLP layer can output 180 scores, each score corresponding to a range of directions. In this way, the direction range corresponding to the highest score among these 180 scores can be determined as the orientation of the candidate objects corresponding to point cloud clusters.

[0176] Since there is usually a correspondence between the size of an object and the object type, for example, a truck can correspond to one size and a car can correspond to another size, when detecting an object, the size of the object can be detected and the type of the object can be determined based on the detected object size.

[0177] In one embodiment of the present invention, see Figure 7 The flowchart of the seventh object detection method is provided. In this embodiment, the above step S104 can be implemented by the following steps S104F-S104G.

[0178] Step S104F: For each point cloud cluster, determine the size of the candidate object corresponding to the point cloud cluster based on the position information of the laser points in the point cloud cluster and the orientation predicted based on the global features of the laser points in the point cloud cluster.

[0179] Specifically, based on the position information of laser points in a point cloud cluster, the three dimensional directions of the candidate object corresponding to the point cloud cluster, as well as the size in each dimensional direction, can be determined. After predicting the orientation of the candidate object corresponding to the point cloud cluster, the dimensional dimensions corresponding to the three dimensional directions can be determined based on the pre-obtained correspondence between the object orientation and the object dimensional direction. This determines the dimensional direction and size of the candidate object in the three dimensional dimensions.

[0180] Step S104G: For each point cloud cluster, determine the candidate object corresponding to the point cloud cluster based on the size of the candidate object corresponding to the point cloud cluster.

[0181] Specifically, different types of objects have different sizes; for example, the size of an electric vehicle is different from that of a large truck. When determining candidate objects, a correspondence between the size and type of the candidate object can be obtained in advance. Thus, after obtaining the size of the candidate object corresponding to the point cloud cluster, the type of the candidate object can be determined based on this correspondence and the obtained size.

[0182] As can be seen from the above, when using the solution provided in the embodiments of the present invention to detect objects, since there is usually a correspondence between the size of the candidate object and the object type, after determining the size of the candidate object corresponding to the point cloud cluster based on the position information of the laser points in the point cloud cluster and the predicted orientation, the candidate object corresponding to the point cloud cluster can be accurately determined based on its size. Therefore, applying the object detection solution provided in the embodiments of the present invention can improve the accuracy of object detection.

[0183] Corresponding to the object detection method described above, this embodiment of the invention also provides an object detection device.

[0184] In one embodiment of the present invention, see Figure 8 A schematic diagram of the structure of a first object detection device is provided. In this embodiment, the device includes:

[0185] Point cloud acquisition module 801 is used to acquire point cloud data to be detected;

[0186] The point cloud segmentation module 802 is used to perform point cloud segmentation processing on the point cloud data to be detected to obtain non-ground points in the point cloud data to be detected.

[0187] The point cloud clustering module 803 is used to perform point cloud clustering processing on the non-ground points to obtain multiple point cloud clusters including laser points in the non-ground points;

[0188] The object detection module 804 is used to extract the global features of the laser points in each point cloud cluster, and detect the candidate objects corresponding to the point cloud cluster based on the extracted global features.

[0189] As can be seen from the above, when performing object detection on point cloud data using the scheme provided in this embodiment of the invention, point cloud clustering is performed on the non-ground points in the point cloud data to be detected. The laser points in each point cloud cluster can be considered as laser points corresponding to the same candidate object. Thus, for each point cloud cluster, after extracting the global features of the candidate object corresponding to that point cloud cluster, the candidate object corresponding to the point cloud cluster can be accurately detected based on the global features of each laser point in that point cloud cluster. Therefore, applying the object detection scheme provided in this embodiment of the invention can improve the accuracy of object detection.

[0190] In one embodiment of the present invention, see Figure 9 A schematic diagram of a second object detection device is provided. In this embodiment, the object detection module 804 includes:

[0191] The first processing submodule 804A is used to rotate and upgrade the laser points in each point cloud cluster to obtain the processed first feature.

[0192] The second processing submodule 804B is used to perform dimensionality-upgrading processing on the laser points in each point cloud cluster to obtain the processed second feature.

[0193] The feature overlay submodule 804C is used to overlay the features corresponding to the first and second features of the same laser point in each point cloud cluster to obtain a third feature;

[0194] The pooling processing submodule 804D is used to perform pooling processing on the third feature for each point cloud cluster to obtain the global feature of the laser point in the point cloud cluster.

[0195] The object detection submodule 804E is used to detect candidate objects corresponding to each point cloud cluster based on the extracted global features.

[0196] As can be seen from the above, when detecting objects using the scheme provided in the embodiments of the present invention, a first feature is obtained when the laser points in the point cloud cluster are rotated. The first feature can be considered as a rotation-independent feature of the laser points in the point cloud cluster. A second feature is obtained when the laser points in the point cloud cluster are not rotated. The second feature can be considered as a rotation-dependent feature of the laser points in the point cloud cluster. The global feature is obtained based on the first and second features. Therefore, the global feature considers both rotation-independent and rotation-dependent features, thus the global feature has a stronger representation of the laser points in the point cloud cluster. In this way, object detection based on the global feature can improve the accuracy of object detection.

[0197] In one embodiment of the present invention, the first processing submodule 804A is specifically used for:

[0198] Based on the preset first rotation parameters, the laser points in the point cloud cluster are rotated.

[0199] The position information of the laser points in the point cloud cluster after rotation is subjected to dimensionality-up processing to obtain the intermediate features of the laser points in the point cloud cluster.

[0200] The intermediate feature is rotated according to the preset second rotation parameters;

[0201] The intermediate features after rotation are upgraded to obtain upgraded intermediate features, which are used as the first features.

[0202] As can be seen from the above, when using the solution provided in the embodiments of the present invention to detect objects, the laser points in the point cloud cluster are rotated, upgraded in dimensionality, rotated again, and upgraded in dimensionality again in sequence. By alternating multiple rotation and dimensionality upgrade processes, the aforementioned first feature can be accurately obtained, thereby improving the accuracy of global features. Furthermore, by detecting objects based on global features, the accuracy of object detection can be improved.

[0203] In one embodiment of the present invention, see Figure 10 A schematic diagram of a third object detection device is provided. In this embodiment, the device further includes:

[0204] The type determination submodule 804F is used to determine, for each point cloud cluster, whether the type of the candidate object corresponding to the point cloud cluster belongs to a valid object type based on the first feature before the third feature is obtained by superimposing the features corresponding to the same laser point in the first feature and the second feature. If yes, the feature superposition submodule 804C is triggered.

[0205] As can be seen from the above, when using the scheme provided by the embodiments of the present invention to detect objects, the feature overlay step and subsequent steps are only executed when the type of the candidate object corresponding to the point cloud cluster is a valid object type. This eliminates the need to process each point cloud cluster, thereby reducing the amount of data for object detection and improving the efficiency of object detection.

[0206] In one embodiment of the present invention, see Figure 11 A schematic diagram of a fourth object detection device is provided. In this embodiment, the device further includes:

[0207] The orientation prediction module 805 is used to predict the orientation of the candidate object corresponding to the point cloud cluster based on the extracted global features.

[0208] As can be seen from the above, when using the solution provided in the embodiments of the present invention to detect objects, based on the global features of the laser points in the point cloud cluster, it is possible not only to predict the candidate objects corresponding to the point cloud cluster, but also to predict the orientation of the candidate objects corresponding to the point cloud cluster. Therefore, the object detection solution provided in the embodiments of the present invention can expand the application scenarios of object detection.

[0209] In one embodiment of the present invention, the object detection module 804 is specifically used for:

[0210] For each point cloud cluster, global features of the laser points in the point cloud cluster are extracted. Based on the position information of the laser points in the point cloud cluster and the orientation predicted based on the global features, the size of the candidate object corresponding to the point cloud cluster is determined. Based on the size of the candidate object corresponding to the point cloud cluster, the candidate object corresponding to the point cloud cluster is determined.

[0211] As can be seen from the above, when using the solution provided in the embodiments of the present invention to detect objects, since there is usually a correspondence between the size of the candidate object and the object type, after determining the size of the candidate object corresponding to the point cloud cluster based on the position information of the laser points in the point cloud cluster and the predicted orientation, the candidate object corresponding to the point cloud cluster can be accurately determined based on its size. Therefore, applying the object detection solution provided in the embodiments of the present invention can improve the accuracy of object detection.

[0212] In one embodiment of the present invention, the point cloud clustering module 803 is specifically used for:

[0213] According to the preset region growth rule, point cloud clustering is performed on all non-ground points to obtain multiple point cloud clusters including laser points among the non-ground points. The preset region growth rule is a region growth rule for the angle between laser point normals, and the laser point normal is the normal of the line segment determined by adjacent laser points.

[0214] As can be seen from the above, when using the scheme provided by the embodiments of the present invention to detect objects, according to the preset region growth rules, all non-ground points can be divided into different point cloud clusters. In this way, when the candidate objects corresponding to each point cloud cluster are detected, the candidate objects corresponding to each non-ground point can be known. Therefore, the object detection scheme provided by the embodiments of the present invention can reduce the missed identification rate of non-ground points in point cloud data.

[0215] This invention also provides an electronic device, such as... Figure 12 As shown, it includes a processor 1201, a communication interface 1202, a memory 1203, and a communication bus 1204. The processor 1201, communication interface 1202, and memory 1203 communicate with each other via the communication bus 1204.

[0216] Memory 1203 is used to store computer programs;

[0217] When processor 1201 executes the program stored in memory 1203, it performs the following steps:

[0218] Obtain the point cloud data to be detected;

[0219] The point cloud data to be detected is segmented to obtain non-ground points in the point cloud data to be detected.

[0220] Perform point cloud clustering processing on the non-ground points to obtain multiple point cloud clusters including laser points in the non-ground points;

[0221] For each point cloud cluster, extract the global features of the laser points in the point cloud cluster, and detect the candidate objects corresponding to the point cloud cluster based on the extracted global features.

[0222] The processor 1201 executes the program stored in the memory 1203 to implement other schemes for object detection, which are the same as the schemes mentioned in the aforementioned method embodiments, and will not be described again here.

[0223] The communication bus mentioned in the above electronic devices can be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. This communication bus can be divided into address bus, data bus, control bus, etc. For ease of illustration, only one thick line is used to represent it in the diagram, but this does not mean that there is only one bus or one type of bus.

[0224] The communication interface is used for communication between the aforementioned electronic devices and other devices.

[0225] The memory may include random access memory (RAM) or non-volatile memory (NVM), such as at least one disk storage device. Optionally, the memory may also be at least one storage device located remotely from the aforementioned processor.

[0226] The processors mentioned above can be general-purpose processors, including central processing units (CPUs), network processors (NPs), etc.; they can also be digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components.

[0227] In another embodiment of the present invention, a computer-readable storage medium is also provided, which stores a computer program that, when executed by a processor, implements the steps of any of the above-described object detection methods.

[0228] In another embodiment of the present invention, a computer program product containing instructions is also provided, which, when run on a computer, causes the computer to execute any of the object detection methods described above.

[0229] In the above embodiments, implementation can be achieved entirely or partially through software, hardware, firmware, or any combination thereof. When implemented using software, it can be implemented entirely or partially in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, all or part of the processes or functions described in the embodiments of the present invention are generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired (e.g., coaxial cable, fiber optic, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that integrates one or more available media. The available medium can be a magnetic medium (e.g., floppy disk, hard disk, magnetic tape), an optical medium (e.g., DVD), or a semiconductor medium (e.g., solid state disk (SSD)).

[0230] It should be noted that, in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0231] The various embodiments in this specification are described in a related manner. Similar or identical parts between embodiments can be referred to mutually. Each embodiment focuses on describing the differences from other embodiments. In particular, the embodiments of apparatus, electronic devices, computer-readable storage media, and computer program products are basically similar to the method embodiments, and therefore the descriptions are relatively simple; relevant parts can be referred to the descriptions of the method embodiments.

[0232] The above description is merely a preferred embodiment of the present invention and is not intended to limit the scope of protection of the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention are included within the scope of protection of the present invention.

Claims

1. An object detection method, characterized in that, The method includes: Obtain the point cloud data to be detected; The point cloud data to be detected is segmented to obtain non-ground points in the point cloud data to be detected. Perform point cloud clustering processing on the non-ground points to obtain multiple point cloud clusters including laser points in the non-ground points; For each point cloud cluster, extract the global features of the laser points in the point cloud cluster, and detect the candidate objects corresponding to the point cloud cluster based on the extracted global features. The extraction of global features of laser points in the point cloud cluster includes: The laser points in the point cloud cluster are rotated and upgraded in dimensionality to obtain the first feature after processing. The laser points in the point cloud cluster are subjected to dimensionality-upgrading processing to obtain the processed second feature; The third feature is obtained by superimposing the features corresponding to the first and second features of the same laser point in the point cloud cluster; The third feature is subjected to pooling to obtain the global features of the laser points in the point cloud cluster.

2. The method according to claim 1, characterized in that, The process of rotating and increasing the dimensionality of the laser points in the point cloud cluster to obtain the processed first feature includes: Based on the preset first rotation parameters, the laser points in the point cloud cluster are rotated. The position information of the laser points in the point cloud cluster after rotation is subjected to dimensionality-up processing to obtain the intermediate features of the laser points in the point cloud cluster. The intermediate feature is rotated according to the preset second rotation parameters; The intermediate features after rotation are upgraded to obtain upgraded intermediate features, which are used as the first features.

3. The method according to claim 1 or 2, characterized in that, Before superimposing the features corresponding to the first and second features of the same laser point in the point cloud cluster to obtain the third feature, the method further includes: Based on the first feature, determine whether the type of the candidate object corresponding to the point cloud cluster belongs to a valid object type; If so, then the step of superimposing the features corresponding to the first and second features of the same laser point in the point cloud cluster to obtain the third feature is performed.

4. The method according to claim 1 or 2, characterized in that, The method further includes: Based on the extracted global features, predict the orientation of the candidate objects corresponding to the point cloud cluster.

5. The method according to claim 4, characterized in that, The step of detecting candidate objects corresponding to the point cloud cluster based on the extracted global features includes: Based on the position information of the laser points in the point cloud cluster and the orientation predicted based on the global features, the size of the candidate object corresponding to the point cloud cluster is determined; Based on the size of the candidate object corresponding to the point cloud cluster, determine the candidate object corresponding to the point cloud cluster.

6. The method according to claim 1 or 2, characterized in that, The point cloud clustering process performed on the non-ground points yields multiple point cloud clusters including laser points among the non-ground points, including: According to the preset region growth rule, point cloud clustering is performed on all non-ground points to obtain multiple point cloud clusters including laser points among the non-ground points. The preset region growth rule is a region growth rule for the angle between laser point normals, and the laser point normal is the normal of the line segment determined by adjacent laser points.

7. An object detection device, characterized in that, The device includes: The point cloud acquisition module is used to acquire the point cloud data to be detected; The point cloud segmentation module is used to perform point cloud segmentation processing on the point cloud data to be detected, and to obtain non-ground points in the point cloud data to be detected. The point cloud clustering module is used to perform point cloud clustering processing on the non-ground points to obtain multiple point cloud clusters including laser points in the non-ground points; The object detection module is used to extract the global features of the laser points in each point cloud cluster, and detect the candidate objects corresponding to the point cloud cluster based on the extracted global features. The object detection module includes: The first processing submodule is used to rotate and upgrade the laser points in each point cloud cluster to obtain the processed first feature. The second processing submodule is used to perform dimensionality-upgrading processing on the laser points in each point cloud cluster to obtain the processed second feature. The feature overlay submodule is used to overlay the features corresponding to the first and second features of the same laser point in each point cloud cluster to obtain a third feature. The pooling processing submodule is used to perform pooling processing on the third feature for each point cloud cluster to obtain the global feature of the laser point in the point cloud cluster. The object detection submodule is used to detect candidate objects corresponding to each point cloud cluster based on the extracted global features.

8. An electronic device, characterized in that, It includes a processor, a communication interface, a memory, and a communication bus, wherein the processor, the communication interface, and the memory communicate with each other through the communication bus; Memory, used to store computer programs; A processor, when executing a program stored in memory, implements the steps of the method described in any one of claims 1-6.

9. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, implements the steps of the method described in any one of claims 1-6.

Images