Method, apparatus, storage medium, and electronic device for processing lidar point cloud data

Abstract

The present application provides a method, device, storage medium and electronic device for processing lidar point cloud data. Based on the point cloud data of the target area transmitted by the lidar, a corresponding front view is constructed; based on the front view and the background map corresponding to the target area, the moving pixel points in the front view are determined; the moving pixel points in the front view are segmented to obtain a classification result; labels are added to the coordinate points corresponding to the moving pixel points in the point cloud data based on the classification result. By converting the three-dimensional point cloud data into a two-dimensional front view and performing calculations at the two-dimensional level, the moving pixel points in the front view can be determined without the need for point cloud rotation operations, greatly reducing the amount of calculation. By segmenting the moving pixel points in the front view and classifying the moving pixel points belonging to the same moving object, different moving objects can be distinguished. Finally, by adding labels, it is convenient for users to observe.

Description

Technical Field

[0001] This application relates to the field of lidar, and in particular, to a method, device, storage medium, and electronic device for processing lidar point cloud data. Background Art

[0002] In the field of traffic applications, it is necessary to identify moving objects in a target scene. Specifically, information about the target scene can be collected, and then identification can be performed based on the collected information to determine the moving objects in the target scene.

[0003] As an information capture tool, lidar has the advantages of accuracy and efficiency. It can be used to collect information about a target scene (or target area) to obtain corresponding point cloud data. How to complete the identification of moving objects based on the point cloud data collected by lidar has become a difficult problem that those skilled in the art have been continuously concerned about. Summary of the Invention

[0004] The purpose of this application is to provide a method, device, storage medium, and electronic device for processing lidar point cloud data to at least partially improve the above problems.

[0005] To achieve the above purpose, the technical solutions adopted in the embodiments of this application are as follows:

[0006] In a first aspect, an embodiment of this application provides a method for processing lidar point cloud data. The method includes: constructing a corresponding front view based on the point cloud data of a target area transmitted by the lidar; determining moving pixel points in the front view based on the front view and the background map corresponding to the target area; segmenting the moving pixel points in the front view, classifying the moving pixel points belonging to the same moving object to obtain a classification result, where the classification result includes the mapping relationship between each moving pixel point and the moving object to which it belongs; adding labels to the coordinate points corresponding to the moving pixel points in the point cloud data based on the classification result.

[0007] In the solution of this application, by converting three-dimensional point cloud data into a two-dimensional front view and performing calculations at the two-dimensional level, moving pixel points in the front view can be determined without the need for point cloud rotation operations, greatly reducing the amount of calculation. By segmenting the moving pixel points in the front view and classifying the moving pixel points belonging to the same moving object, different moving objects can be distinguished. Finally, by adding labels, it is convenient for users to observe.

[0008] Optionally, the step of constructing a corresponding front view based on the point cloud data of the target area transmitted by the lidar includes: determining the position information of the coordinate point in the front view based on the coordinate information of the coordinate point in the point cloud data; determining the distance value information between the coordinate point and the lidar based on the coordinate information of the coordinate point in the point cloud data; filling the distance value information as the pixel value of the pixel point corresponding to the position information in the front view to complete the construction of the front view.

[0009] In the solution of this application, the front view serves as the basis for identifying moving pixel points in this application, and the front view is constructed quickly, accurately, and reasonably to ensure the accuracy of moving object recognition.

[0010] Optionally, the position information includes the abscissa index in the front view and the ordinate index in the front view, and the coordinate information of the coordinate point includes the X-axis coordinate, Y-axis coordinate, and Z-axis coordinate in the Cartesian coordinate system of the lidar; the step of determining the position information of the coordinate point in the front view based on the coordinate information of the coordinate point in the point cloud data includes: determining the abscissa index based on the X-axis coordinate and the Y-axis coordinate of the coordinate point; determining the ordinate index based on the X-axis coordinate, the Y-axis coordinate, and the Z-axis coordinate of the coordinate point.

[0011] In the solution of this application, the reasonable construction of the front view is ensured by accurately obtaining the position information.

[0012] Optionally, the step of determining the moving pixel points in the front view based on the front view and the background map corresponding to the target area includes: determining whether the i-th pixel point in the front view is the moving pixel point based on the pixel value of the i-th pixel point in the front view and the pixel value of the i-th pixel point in the background map.

[0013] In the solution of this application, the accuracy of subsequent moving object recognition is ensured by accurately identifying the moving pixel points in the front view.

[0014] Optionally, the step of determining whether the i-th pixel point in the front view is the moving pixel point based on the pixel value of the i-th pixel point in the front view and the pixel value of the i-th pixel point in the background image includes: determining whether the i-th pixel point satisfies any one of a first condition and a second condition; when the i-th pixel point satisfies any one of the first condition and the second condition, determining that the i-th pixel point in the front view is the moving pixel point; wherein, the first condition is that the difference between the pixel value of the i-th pixel point in the background image and the pixel value of the i-th pixel point in the front view is greater than a first preset value, and the second condition is that the pixel value of the i-th pixel point in the front view is not 0, and the pixel value of the i-th pixel point in the background image is 0.

[0015] In the solution of this application, all moving pixel points in the front view can be accurately identified.

[0016] Optionally, the method further includes: when the i-th pixel point in the front view satisfies a third condition, determining whether the number of consecutive times the i-th pixel point satisfies the third condition is greater than a second preset number; wherein, the third condition is that the difference between the pixel value of the i-th pixel point in the front view and the pixel value of the i-th pixel point in the background image is greater than a first preset value; if it is greater than the second preset number, then assign the pixel value of the latest i-th pixel point in the front view to the pixel value of the i-th pixel point in the background image.

[0017] In the process of determining the moving pixel points in the solution of this application, the pixel values of each pixel point in the background image need to be referred to. The accuracy of the information in the background image directly affects the recognition result of the moving pixel points. Therefore, it is necessary to update the background image to ensure the accuracy of the recognition result of the moving pixel points.

[0018] Optionally, the method further includes: when the pixel value of the i-th pixel point in the front view is 0, determining whether the number of consecutive times the pixel value of the i-th pixel point is 0 is greater than a third preset number; if it is greater than the third preset number, then assign the pixel value of the i-th pixel point in the background image to 0.

[0019] In the process of determining the moving pixel points in the solution of this application, the pixel values of each pixel point in the background image need to be referred to. The accuracy of the information in the background image directly affects the recognition result of the moving pixel points. Therefore, it is necessary to update the background image to ensure the accuracy of the recognition result of the moving pixel points.

[0020] Optionally, the step of segmenting the moving pixel points in the front view and classifying the moving pixel points belonging to the same moving object to obtain a classification result includes: performing rough segmentation on the moving pixel points in the front view by using a 2D region growing algorithm to obtain a preliminary classification result.

[0021] In the solution of this application, through rough segmentation, a preliminary classification result can be obtained quickly and accurately.

[0022] Optionally, the step of using the 2D region growing algorithm to roughly segment the moving pixel points in the front view to obtain a preliminary classification result includes: selecting any one of the unclassified moving pixel points in the front view as a seed point; determining whether there are adjacent moving pixel points for the seed point, where the adjacent moving pixel points are moving pixel points with a distance less than the fourth distance threshold from the seed point; if so, marking the adjacent moving pixel points as belonging to the same moving object as the seed point; taking the adjacent moving pixel points as new seed points, and repeating the determination of whether there are adjacent moving pixel points for the seed point until there are no adjacent moving pixel points for all seed points.

[0023] Optionally, the step of segmenting the moving pixel points in the front view, classifying the moving pixel points belonging to the same moving object to obtain a classification result further includes: restoring the moving pixel points in the preliminary classification result to a point cloud view, where the coordinate points after conversion of the moving pixel points belonging to the same moving object belong to the same cluster of point clouds; performing 3D clustering operations on each cluster of point clouds in the point cloud view, classifying the coordinate points belonging to different moving objects in the same cluster of point clouds to determine the mapping relationship between each moving pixel point and the moving object it belongs to, and obtaining the final classification result.

[0024] In the solution of this application, the problem that the rough segmentation has poor segmentation ability for the X-axis is improved in the above way, and the accuracy of the segmentation result is ensured.

[0025] Optionally, the step of segmenting the moving pixel points in the front view, classifying the moving pixel points belonging to the same moving object to obtain a classification result further includes: performing a 2D watershed algorithm on the moving pixel points in the preliminary classification result, and further classifying the moving pixel points determined by the rough segmentation to belong to the same moving object to determine the mapping relationship between each moving pixel point and the moving object it belongs to, and obtaining the final classification result.

[0026] In the solution of this application, the problem that the rough segmentation has poor segmentation ability for the X-axis is improved in the above way, and the accuracy of the segmentation result is ensured.

[0027] In a second aspect, an embodiment of the present application provides a device, which includes: a first processing unit, configured to construct a corresponding front view based on the point cloud data of a target area transmitted by a lidar; the first processing unit is further configured to determine moving pixel points in the front view based on the front view and a background map corresponding to the target area; the first processing unit is further configured to segment the moving pixel points in the front view and classify the moving pixel points belonging to the same moving object to obtain a classification result, where the classification result includes the mapping relationship between each moving pixel point and the moving object to which it belongs; a second processing unit, configured to add labels to the coordinate points corresponding to the moving pixel points in the point cloud data based on the classification result.

[0028] The device provided in the second aspect can execute the method described in any implementation manner of the first aspect, which will not be elaborated here one by one. Optionally, the first processing unit and the second processing unit can be separately provided or integrated into one unit, that is, the processing unit. The present application does not make specific limitations on the specific implementation manners of the first processing unit and the second processing unit.

[0029] Optionally, the device provided in the second aspect may further include a storage unit, which stores programs or instructions. When the first processing unit and the second processing unit execute the programs or instructions, the device provided in the second aspect can execute any possible implementation manner of the first aspect.

[0030] The device provided in the second aspect can be a lidar or a central control system of a lidar, or a computer device communicatively connected to the lidar, such as a mobile phone, a vehicle computer, a server, etc. It can also be a chip (system) or other components or assemblies that can be provided in a lidar (or a central control system of a lidar, or a computer device communicatively connected to the lidar). The present application does not make limitations on this.

[0031] In addition, the technical effects of the device described in the second aspect can refer to the technical effects of the method described in any implementation manner of the first aspect, which will not be elaborated here.

[0032] In a third aspect, an embodiment of the present application provides a storage medium, on which a computer program is stored, and when the computer program is executed by a processor, it implements the method described in any possible implementation manner of the first aspect.

[0033] In a fourth aspect, an embodiment of the present application provides an electronic device, which includes: a processor and a memory, and the memory is used to store one or more programs; when the one or more programs are executed by the processor, the method described in any possible implementation manner of the first aspect is implemented.

[0034] Fifth aspect, an embodiment of the present application provides a computer program product, which includes: a computer program (which can also be referred to as code or instruction), when the computer program is run, it causes the computer to execute the method described in any possible implementation manner of the first aspect.

[0035] Sixth aspect, an embodiment of the present application provides a chip system, the chip system includes: a processor and a memory, the memory is used to store one or more programs; when the one or more programs are executed by the processor, the method described in any possible implementation manner of the first aspect is implemented.

[0036] To make the above objects, features and advantages of the present application more obvious and understandable, the following specifically gives preferred embodiments and cooperates with the attached drawings for detailed description as follows. Description of the Drawings

[0037] To more clearly illustrate the technical solutions of the embodiments of the present application, the drawings required to be used in the embodiments will be briefly introduced below. It should be understood that the following drawings only show some embodiments of the present application, and therefore should not be regarded as limiting the scope. For those of ordinary skill in the art, other related drawings can also be obtained based on these drawings without creative efforts.

[0038] Figure 1 It is a schematic structural diagram of an electronic device provided by an embodiment of the present application;

[0039] Figure 2 It is one of the schematic flowcharts of the lidar point cloud data processing method provided by an embodiment of the present application;

[0040] Figure 3 It is a measurement schematic diagram of a lidar provided by an embodiment of the present application;

[0041] Figure 4 It is another schematic flowchart of the lidar point cloud data processing method provided by an embodiment of the present application;

[0042] Figure 5 It is a third schematic flowchart of the lidar point cloud data processing method provided by an embodiment of the present application;

[0043] Figure 6 It is a unit schematic diagram of the device provided by an embodiment of the present application.

[0044] In the figure: 10 - processor; 11 - memory; 12 - bus; 13 - communication interface; 201 - first processing unit; 202 - second processing unit. Detailed Embodiments

[0045] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the technical solutions in the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings in the embodiments of this application. Apparently, the described embodiments are some but not all of the embodiments of this application. The components of the embodiments of this application described and illustrated herein can be arranged and designed in various different configurations.

[0046] Therefore, the detailed description of the embodiments of this application provided in the accompanying drawings is not intended to limit the scope of the claimed application, but merely represents selected embodiments of this application. All other embodiments obtained by those of ordinary skill in the art based on the embodiments in this application without creative efforts fall within the scope of protection of this application.

[0047] It should be noted that similar reference numerals and letters denote similar items in the following drawings. Therefore, once an item is defined in one drawing, it does not require further definition and explanation in subsequent drawings. At the same time, in the description of this application, the terms "first", "second", etc. are only used for descriptive distinction and cannot be construed as indicating or implying relative importance.

[0048] It should be noted that, in this document, relational terms such as "first" and "second" are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any actual relationship or sequence between these entities or operations. Moreover, the term "comprising", "including" or any other variation thereof is intended to cover a non-exclusive inclusion, such that a process, method, article, or device comprising a series of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such process, method, article, or device. Without further limitation, an element defined by the phrase "comprising a..." does not exclude the presence of additional identical elements in the process, method, article, or device comprising the element.

[0049] In the description of this application, it should be noted that the orientation or positional relationship indicated by the terms "upper", "lower", "inner", "outer", etc. is based on the orientation or positional relationship shown in the drawings, or the orientation or positional relationship in which the product of this application is customarily placed during use. It is only for the convenience of describing this application and simplifying the description, and does not indicate or imply that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore should not be construed as a limitation of this application.

[0050] In the description of the present application, it should also be noted that unless otherwise clearly specified and defined, the terms "set" and "connect" should be understood in a broad sense. For example, it can be a fixed connection, a detachable connection, or an integral connection; it can be a mechanical connection or an electrical connection; it can be directly connected or indirectly connected through an intermediate medium, and it can be the communication inside two components. For those of ordinary skill in the art, the specific meanings of the above terms in the present application can be understood according to specific situations.

[0051] The following will describe in detail some embodiments of the present application with reference to the accompanying drawings. Without conflict, the following embodiments and the features in the embodiments can be combined with each other.

[0052] In order to complete the recognition of moving objects for the point cloud data collected by the lidar and achieve dynamic target segmentation (or classification), for example, in the transportation field, to classify moving objects such as people and vehicles on the road, an optional implementation method is as follows.

[0053] By performing a rotation operation on the point cloud data, calculating the ground equation based on the rotated point cloud after the rotation operation, using the ground equation to remove the ground point cloud, and calculating the coordinate points on the ground point cloud as targets for recognition. The algorithm of this method is relatively complex, with a large amount of calculation and low overall efficiency.

[0054] The point cloud is dense near and sparse far away, and there may be a failure situation at too far distances. If the above method is adopted and the same set of three-dimensional (3-Dimension, abbreviated as 3D) segmentation parameters are used, problems of under-segmentation and over-segmentation are likely to occur.

[0055] The embodiment of the present application provides a lidar point cloud data processing method, which can complete the recognition of moving objects based on the point cloud data collected by the lidar, and can overcome the problems brought by the above method. It does not require a rotation operation on the point cloud data, does not require an operation to remove the ground, reduces the coupling of the algorithm, improves the overall efficiency, and can solve the problems of over-segmentation and under-segmentation in the 3D state.

[0056] The embodiment of the present application provides an electronic device, which can be the central control system of a lidar, or a computer device communicatively connected to the lidar, such as a mobile phone, a vehicle computer, a server, etc. The electronic device can also be a lidar. Please refer to Figure 1 , the structural schematic diagram of the electronic device. The electronic device includes a processor 10, a memory 11, and a bus 12. The processor 10 and the memory 11 are connected through the bus 12, and the processor 10 is used to execute the executable module stored in the memory 11, such as a computer program.

[0057] The processor 10 can be an integrated circuit chip with signal processing capabilities. During the implementation process, each step of the lidar point cloud data processing method can be completed by the hardware integrated logic circuit in the processor 10 or the instructions in the form of software. The above-mentioned processor 10 can be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it can also be a digital signal processor (DSP), an application specific integrated circuit (ASIC), a field programmable gate array (FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components.

[0058] The memory 11 may include a high-speed random access memory (RAM), and may also include a non-volatile memory, such as at least one disk memory.

[0059] The bus 12 may be an ISA (Industry Standard Architecture) bus, a PCI (Peripheral Component Interconnect) bus, or an EISA (Extended Industry Standard Architecture) bus. Figure 1 Only one bidirectional arrow is used in the figure, but it does not mean that there is only one bus 12 or one type of bus 12.

[0060] Memory 11 is used to store programs, such as a program for a LiDAR point cloud data processing device. The LiDAR point cloud data processing device includes at least one software functional module, which can be stored in memory 11 in the form of software or firmware, or embedded in the operating system (OS) of the electronic device. Upon receiving an execution instruction, processor 10 executes the program to implement the LiDAR point cloud data processing method.

[0061] Optionally, the electronic device provided in the embodiment of the present application further includes a communication interface 13. The communication interface 13 is connected to the processor 10 via a bus.

[0062] It should be understood that Figure 1The structure shown is only a schematic diagram of a part of the electronic device. The electronic device may also include more or fewer components than those shown in Figure 1 or have a different configuration from that shown in Figure 1 . Figure 1 Each component shown in can be implemented by hardware, software, or a combination thereof.

[0063] A method for processing lidar point cloud data provided by an embodiment of the present application can be, but is not limited to, applied to the Figure 1 shown electronic device. For the specific process, please refer to Figure 2 . The lidar point cloud data processing method includes: S101, S102, S103, and S104, which are specifically described as follows.

[0064] S101. Based on the point cloud data of the target area transmitted by the lidar, construct a corresponding front view.

[0065] It should be noted that the lidar can be deployed on the monitoring platform and scan the target area at a preset frequency to obtain the point cloud data of the target area. This point cloud data is three-dimensional data.

[0066] During the process of processing the point cloud data, if the three-dimensional data is directly processed, there will be problems such as large computational complexity, low efficiency, and under-(over-)segmentation. In the solution of the present application, by constructing a front view, the point cloud data of the target area transmitted by the lidar is converted into a front view. This front view is a two-dimensional matrix, and the number of rows and columns in the two-dimensional matrix of the front view are both preset. For example, the front view is a 100-column × 100-row two-dimensional matrix.

[0067] In the initial two-dimensional matrix (template), the pixel value of each pixel point is 0, that is, the pixel value of the (a, b) pixel point is 0. The (a, b) pixel point represents the pixel point in the a-th row and b-th column of the two-dimensional matrix, where 1 ≤ a ≤ A and 1 ≤ b ≤ B. A represents the total number of rows of the two-dimensional matrix, and B represents the total number of columns of the two-dimensional matrix.

[0068] In the solution of the present application, constructing the front view corresponding to the point cloud data, that is, determining the pixel value of each pixel point in the front view, and adding the corresponding pixel value to the initial two-dimensional matrix to complete the construction of the front view.

[0069] It should be noted that the pixel value of the (a, b) pixel point being 0 indicates that there is no acquisition object at the position corresponding to the (a, b) pixel point in the target area, that is, the lidar does not collect an effective laser signal at this position.

[0070] S102. Based on the front view and the background map corresponding to the target area, determine the moving pixel points in the front view.

[0071] It should be noted that the front view and the background image have the same size and dimensions, and both are the acquisition results corresponding to the target area.

[0072] In the solution of this application, both the front view and the background image are two-dimensional matrices. In step S102, by performing calculations on a two-dimensional level, the moving pixel points in the front view can be determined without the need for point cloud rotation operations, greatly reducing the computational amount.

[0073] Among them, a moving pixel point means that the pixel value of this pixel point in the front view has changed relative to the pixel value of this pixel point in the background image.

[0074] S103, segment the moving pixel points in the front view, and classify the moving pixel points belonging to the same moving object to obtain a classification result.

[0075] Among them, the classification result includes the mapping relationship between each moving pixel point and the moving object to which it belongs.

[0076] It should be noted that when the lidar acquires the target area at a certain time point, there may be multiple moving objects of the same or different types in the target area, that is, the moving pixel points in the front view may belong to multiple different moving objects. The moving object can be but is not limited to cars, buses, sprinkler trucks, motorcycles, bicycles, pedestrians, and so on.

[0077] In order to distinguish different moving objects, it is necessary to segment the moving pixel points in the front view and classify the moving pixel points belonging to the same moving object to obtain a classification result.

[0078] S104, add labels to the coordinate points in the point cloud data corresponding to the moving pixel points based on the classification result.

[0079] It should be noted that there is a corresponding relationship between the moving pixel points in the front view and the coordinate points in the point cloud data, that is, one moving pixel point corresponds to one coordinate point in the point cloud data. When this moving pixel point belongs to a certain moving object, the coordinate point corresponding to this moving pixel point also belongs to this moving object, and thus the corresponding label can be added to this coordinate point. Among them, the label can be the identifier of the moving object to which the coordinate point belongs, and the labels of the coordinate points belonging to the same moving object are the same.

[0080] It should be noted that for the coordinate points in the point cloud data corresponding to the non-moving pixel points in the front view, also known as static coordinate points, a default label can be added or no label can be added.

[0081] Optionally, after the label addition in S104 is completed, the point cloud data carrying the labels can be output for the user to observe.

[0082] In the solution of this application, by converting three-dimensional point cloud data into a two-dimensional front view and performing calculations at the two-dimensional level, moving pixel points in the front view can be determined without the need for point cloud rotation operations, greatly reducing the amount of calculation. By segmenting the moving pixel points in the front view and classifying the moving pixel points belonging to the same moving object, different moving objects can be distinguished. Finally, by adding labels, it is convenient for users to observe.

[0083] It should be noted that the front view is the basis for identifying moving pixel points in this application. Whether the construction of the front view is accurate and reasonable directly affects the accuracy of moving object recognition in this application.

[0084] In Figure 2 On this basis, regarding how to quickly, accurately, and reasonably complete the construction of the front view, the embodiments of this application also provide an optional implementation manner. Please refer to the following. S101, the steps of constructing a corresponding front view based on the point cloud data of the target area transmitted by the lidar include: S101-1, S101-2, and S101-3, which are specifically described as follows.

[0085] S101-1, based on the coordinate information of the coordinate points in the point cloud data, determine the position information of the coordinate points in the front view.

[0086] Among them, the coordinate information of the coordinate points in the point cloud data is the data information in the coordinate system corresponding to the lidar.

[0087] As described above, the front view is a two-dimensional matrix, which includes multiple pixel points arranged in rows and columns, A×B pixel points. A represents the total number of rows of the two-dimensional matrix, and B represents the total number of columns of the two-dimensional matrix. To ensure the accuracy and reasonableness of the converted front view, it is necessary to keep the rules for constructing the front view consistent each time, that is, the mapping relationship between the coordinate points in the point cloud data and the pixel points in the front view remains consistent.

[0088] Therefore, it is necessary to determine the position information of the coordinate points in the front view (such as which row and which column) based on the position mapping relationship and the coordinate information of the coordinate points in the point cloud data, that is, the coordinate points are matched with the pixel points at this position information.

[0089] S101-2, based on the coordinate information of the coordinate points in the point cloud data, determine the distance value information between the coordinate points and the lidar.

[0090] Optionally, the distance value information can be the Euclidean distance value calculated based on the coordinate information of the coordinate point.

[0091] S101-3, fill the distance value information into the pixel value of the pixel point corresponding to the position information in the front view to complete the construction of the front view.

[0092] Please refer to Figure 3 , Figure 3 which is a measurement schematic diagram of a lidar provided by an embodiment of the present application. As Figure 3 shown, when there is no obstacle (such as a car) on the road surface, the laser signal can directly reach a certain point on the road surface. At this time, the measured distance is the distance from the lidar to this point on the road surface. When there is an obstacle (such as a car) on the road surface, the laser signal will be reflected back to the lidar via the obstacle. At this time, the measured distance is the distance between the lidar and the obstacle, and the corresponding distance value information changes. When the obstacle moves away, the distance value information will return. Therefore, the distance value information can be used as an observable quantity of the solution of the present application, and it is equivalently used as the pixel value of the corresponding pixel point, thereby completing the construction of the front view.

[0093] For pixel points that do not receive laser signals, that is, pixel points without distance values, their pixel values can be set to 0.

[0094] In an alternative embodiment, the position information includes the abscissa index in the front view and the ordinate index in the front view. Optionally, the abscissa index is also called the column index in the front view, and the ordinate index is also called the row index in the front view. The coordinate information of the coordinate point includes the X-axis coordinate (x), Y-axis coordinate (y), and Z-axis coordinate (z) in the Cartesian coordinate system of the lidar. On this basis, regarding how to accurately obtain the position information to ensure the reasonable construction of the front view, the embodiment of the present application also provides an alternative embodiment. Please refer to the following, S101-1, the steps of determining the position information of the coordinate point in the front view based on the coordinate information of the coordinate point in the point cloud data, including: S101-1A and S101-1B, which are specifically described as follows.

[0095] S101-1A, determining the abscissa index based on the X-axis coordinate and Y-axis coordinate of the coordinate point.

[0096] Optionally, the abscissa index is obtained by performing an arctangent calculation according to the formula atan2(y, x).

[0097] S101-1B, determining the ordinate index based on the X-axis coordinate, Y-axis coordinate, and Z-axis coordinate of the coordinate point.

[0098] Optionally, the ordinate index is calculated according to the formula atan2(z, sqrt(x × x + y × y)).

[0099] Optionally, given the horizontal and vertical resolutions of the known lidar, the total number of rows (A) of the two-dimensional matrix in the front view and the total number of columns (B) of the two-dimensional matrix can be deduced. The calculated vertical coordinate index can be verified based on A. When the vertical coordinate index is less than or equal to A, including the vertical coordinate index, otherwise it is deleted. The calculated horizontal coordinate index can be verified based on B. When the vertical coordinate index is less than or equal to B, including the horizontal coordinate index, otherwise it is deleted.

[0100] There are many pixel points in the front view constructed by the solution of this application. Whether the moving pixel points can be accurately identified has a great impact on the subsequent identification of moving objects. Figure 2 On this basis, regarding how to accurately identify the moving pixel points in the front view to ensure the accuracy of subsequent moving object identification, the embodiment of this application also provides an optional implementation manner. Please refer to the following, S102, the steps of determining the moving pixel points in the front view based on the front view and the background map corresponding to the target area, include: S102-1, which is specifically described as follows.

[0101] S102-1, determining whether the i-th pixel point in the front view is a moving pixel point based on the pixel value of the i-th pixel point in the front view and the pixel value of the i-th pixel point in the background map.

[0102] Among them, the position information corresponding to the i-th pixel point in the front view (including the horizontal coordinate index in the front view and the vertical coordinate index in the front view) is the same as the position information corresponding to the i-th pixel point in the background map.

[0103] It should be noted that the pixel value of the i-th pixel point represents the distance value information obtained by the lidar collecting the target object corresponding to the i-th pixel point. Therefore, it is possible to determine whether the i-th pixel point in the front view is a moving pixel point based on the pixel value of the i-th pixel point in the front view and the pixel value of the i-th pixel point in the background map.

[0104] Optionally, S102-1, the steps of determining whether the i-th pixel point in the front view is a moving pixel point based on the pixel value of the i-th pixel point in the front view and the pixel value of the i-th pixel point in the background map, include: S102-1A, S102-1B, and S102-1C, which are specifically described as follows.

[0105] S102-1A, determining whether the i-th pixel point satisfies any one of the first condition and the second condition. If so, execute S102-1B, if not, execute S102-1C.

[0106] Among them, the first condition is that the difference between the pixel value of the i-th pixel in the background image and the pixel value of the i-th pixel in the front view is greater than a first preset value. Specifically, the difference obtained by subtracting the pixel value of the i-th pixel in the front view from the pixel value of the i-th pixel in the background image is greater than the first preset value. The first preset value can be, but is not limited to, 3m. The second condition is that the pixel value of the i-th pixel in the front view is not 0, and the pixel value of the i-th pixel in the background image is 0.

[0107] Assume that the i-th pixel is the (a, b) pixel. The pixel value of the (a, b) pixel being 0 indicates that there is no acquisition object at the position corresponding to the (a, b) pixel in the target area, representing an infinite distance or that the lidar cannot effectively detect, that is, the lidar does not collect an effective laser signal at this position.

[0108] Optionally, the corresponding first preset value can be set according to the installation angle and height of the lidar.

[0109] S102-1B, when the i-th pixel satisfies any one of the first condition and the second condition, determine the i-th pixel in the front view as a moving pixel.

[0110] Based on this, all moving pixels in the front view can be accurately identified.

[0111] S102-1C, when neither the first condition nor the second condition is satisfied, determine that the i-th pixel in the front view is not a moving pixel, that is, a non-moving pixel.

[0112] In the process of determining moving pixels in the solution of this application, the pixel values of each pixel in the background image need to be referred to. The accuracy of the information in the background image directly affects the recognition result of moving pixels. On this basis, regarding how to obtain the initial background image to ensure its accuracy, the embodiments of this application also provide an optional implementation manner. Please refer to the following. The lidar point cloud data processing method further includes: after the electronic device is started, directly using the acquired first frame of the front view as the background image, or assigning the values of each pixel in the first frame of the front view to the background image.

[0113] In Figure 2 On the basis of, regarding how to update the background image to ensure the accuracy of the recognition result of moving pixels, the embodiments of this application also provide an optional implementation manner. Please refer to Figure 4 , after S101 constructs the front view, the lidar point cloud data processing method further includes: S105 and S106, which are specifically described as follows.

[0114] S105. When the i-th pixel point in the front view satisfies the third condition, determine whether the number of consecutive times the i-th pixel point satisfies the third condition is greater than a second preset number. If so, execute S106; if not, execute other steps or skip.

[0115] Wherein, the third condition is that the difference between the pixel value of the i-th pixel point in the front view and the pixel value of the i-th pixel point in the background image is greater than a first preset value.

[0116] Optionally, set the second preset number according to the sampling frequency of the lidar and the motion state (such as speed) of the moving object to be monitored.

[0117] For example, when there starts to be an obstacle (such as a car) on the road surface, the pixel value of the i-th pixel point in the background image is 50. After the car moves away, the pixel value of the i-th pixel point in the front view is 53.2, which satisfies that the difference between the pixel value of the i-th pixel point in the front view and the pixel value of the i-th pixel point in the background image is greater than the first preset value. When the number of consecutive times the i-th pixel point satisfies the third condition is greater than the second preset number, it indicates that the car has moved away. At this time, it is necessary to update the pixel value of the i-th pixel point in the background image and execute S106. Otherwise, it is not necessary and other steps can be executed or skipped.

[0118] S106. If it is greater than the second preset number, assign the pixel value of the i-th pixel point in the latest front view to the pixel value of the i-th pixel point in the background image.

[0119] Based on Figure 2 above, regarding how to update the background image to ensure the accuracy of the recognition result of moving pixel points, the embodiments of the present application further provide an optional implementation manner. Please refer to Figure 5 . After S101 constructs the front view, the lidar point cloud data processing method further includes: S107 and S108, which are specifically described as follows.

[0120] S107. When the pixel value of the i-th pixel point in the front view is 0, determine whether the number of consecutive times the pixel value of the i-th pixel point is 0 is greater than a third preset number. If so, execute S108; if not, execute other steps or skip.

[0121] Optionally, set the third preset number according to the sampling frequency of the lidar and the motion state (such as speed) of the moving object to be monitored.

[0122] It should be noted that there is no point cloud on the road surface in the distance. If there is a point cloud, it can only be an obstacle such as a car. After the obstacle moves away, the pixel value of the i-th pixel point in the front view becomes 0. If the pixel value of the i-th pixel point in the background image is not 0 at this time, then S108 needs to be executed; otherwise, it is not necessary and other steps can be executed or skipped.

[0123] S108, if it is greater than the third preset number of times, then assign the pixel value of the i-th pixel point in the background image to 0.

[0124] In an alternative embodiment, the steps of Figure 4 and Figure 5 may be executed simultaneously. For example, first execute S105. If the execution result of S105 is yes, then execute S106; if the execution result of S105 is no, then execute S107. Of course, it is also possible to first execute S107, and when the execution result of S107 is no, execute S105.

[0125] It should be noted that if it has been determined that the i-th pixel point is a moving pixel point, then it is not necessary to update the pixel value of the i-th pixel point in the background image.

[0126] Based on Figure 2 , the embodiments of the present application further provide an alternative segmentation method to ensure the accuracy of the classification result. Please refer to the following. The steps of S103, segmenting the moving pixel points in the front view and classifying the moving pixel points belonging to the same moving object to obtain the classification result include: S103-1, which is specifically described as follows.

[0127] S103-1, coarsely segment the moving pixel points in the front view by using a 2D region growing algorithm to obtain a preliminary classification result.

[0128] Among them, 2D is the abbreviation of two-dimensional. The 2D region growing algorithm is an image segmentation method based on pixel adjacency relationships. Its basic idea is to start from a seed point and gradually expand according to certain growth criteria, adding the surrounding qualified pixel points to the same region until all qualified pixel points are added to the region.

[0129] Optionally, the steps of S103-1, coarsely segmenting the moving pixel points in the front view by using a 2D region growing algorithm to obtain a preliminary classification result include: S103-1A, S103-1B, S103-1C, and S103-1D, which are specifically described as follows.

[0130] S103-1A, select any one of the unclassified moving pixel points in the front view as the seed point.

[0131] Optionally, randomly select any one of the unclassified moving pixel points in the front view as the seed point, and then execute the 2D region growing algorithm based on this seed point to complete the coarse segmentation.

[0132] S103-1B, Determine whether there are adjacent moving pixel points for the seed points. If there are, execute S103-1C; if there are no adjacent moving pixel points for all seed points, end.

[0133] Among them, the adjacent moving pixel points are the moving pixel points whose distance from the seed points is less than the fourth distance threshold and have not been marked.

[0134] Optionally, the fourth distance threshold can also be understood as the neighborhood radius, and its value can be, but is not limited to, 2.

[0135] Optionally, screen the adjacent moving pixel points of the seed points by traversing. If there are adjacent moving pixel points for the seed points, S103-1C can be executed to mark the adjacent moving pixel points as belonging to the same moving object as the seed points. If there are no adjacent moving pixel points for all seed points, the rough segmentation is completed and ended.

[0136] S103-1C, Mark the adjacent moving pixel points as belonging to the same moving object as the seed points.

[0137] S103-1D, Use the adjacent moving pixel points as new seed points.

[0138] After S103-1C, execute S103-1D. At this time, multiple new seed points may be obtained. For each seed point, repeat S103-1B to determine whether there are adjacent moving pixel points for the seed points until there are no adjacent moving pixel points for all seed points. At this time, all the moving pixel points belonging to the same moving object as the initial seed points in the initial S103-1A can be screened out, and the rough segmentation is completed.

[0139] It should be noted that the rough segmentation is a segmentation method based on moving pixel points. The basis is to judge whether the pixels in the front view have values. Using the region growing method, it can better segment in the two dimensions of the Y-axis and the Z-axis, but the segmentation ability for the X-axis is poor.

[0140] To improve the problem that the rough segmentation has poor segmentation ability for the X-axis and ensure the accuracy of the segmentation result, the embodiments of the present application also provide an optional implementation manner. Please refer to the following. After S103-1, in S103, for the steps of segmenting the moving pixel points in the front view and classifying the moving pixel points belonging to the same moving object to obtain the classification result, it further includes: S103-2 and S103-3, which are specifically described as follows.

[0141] S103-2, Restore the moving pixel points in the preliminary classification result to the point cloud view.

[0142] Among them, the coordinate points after conversion of the motion pixel points belonging to the same moving object belong to the same cluster of point clouds.

[0143] S103-3. Perform 3D clustering operations on each cluster of point clouds in the point cloud view, classify the coordinate points belonging to different moving objects in the same cluster of point clouds, so as to determine the mapping relationship between each motion pixel point and the moving object it belongs to, and obtain the final classification result.

[0144] It should be noted that because the rough segmentation has poor segmentation ability for the X-axis, two or more different moving objects may be classified as the same moving object. To improve this, the motion pixel points in the preliminary classification result are restored to a three-dimensional point cloud view. Then, based on the three-dimensional point cloud view, perform 3D clustering operations on each cluster of point clouds in the point cloud view to determine the moving object to which the coordinate points in each cluster of point clouds belong, that is, determine the mapping relationship between each motion pixel point and the moving object it belongs to, and obtain the final accurate classification result.

[0145] To improve the problem that the rough segmentation has poor segmentation ability for the X-axis and ensure the accuracy of the segmentation result, the embodiment of the present application also provides an optional implementation manner. Please refer to the following. After S103-1, in S103, the step of segmenting the motion pixel points in the front view and classifying the motion pixel points belonging to the same moving object to obtain the classification result further includes: S103-4, which is specifically described as follows.

[0146] S103-4. Perform the 2D watershed algorithm on the motion pixel points in the preliminary classification result, and further classify the motion pixel points determined by the rough segmentation to belong to the same moving object, so as to determine the mapping relationship between each motion pixel point and the moving object it belongs to, and obtain the final classification result.

[0147] Please refer to Figure 6 , Figure 6 A device provided by an embodiment of the present application. Optionally, the device is applied to the electronic device described above. The device includes: a first processing unit 201 and a second processing unit 202.

[0148] The first processing unit 201 is used to construct a corresponding front view based on the point cloud data of the target area transmitted by the lidar;

[0149] The first processing unit 201 is further used to determine the motion pixel points in the front view based on the front view and the background map corresponding to the target area;

[0150] The first processing unit 201 is further configured to segment the moving pixel points in the front view and classify the moving pixel points belonging to the same moving object to obtain a classification result, where the classification result includes the mapping relationship between each moving pixel point and the moving object to which it belongs;

[0151] The second processing unit 202 is configured to add labels to the coordinate points corresponding to the moving pixel points in the point cloud data based on the classification result.

[0152] Optionally, the first processing unit 201 may execute the above S101 - S103 and S105 - S108, and the second processing unit 202 may execute the above S104.

[0153] It should be noted that the device provided in this embodiment can execute the method flow shown in the above method flow embodiment to achieve the corresponding technical effects. For a brief description, for the parts not mentioned in this embodiment, reference may be made to the corresponding content in the above embodiments.

[0154] Optionally, the first processing unit 201 and the second processing unit 202 may be separately provided or integrated into one unit, that is, the processing unit. The present application does not make specific limitations on the specific implementation manners of the first processing unit 201 and the second processing unit 202.

[0155] Optionally, Figure 6 the shown device may further include a storage unit ( Figure 6 not shown in the figure), and this storage unit stores programs or instructions. When the first processing unit 201 and the second processing unit 202 execute this program or instruction, it makes Figure 6 the shown device can execute any possible implementation manner in the method embodiment of the present application.

[0156] Figure 6 The shown device may be a lidar or the central control system of a lidar, or a computer device communicatively connected to the lidar, such as a mobile phone, a vehicle computer, and a server, etc. It may also be a chip (system) or other components or assemblies that can be set in the lidar (or the central control system of the lidar, or a computer device communicatively connected to the lidar). The present application does not make limitations on this.

[0157] In addition, Figure 6 the technical effects of the shown device may be the technical effects of the method shown in the above method embodiment, which will not be elaborated here.

[0158] The embodiment of the present application further provides a storage medium, which stores computer instructions and programs. When the computer instructions and programs are read and run, they execute the lidar point cloud data processing method in the above embodiment. The storage medium may include memory, flash memory, registers, or a combination thereof, etc.

[0159] The following provides an electronic device, which can be the central control system of a lidar, or a computer device communicatively connected to the lidar, such as a mobile phone, a vehicle computer, a server, etc. The electronic device can also be a lidar. As shown in the electronic device Figure 1 it can implement the above-mentioned lidar point cloud data processing method; specifically, the electronic device includes: a processor 10, a memory 11, and a bus 12. The processor 10 can be a CPU. The memory 11 is used to store one or more programs. When the one or more programs are executed by the processor 10, the lidar point cloud data processing method of the above-mentioned embodiment is executed.

[0160] The embodiment of the present application provides a computer program product, which includes: a computer program (which can also be called code or instruction). When the computer program is run, it causes the computer to execute the method described in any possible implementation manner of the method embodiment of the present application.

[0161] The embodiment of the present application provides a chip system, which includes: a processor and a memory. The memory is used to store one or more programs; when the one or more programs are executed by the processor, the method described in any possible implementation manner of the method embodiment of the present application is implemented.

[0162] In summary, the lidar point cloud data processing method, device, storage medium, and electronic device provided by the embodiment of the present application are based on the point cloud data of the target area transmitted by the lidar to construct a corresponding front view; based on the front view and the background map corresponding to the target area, determine the moving pixel points in the front view; segment the moving pixel points in the front view, and classify the moving pixel points belonging to the same moving object to obtain a classification result, where the classification result includes the mapping relationship between each moving pixel point and the moving object to which it belongs; add labels to the coordinate points corresponding to the moving pixel points in the point cloud data based on the classification result. By converting the three-dimensional point cloud data into a two-dimensional front view and performing calculations at the two-dimensional level, the moving pixel points in the front view can be determined without performing point cloud rotation operations, greatly reducing the amount of calculation. By segmenting the moving pixel points in the front view and classifying the moving pixel points belonging to the same moving object, different moving objects can be distinguished. Finally, by adding labels, it is convenient for users to observe.

[0163] The above are only the preferred embodiments of the present application and are not used to limit the present application. For those skilled in the art, the present application can have various changes and modifications. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principle of the present application shall be included in the protection scope of the present application.

[0164] It is obvious to those skilled in the art that the present application is not limited to the details of the above-described exemplary embodiments, and that the present application can be implemented in other specific forms without departing from the spirit or essential characteristics of the present application. Therefore, from any point of view, the embodiments should be regarded as exemplary and non-limiting. The scope of the present application is defined by the appended claims rather than the above description. Therefore, all changes falling within the meaning and scope of the equivalent elements of the claims are intended to be embraced within the present application. Any reference signs in the claims should not be construed as limiting the claims concerned.

Claims

1. A method for processing lidar point cloud data, characterized in that, The method includes: Based on the point cloud data of the target area transmitted by the lidar, construct a corresponding front view; Based on the front view and the background map corresponding to the target area, determine the moving pixel points in the front view; Segment the moving pixel points in the front view, classify the moving pixel points belonging to the same moving object to obtain a classification result, where the classification result includes the mapping relationship between each moving pixel point and the moving object to which it belongs; Add labels to the coordinate points corresponding to the moving pixel points in the point cloud data based on the classification result; The step of constructing a corresponding front view based on the point cloud data of the target area transmitted by the lidar includes: determining the position information of the coordinate point in the front view based on the coordinate information of the coordinate points in the point cloud data; determining the distance value information between the coordinate point and the lidar based on the coordinate information of the coordinate points in the point cloud data; filling the distance value information as the pixel value of the pixel point corresponding to the position information in the front view to complete the construction of the front view; The step of segmenting the moving pixel points in the front view, classifying the moving pixel points belonging to the same moving object to obtain a classification result includes: coarsely segmenting the moving pixel points in the front view using a 2D region growing algorithm to obtain a preliminary classification result; restoring the moving pixel points in the preliminary classification result to a point cloud view, where the coordinate points after conversion of the moving pixel points belonging to the same moving object belong to the same cluster of point clouds; performing 3D clustering operations on each cluster of point clouds in the point cloud view, classifying the coordinate points belonging to different moving objects in the same cluster of point clouds to determine the mapping relationship between each moving pixel point and the moving object to which it belongs, and obtaining the final classification result.

2. The method for processing lidar point cloud data according to claim 1, wherein The position information includes the abscissa index in the front view and the ordinate index in the front view, and the coordinate information of the coordinate point includes the X-axis coordinate, Y-axis coordinate, and Z-axis coordinate in the Cartesian coordinate system of the lidar; The step of determining the position information of the coordinate point in the front view based on the coordinate information of the coordinate points in the point cloud data includes: Determining the abscissa index based on the X-axis coordinate and the Y-axis coordinate of the coordinate point; Determining the ordinate index based on the X-axis coordinate, the Y-axis coordinate, and the Z-axis coordinate of the coordinate point.

3. The method for processing lidar point cloud data according to claim 1, wherein The step of determining the moving pixel points in the front view based on the front view and the background map corresponding to the target area includes: Determining whether the i-th pixel point in the front view is the moving pixel point based on the pixel value of the i-th pixel point in the front view and the pixel value of the i-th pixel point in the background map.

4. The method for processing lidar point cloud data according to claim 3, wherein The step of determining whether the i-th pixel point in the front view is the moving pixel point based on the pixel value of the i-th pixel point in the front view and the pixel value of the i-th pixel point in the background map includes: Determining whether the i-th pixel point satisfies any one of the first condition and the second condition; When the i-th pixel point in the front view satisfies any one of the first condition and the second condition, determine the i-th pixel point in the front view as the moving pixel point; Wherein, the first condition is that the difference between the pixel value of the i-th pixel point in the background image and the pixel value of the i-th pixel point in the front view is greater than a first preset value, and the second condition is that the pixel value of the i-th pixel point in the front view is not 0, and the pixel value of the i-th pixel point in the background image is 0.

5. The method for processing lidar point cloud data according to claim 1, characterized in that The method further includes: When the i-th pixel point in the front view satisfies the third condition, determine whether the number of times the i-th pixel point continuously satisfies the third condition is greater than a second preset number of times; Wherein, the third condition is that the difference between the pixel value of the i-th pixel point in the front view and the pixel value of the i-th pixel point in the background image is greater than a first preset value; If it is greater than the second preset number of times, assign the pixel value of the i-th pixel point in the latest front view to the pixel value of the i-th pixel point in the background image.

6. The method for processing lidar point cloud data according to claim 1, characterized in that, The method further includes: When the pixel value of the i-th pixel point in the front view is 0, determine whether the number of consecutive times the pixel value of the i-th pixel point is 0 is greater than a third preset number of times; If it is greater than the third preset number of times, assign the pixel value of the i-th pixel point in the background image to 0.

7. The method for processing lidar point cloud data according to claim 1, wherein The step of using the 2D region growing algorithm to roughly segment the moving pixel points in the front view to obtain a preliminary classification result includes: Select any one of the unclassified moving pixel points in the front view as a seed point; Determine whether there are adjacent moving pixel points for the seed point, where the adjacent moving pixel points are moving pixel points whose distance from the seed point is less than a fourth distance threshold; If so, mark the adjacent moving pixel points as belonging to the same moving object as the seed point; Take the adjacent moving pixel points as new seed points, and repeat to determine whether there are adjacent moving pixel points for the seed point until there are no adjacent moving pixel points for all seed points.

8. The method for processing lidar point cloud data according to claim 1, wherein The step of segmenting the moving pixel points in the front view, classifying the moving pixel points belonging to the same moving object to obtain a classification result further includes: Perform a 2D watershed algorithm on the moving pixel points in the preliminary classification result, and further classify the moving pixel points determined to belong to the same moving object by rough segmentation to determine the mapping relationship between each moving pixel point and the moving object it belongs to, and obtain a final classification result.

9. A lidar point cloud data processing device, characterized in that The device includes: A first processing unit, configured to construct a corresponding front view based on the point cloud data of the target area transmitted by the lidar, including: determining the position information of the coordinate point in the front view based on the coordinate information of the coordinate point in the point cloud data; determining the distance value information between the coordinate point and the lidar based on the coordinate information of the coordinate point in the point cloud data; filling the distance value information into the pixel value of the pixel point corresponding to the position information in the front view to complete the construction of the front view; The first processing unit is further configured to determine moving pixel points in the front view based on the front view and the background image corresponding to the target area; The first processing unit is further configured to segment the moving pixel points in the front view, classify the moving pixel points belonging to the same moving object, so as to obtain a classification result, where the classification result includes the mapping relationship between each moving pixel point and the moving object to which it belongs; A second processing unit, configured to add labels to the coordinate points corresponding to the moving pixel points in the point cloud data based on the classification result; The segmenting the moving pixel points in the front view and classifying the moving pixel points belonging to the same moving object to obtain a classification result includes: coarsely segmenting the moving pixel points in the front view by using a 2D region growing algorithm to obtain a preliminary classification result; restoring the moving pixel points in the preliminary classification result to a point cloud view, where the coordinate points after conversion of the moving pixel points belonging to the same moving object belong to the same cluster of point clouds; performing 3D clustering operations on each cluster of point clouds in the point cloud view, classifying the coordinate points belonging to different moving objects in the same cluster of point clouds, so as to determine the mapping relationship between each moving pixel point and the moving object to which it belongs, and obtaining a final classification result.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, the method according to any one of claims 1-8 is implemented.

11. An electronic device, characterized in that, Including: A processor and a memory, where the memory is used to store one or more programs; When the one or more programs are executed by the processor, the method according to any one of claims 1-8 is implemented.

Images